diff --git a/CHANGELOG.md b/CHANGELOG.md --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,1005 @@ # CHANGELOG for crypton +## 2.0.0 + +* fix(docs): export the names the documentation already referred to. + `Crypto.Number.ModArithmetic` throws `CoprimesAssertionError` and + `ModulusAssertionError` from `inverseCoprimes` and `squareRoot`, and said so + in the haddock, without exporting either, so a caller could not name the + exception it was told to expect; `Crypto.PubKey.Rabin.Types.generatePrimes` + takes a `PrimeCondition` in its exported signature and that synonym was not + exported either. All three are now exported + [#195](https://github.com/kazu-yamamoto/crypton/pull/195) + +* Breaking change: fix(bcrypt): refuse a cost bcrypt does not have rather than + substituting one. A cost below 4 came back as a cost-10 hash and a cost + above 31 as a cost-31 one, with nothing said either way, so a caller asking + for something bcrypt does not do was answered with something else and had no + way to tell -- `hashPassword 3` and `hashPassword 10` returned the same + thing. Both ends are now reported as `CryptoError_ParameterInvalid`, which + is what every other KDF here already did for a refused parameter. + `hashPassword` can therefore fail where it could not before, so + `tryHashPassword` is added beside it; the salt it generates is always the + right length, so the cost is the only thing it can report + [#59](https://github.com/kazu-yamamoto/crypton/issues/59) + +* Breaking change: fix(poly1305): take a checked key, so that initializing + cannot fail. A Poly1305 key is thirty-two bytes and nothing else about it + can be wrong, so `initialize` returning a `CryptoFailable` put an error case + in front of every caller for a length most of them know is right -- and they + answered it with `throwCryptoError`, this library included: the one in + `Crypto.Cipher.ChaChaPoly1305` guarded a `B.take 32`, and `auth` did not + even do that, it called `error`. There is now a `Key` with `key` to build + one, the length is checked there, and `initialize :: Key -> State` and + `auth :: Key -> ba -> Auth` are total. A caller checks once and then + initializes as often as it likes with nothing to handle. `initialize k` + becomes `initialize <$> key k` where the length is unknown, and where it is + known the check moves to where the key is made. `Key` has no `Show`, as + key material should not + [#28](https://github.com/kazu-yamamoto/crypton/issues/28) + +* fix(pubkey): stop printing private keys. `Show` is what `print`, a message + built with `error`, an exception and a test framework's failure output all + reach for, so it is the instance a key travels on when nobody meant to send + it anywhere; the library already kept that promise for the secret keys held + in a `ScrubbedBytes`, and the documentation for `ScrubbedBytes` advertises + it, while ten other types printed theirs in full. Those ten -- + `RSA.PrivateKey` and `RSA.KeyPair`, `DSA.PrivateKey` and `DSA.KeyPair`, + `ECDSA.PrivateKey` and `ECDSA.KeyPair`, `DH.PrivateNumber`, and the + `PrivateKey` of `Rabin.Basic`, `Rabin.Modified` and `Rabin.RW` -- now render + the public part and `` for the rest. Nothing else about them + changes: `Read`, `Eq`, `Data`, `Generic` and `NFData` are all still derived. + The new `Crypto.Debug` exports a class `DebugShow` whose `debugShow` returns + exactly what the derived `Show` used to return, for those ten and for the + five `ScrubbedBytes` secret keys as well, which never had a `Show` that + spoke. **Code that serialized a key through `show` has to say `debugShow` + instead**; `read (debugShow k) == k` still holds, but `read` given the + output of `show` will now fail at run time, which the compiler cannot point + at + [#72](https://github.com/kazu-yamamoto/crypton/issues/72) + +* deprecate(ecc): the curves over a binary field. They are obsolete, they are + the curves a cofactor makes delicate, and pyca/cryptography deprecated them + for removal in the release that fixed CVE-2026-26007. A `DEPRECATED` pragma + now covers the eighteen `SEC_t*` constructors of `CurveName` and the + eighteen types of the same names in `Crypto.ECC.Simple.Types`; nothing is + removed, so the only effect is a warning where one of them is named, and + they will go in a later major version. The prime curves with a cofactor, + `SEC_p112r2` and `SEC_p128r2`, are not deprecated: the check above covers + them + [#66](https://github.com/kazu-yamamoto/crypton/issues/66) + +* fix(ecc): refuse a public point outside the prime-order subgroup. A point + that satisfies the curve equation is not necessarily in the subgroup the + base point generates; the two coincide only where the cofactor is 1. Of the + curves in `CurveName` twenty have a cofactor -- the eighteen binary ones, + and `SEC_p112r2` and `SEC_p128r2`, whose cofactor is 4 -- and on those the + other party could offer a point of small order, at which point the value + that came back depended on our private number only through its residue + modulo that order, and offering it and watching the answer handed them those + bits. This is the flaw pyca/cryptography fixed as CVE-2026-26007; what is + fixed here is the same one, found by following that report. + `Crypto.PubKey.ECC.DH.getShared` and `tryGetShared`, and + `Crypto.ECC.Simple.Prim.pointFromIntegers`, now require the point to be in + the subgroup and report `CryptoError_PointSubgroupInvalid` when it is not. + The check is `isPointInSubgroup`, newly exported from both prim modules: + where the cofactor is 1 it answers without work, and otherwise it multiplies + by the group order and requires the point at infinity, which is what + OpenSSL's `EC_KEY_check_key` does and costs one further scalar + multiplication -- an exchange on an affected curve is about twice the price, + and one on every other curve is unchanged. The typed `Crypto.ECC` interface + offers only cofactor-1 curves and dedicated implementations, so nothing + reaching elliptic curves through it, `tls` among them, was affected + +* perf(p256): inline the field arithmetic on AArch64. `felem_mul` and + `felem_square` end in `felem_reduce_degree`, a carry chain the whole width + of the number, and that chain is what their latency is: one product feeding + the next costs 18.1 ns on an Apple M4, while four independent ones cost 11.4 + ns each. The curve arithmetic has independent products to offer -- the two + squarings that open a point doubling, the multiplication and the squaring + that close it -- but only if the compiler inlines the reduction instead of + calling it, since a call is a fence. Plain `inline` does not change its + mind; `always_inline` does, and it is worth asking where there are registers + to hold two carry chains at once and not where there are not: 1.23x on an + M4, 1.12x and 1.05x on a Neoverse under clang and gcc, and 0.95x and 0.82x + on x86-64, whose fifteen general-purpose registers are not enough. So it is + gated on the architecture and x86-64 is left byte-identical. ECDH P-256 on + an M4: 69.16 to 56.24 us, against openssl's 24.68, so 0.36 becomes 0.44; + ECDSA P-256 signing and verification move with it. The cost is code, 30 to + 116 kilobytes of it + [#188](https://github.com/kazu-yamamoto/crypton/pull/188) +* perf(number): count bytes from the bit count, not from base 256. `numBytes` + asked GMP how many base-256 digits a number has, and GHC's bignum answers + that by dividing the number down to nothing, one digit at a time, where the + same question in base two is a look at the highest limb. On a 2048-bit + `Integer` that is 1.65 us against 0.01. Every serialization here asks for + the size before it allocates and `i2ospOf` asks twice, so the cost landed on + every RSA, DSA and DH operation leaving the `Integer` world: `i2ospOf_` at + 256 bytes goes from 2.87 to 0.09 us and RSA-2048 verification from 18.4 to + 15.5 us on an M4. Signing moves by a percent; it is two exponentiations and + hardly touches this + [#187](https://github.com/kazu-yamamoto/crypton/pull/187) +* perf(ecc): stop sharing the doublings in the double multiplication. + `pointAddTwoMuls` was Shamir's trick, one pass over the bits of both scalars + at once in `Integer` arithmetic, which is the right trade when the two + multiplications would cost the same. They have not for a while: `pointMul` + goes to C, and over a prime field it multiplies the base point through a + table of its multiples at about a third of the price -- and the base point + is one of the two, since ECDSA verification is the only caller. Doing them + separately and adding: ECDSA P-384 verification 1397 to 698 us on an M4 in + the typed API, 9839 to 738 in the older one, and a curve over a binary field + 641 ms to 2.6. P-256 keeps the double multiplication it has in C + [#186](https://github.com/kazu-yamamoto/crypton/pull/186) +* perf(sha3): take the CRYPTOGAMS Keccak for x86-64 as well. The same module + as [#181](https://github.com/kazu-yamamoto/crypton/pull/181) on the other + architecture, and the reason it was not taken at the time was a measurement + taken on the wrong machine: crypton on Apple silicon against openssl on + x86-64, which said there was nothing to gain. Measured on one machine there + was: SHA3-256 on an EPYC 7763 goes from 109 to 421 MB/s, against openssl's + 426, so 0.26 becomes 0.99 + [#184](https://github.com/kazu-yamamoto/crypton/pull/184) +* perf(sha1): take the CRYPTOGAMS SHA-1 for AArch64. The instructions are the + ones [#170](https://github.com/kazu-yamamoto/crypton/pull/170) put in, and + the arrangement is what the module has over them: the message schedule of + the next four rounds runs against the rounds of this one, which is not + something a C function is going to be made to do -- + [#179](https://github.com/kazu-yamamoto/crypton/pull/179) tried the one + thing C can do here, handing over a run of blocks, and on this processor it + measured nothing at all. The entry point for processors that have the + instructions is not exported, so the module's own dispatch picks it and the + answer to the runtime question goes into the word that dispatch reads. On + Apple silicon: 3155 to 3379 MB/s at 16 KiB, against openssl's 3350 on the + same machine, so where this was at 0.94 it is now a shade ahead. The + intrinsics stay for the block a message ends with, and for any processor + that has the instructions but is built without the assembly + [#182](https://github.com/kazu-yamamoto/crypton/pull/182) +* perf(sha3): take the CRYPTOGAMS Keccak for AArch64. The instructions are + the ones [#171](https://github.com/kazu-yamamoto/crypton/pull/171) put in -- + EOR3, RAX1, XAR and BCAX -- and what this module does with them is take a + run of blocks rather than one at a time, and schedule the round it is in + against the next one. The absorb loop hands over the whole run, which also + drops the alignment trampoline on that path: the assembly reads the message + as bytes. SHA3-256 on Apple silicon: 1002 to 1104 MB/s at 16 KiB, against + openssl's 1058 on the same machine. Only the absorb side is handed over; + the squeeze, which SHAKE uses to produce output, is entangled with this + side's buffer bookkeeping and is not where the time goes + [#181](https://github.com/kazu-yamamoto/crypton/pull/181) +* perf(xts): double the XTS tweak in the integer registers. The tweak + advances by doubling in GF(2^128) once per block, and it was doing that in a + vector register: six operations on the same units that are running the + rounds and the exclusive ors, in a chain where each waits for the one + before. On a processor whose AES is fast that is not a detail -- taking the + doubling out of a diagnostic build, which gives the wrong answer but says + where the time goes, left XTS running at the speed of ECB. It costs three + integer operations instead, and the integer units have nothing else to do + here; what crosses over is one move per block. On x86-64 that also gets + eight values out of a register file with sixteen entries, so the round keys + stay where they were. AES-128-XTS at 16 KiB: 9644 to 18646 MB/s on Apple + silicon and 4504 to 7660 on a Haswell-generation x86-64, against openssl's + 17382 and 6997 on the same machines, so both are now a little ahead where + they were at 0.55 and 0.64. No assembly: the AArch64 module in CRYPTOGAMS + has no XTS, and the x86-64 one's is inside a module this does not otherwise + want + [#180](https://github.com/kazu-yamamoto/crypton/pull/180) +* perf(sha1): hand the SHA-1 block loop a run of blocks rather than one at a + time. A block at a time means the state goes out to memory and comes back + either side of every block, with the two shuffles that put it in the order + the instructions want; against the hundred-odd cycles a block costs with the + SHA extensions that is most of what stood between this and openssl. On an + EPYC 7763: 1364 to 1677 MB/s, against openssl's 1670 on the same machine, + and on a Xeon 8370C 1506 to 1619. On Apple silicon it measures nothing at + all -- that processor hides the cost -- and is kept there only so the two + paths have one shape. The intended file for this was CRYPTOGAMS' + `sha1-x86_64.pl`, which turns out to be the 2006 scalar implementation: no + SSSE3, no AVX, no SHA extensions. Processors without the extensions are + therefore where they were, 664 MB/s against openssl's 791 on a Haswell + [#179](https://github.com/kazu-yamamoto/crypton/pull/179) +* perf(sha2): take the CRYPTOGAMS SHA-256 and SHA-512 for x86-64. One + generator gives both, as on AArch64, and each dispatches on what the + processor has: the SHA extensions, AVX2, AVX, SSSE3 or plain integer code. + That replaces everything written here for x86-64 -- `sha256_x86.c` and + `sha512_x86.c` go -- since it is ahead of all of it either way. At 16 KiB: + on an EPYC 7763, SHA-256 1430 to 1584 MB/s and SHA-512 423 to 769; on a + Haswell-generation part, SHA-256 318 to 379 and SHA-512 488 to 593, where + openssl reports 378 and 589. The block loops hand over the whole run of + blocks rather than one at a time, and the alignment trampoline goes with it. + This also fixes a bug in the capability word + [#176](https://github.com/kazu-yamamoto/crypton/pull/176) added: bit 29 of + leaf 7 EBX is the SHA extensions, not an AVX-512 bit, and was being cleared + along with them -- which cost the SHA-256 assembly two thirds of its speed + on a processor that has them, and which no machine here could have shown, + since none has them + [#178](https://github.com/kazu-yamamoto/crypton/pull/178) +* perf(chacha): take the CRYPTOGAMS ChaCha20 for x86-64 as well. The C here + vectorises from eight blocks up and takes anything shorter one block at a + time, so a message of a few hundred bytes -- a QUIC packet, a small TLS + record -- ran at a fifth of the bulk rate. The module has vector code for + those lengths and is a few per cent ahead in bulk besides: 494 to 1091 MB/s + at 256 bytes, 477 to 701 at 128, and 2201 to 2374 at 16 KiB, which is + openssl's 2389 on the same machine. It is handed everything from one block + up, where the AArch64 module is handed nothing below three, that one's + scalar path measuring level with the C. Keystream generation is still the + C on both, having no input to exclusive-or + [#177](https://github.com/kazu-yamamoto/crypton/pull/177) +* perf(poly1305): take the CRYPTOGAMS Poly1305 for x86-64 as well. The same + module for the other architecture, through the same three functions, so what + this adds is the capability word: where the AArch64 one reads + `crypton_armcap_P`, this one reads `crypton_ia32cap_P`, which is cpuid's own + words in the order OpenSSL keeps them, filled with the bits for anything the + operating system will not preserve cleared. What it brings over the AVX2 + written here is a hand-scheduled scalar path, which is what a message of a + few hundred bytes actually uses, and an AVX path for machines with no AVX2: + on a Haswell-generation x86-64, 4298 to 5345 MB/s at 16 KiB and 556 to 1573 + at 64 bytes, against the roughly 5270 openssl reaches there. `poly1305_avx2.c` + goes the way the NEON did. The module's AVX-512 paths are not taken: the + generator chooses what to emit from the version of the assembler it is told + about, and it is now told one that predates them, no machine here being able + to run them and an assembler still in use being unable to assemble them. + Pinning that version also makes the checked-in assembly independent of the + host that produced it + [#176](https://github.com/kazu-yamamoto/crypton/pull/176) +* perf(sha256): take the CRYPTOGAMS SHA-256 for AArch64. The instructions are + the ones the intrinsics here already use; what the module does with them is + schedule them across a whole run of blocks rather than one at a time, and + keep the message schedule of the next block moving while the rounds of this + one are still going, which a function that is handed one block and returns + cannot do whatever it is written in. So the block loop hands over the whole + run, which also drops the alignment trampoline on this path -- the assembly + reads the message as bytes and wants neither the alignment nor the copy. On + Apple silicon: 2637 to 3279 MB/s at 16 KiB, against openssl's 3323 on the + same machine, and 1576 to 1966 at 64 bytes. SHA-512, which the same + generator emits, is not taken: 1876 here against openssl's 1880, the + ARMv8.2 instructions for it having gone in with #110 + [#175](https://github.com/kazu-yamamoto/crypton/pull/175) +* perf(poly1305): take the CRYPTOGAMS Poly1305 for AArch64. One + multiplication modulo 2^130 - 5 depends on the one before it, so what there + is to win is in how the multiplies and the carries are laid against each + other, and in keeping the accumulator in whichever base costs less: the + module works in base 2^64 while the message is short and switches to base + 2^26 for the four-way vector loop, deciding that for itself. Unlike the + other two it is the whole of the arithmetic rather than a bulk loop bolted + to the side, so the context now holds either the 26-bit limbs the C works in + or the 192 bytes the assembly keeps, as a union, and grows from 84 bytes to + 232. On Apple silicon: 4269 to 8060 MB/s at 16 KiB and 1542 to 4355 at 64 + bytes, and ChaCha20-Poly1305 together, which is what this is for, 1416 to + 2284 against openssl's 2180 on the same machine. `poly1305_neon.c`, which + [#169](https://github.com/kazu-yamamoto/crypton/pull/169) added, goes: the + assembly is faster at every length on every target that gets it, and the + scalar C remains for the targets that do not. The tests came first and + found that the chunking property here had been testing nothing -- it used + the all-zero key, whose r is zero, so both sides were the nonce whatever + they did, which is how it came to feed the chunks to `update` in reverse + order and pass + [#174](https://github.com/kazu-yamamoto/crypton/pull/174) +* perf(chacha): take the CRYPTOGAMS ChaCha20 for AArch64. The vector + registers hold four ChaCha states and there is no room for a fifth, so once + four blocks are in flight the only place further parallelism can come from + is the integer side: that module runs a fifth block through the general + registers alongside four in the vector ones, and above 512 bytes two + alongside six. Which register holds which word is the whole of the trick + and C has no way to say it, which is why the intrinsics here sat at about + 0.63 of what openssl gets out of this very file. On Apple silicon, a + message per call: 1911 to 3069 MB/s at 512 bytes, 1913 to 3093 at 4 KiB and + 2056 to 3112 at 64 KiB, against openssl 3.6's 3164 on the same machine, + which is this code. It is handed only the states it fits -- twenty rounds, + a 256-bit key, and as many blocks as the 32-bit counter has room for, since + crypton's counter is 64 bits wide and carries where the assembly wraps -- + and nothing below 192 bytes, where its own vector path starts. The tests + came first: the properties here generated one shape of state, so the + 256-bit constants were never exercised by them + [#173](https://github.com/kazu-yamamoto/crypton/pull/173) +* perf(gcm): take the CRYPTOGAMS stitched AES-GCM for x86-64, which is the + first assembly in the package. Counter-mode AES and GHASH do not compete + for the same execution ports, so a loop that interleaves them at + instruction granularity runs both in about the time the rounds alone take; + written in C that interleaving does not survive the compiler, which sinks + every multiply to the end of the group, and the disassembly of what + [#160](https://github.com/kazu-yamamoto/crypton/pull/160) produced says so. + On a Haswell-generation x86-64, a message per call, AES-128-GCM: 2672 to + 3172 MB/s at 1152 bytes, 3394 to 4271 at 4 KiB and 3657 to 5110 at 16 KiB, + where openssl speed on the same machine reports 4896; decryption within a + couple of points of that, and AES-256-GCM 3147 to 4297 at 16 KiB against + openssl's 4206. `cbits/asm` holds the module, the translator it needs and the + generated assembly, one file per object format, so that building needs no + perl; `cbits/asm/README.md` records where it came from and what was done to + it, which is to rename the entry points, a program linking both crypton + and openssl being entitled to object to two definitions of + `aesni_gcm_encrypt`. What the assembly reads is laid out OpenSSL's way and + is built per message in `cbits/aes/gcm_x86_asm.c`, the powers of H being + the ones crypton already has, shifted up a bit. Short messages are not + handed over at all. `cabal-version` is now 3.0, for `asm-sources` + [#172](https://github.com/kazu-yamamoto/crypton/pull/172) +* perf(sha3): use the ARMv8.2 SHA-3 instructions. Keccak was the plain C + everywhere, a round at a time over tables of rotation amounts and lane + positions, at half of what openssl manages on the same machine. EOR3, + RAX1, XAR and BCAX exist for exactly this permutation and take a round from + around a hundred and fifty operations to sixty-six; they come with the + SHA-512 extension the tree already asks for. Rho and pi move one lane of + every row into every other row, so the round cannot be done in place, and + four rounds go in an iteration, which is worth a fifth over one. SHA3-256 + 551 to 991 MB/s (openssl 1064), SHAKE128 700 to 1166, Keccak-256 559 to + 944. The body is generated from the definitions in FIPS 202 rather than + copied in, and the script that worked out the rotations and the lane + permutation checked itself against the published digests of the empty + string and of "abc" before emitting any C, which is how a first attempt + with chi reading lanes another row had already overwritten was caught. x86 + is untouched: nothing there has instructions for this + [#171](https://github.com/kazu-yamamoto/crypton/pull/171) +* perf(sha1): use the ARMv8 SHA-1 instructions. The AArch64 paths for + SHA-256 and SHA-512 went in with #104 and #110 and x86 got its SHA-1 + instructions in [#165](https://github.com/kazu-yamamoto/crypton/pull/165), + but the AArch64 SHA-1 ones were never used -- and they are part of the same + optional feature as the SHA-256 ones, so every processor that has those has + these. SHA1C, SHA1P and SHA1M each do four rounds with one of the three + round functions, SHA1H carries E from one group to the next, and SHA1SU0 + and SHA1SU1 do the message schedule between them. On Apple silicon: 1272 + to 3180 MB/s, against openssl 3.6's 3350 on the same machine. Checked + against the hardware rather than through an emulation of the instructions, + the machine here having them: the digests agree with the generic + implementation over every message length from 0 to 2000, with each input + split in two updates + [#170](https://github.com/kazu-yamamoto/crypton/pull/170) +* perf(poly1305): four blocks at a time with NEON. AArch64 had only the + scalar loop, whose five 26-bit limbs and 32-bit multiplies are the shape a + 32-bit machine wants. This is the arithmetic of the AVX2 path in NEON, + written as a transliteration of that file rather than a fresh formulation, + since the maths there is already pinned by the known-answer tests; what + differs is the width, AVX2 holding four 64-bit products in a register where + NEON holds two, so each product becomes a pair and the limbs are packed + back into four 32-bit lanes before the next multiply. On Apple silicon: + Poly1305 2783 to 4840 MB/s, and ChaCha20-Poly1305 together 1164 to 1368. + Checked against the scalar implementation over forty keys and every message + length from 0 to 400, with each input split in two updates. Also measured + and left alone: BLAKE2b at 1612 MB/s against openssl's 1378, the reference + C being the faster of the two + [#169](https://github.com/kazu-yamamoto/crypton/pull/169) +* perf(modes): stop the generic cipher modes allocating per byte. Counter + mode with a cipher whose modes are not in C ran at a third of what the same + cipher managed in ECB, and at an eighth for Blowfish, for two reasons + outside the cipher. The counters were built one at a time by `ivAdd`, + which allocates a block and walks the whole width of the counter from the + original for each of them; they are now one buffer filled in place. And + the exclusive or was `Data.ByteArray`'s, which walks a byte at a time + through an IO applicative -- 420 MB of heap for 8 MiB of counter mode, + against 17 MB for the same data through ECB, which is fifty bytes allocated + per byte produced and cost more than the cipher did. There is a + `crypton_memxor` to call instead, a pass of words, which the modes and CMAC + use. The serial modes also took each block as a copy and take shared + slices now. On Apple silicon, counter mode: Camellia-128 79.7 to 285.9 + MB/s, Blowfish 60.4 to 282.6, DES 46.2 to 114.9, CAST5 40.2 to 86.6, + Twofish-128 37.4 to 57.6, 3DES 25.2 to 36.3, and CBC and CMAC by a third to + a half as much again. AES is unchanged: its modes are in C and never came + this way + [#168](https://github.com/kazu-yamamoto/crypton/pull/168) +* build: compile the C at -O3, which is what came of looking at P-256 against + openssl. The comparison in the problem list was wrong -- a base point + multiplication here against openssl's ECDH, which is a variable point one -- + and measured properly P-256 is 2.8 to 3.3 times slower rather than the 1.27 + claimed. The time is in the field arithmetic, five 51-bit limbs in + Montgomery form at 44.4 ns a multiplication, against hand-written assembly + using `mulx`, `adcx` and `adox`; a four-limb saturated Montgomery + multiplication written in C to see what a compiler would give measured 41.3 + ns, so that is not the way in. What did move is the optimisation level GHC + passes: a P-256 base point multiplication goes from 71.0 to 59.8 us on x86-64 + and 26.0 to 24.3 on Apple silicon, AES-128-GCM from 3455 to 3708 MB/s and + AES-128-OCB from 2187 to 2484, with ChaCha20, Poly1305, SHA-1 and MD5 within + a couple of per cent either way. The masked selections in the curve and + field code compile to no conditional jumps at either level + [#167](https://github.com/kazu-yamamoto/crypton/pull/167) +* refactor(aes): drop the keystream generator nobody can call. `genCTR` and + `genCounter` are exported from a module in `other-modules`, so nothing + outside the library could reach them and nothing inside used them; the only + mention left was a test commented out since the cryptonite days. They were + also the slowest thing in the file, a block at a time through the + single-block entry point at 715 MB/s where counter mode does 5788, and the + three ways of fixing that are each worse than removing them: counter mode + over zeros costs Apple silicon a fifth, counters through ECB costs both, and + a keystream loop written out per key size is eighty lines for an API no + caller can see. Also declares `crypton_aes_encrypt_ctr` and + `crypton_aes_encrypt_c32` in the header, which had them defined and imported + but never declared + [#166](https://github.com/kazu-yamamoto/crypton/pull/166) +* perf(sha1): use the Intel SHA extensions on x86-64. The extension that + carries the SHA-256 instructions carries four for SHA-1 as well, and the same + cpuid bit answers for both, so this is one file and one branch. On an AMD + EPYC 7763: 725.1 to 1363.8 MB/s, against openssl's 1668.2 on the same + machine. 1.9x, where the SHA-256 instructions were worth 4.8x -- SHA-1's + rounds are cheaper to begin with, so there is less for an instruction to + replace. The sequence was checked by replacing the four instructions with C + that follows the SDM and comparing against the generic implementation over + every length from 0 to 1024, which found the same missing schedule step + [#155](https://github.com/kazu-yamamoto/crypton/pull/155) had + [#165](https://github.com/kazu-yamamoto/crypton/pull/165) +* docs(sidechannel): say what the modules that still work in `Integer` keep + from the clock, and fix the two places where something could be done about + it. ElGamal inverted the shared secret with the extended Euclidean + algorithm, whose steps follow the bits it is given -- the modulus is prime, + so Fermat reaches it. Its signing inverts the ephemeral value modulo an even + number, where Fermat does not reach, so `sign` blinds instead: the algorithm + is handed that value times a fresh random unit and the blinder divided out + afterwards. What is left is written down rather than fixed -- the Jacobi + symbols Rabin takes modulo its private primes, and the cost of `Integer` + arithmetic following the size of the numbers -- and `Crypto.Cipher.AES` now + says which implementation a machine gets and that the fallback, being + table-driven, is not constant time + [#164](https://github.com/kazu-yamamoto/crypton/pull/164) +* perf(poly1305): shorten the carry chain and stop the AVX2 loop spilling. The + carries go in pairs, since the two halves of that chain do not depend on each + other; the powers of r are read from memory, there being sixteen registers + and ten of them wanted for the accumulator and the products; and the message + is added limb by limb as the block comes apart rather than five limbs being + formed first. 4203 to 4452 MB/s, and ChaCha20-Poly1305 together from 1412 to + 1488. What is left is the instruction count: 107 per 64 bytes, of which 25 + are the multiply + [#163](https://github.com/kazu-yamamoto/crypton/pull/163) +* perf(chacha): combine as the keystream comes out of the registers. All three + vector implementations wrote it to a buffer on the stack and read it back to + exclusive-or it with the input, which is a pass over every byte for something + the registers were already holding. 2128 to 2230 MB/s on x86-64, and nothing + on Apple silicon, where the round trip was free. Measured while doing it: + the AVX2 path already did eight blocks at a time, and what is left of the gap + to openssl in this AEAD is Poly1305 rather than the cipher + [#162](https://github.com/kazu-yamamoto/crypton/pull/162) +* perf(sha): compute the message schedule in vector registers on x86. SHA-512 + has no instruction there and SHA-256 has none on a processor older than + Goldmont or Zen, which includes the Ice Lake and Cascade Lake server parts. + The rounds are a chain and stay where they are; the schedule is a quarter of + the work, comes out four words at a time and depends on nothing but the + message, so it goes into the vector registers and runs alongside rounds that + need the general ones. SHA-256 228 to 314 MB/s, SHA-512 354 to 480 + [#161](https://github.com/kazu-yamamoto/crypton/pull/161) +* perf(gcm): take the GHASH of the group before, alongside this group's rounds. + Held a group apart the multiply and the rounds run through each other, where + in step neither could start until the other finished. With it, the multiply + called directly rather than through a branch pointer the compiler cannot see + through, and the round keys read from memory rather than spilled: AES-128-GCM + 2797 to 3458 MB/s and AES-256-GCM 2458 to 3053. openssl does 4895 and 4205 + on the same machine; the rest of that is instruction-level interleaving, + which does not survive being written in intrinsics + [#160](https://github.com/kazu-yamamoto/crypton/pull/160) +* perf(ocb): drive OCB through the ECB paths a group at a time. It ran one + block at a time through the single-block entry point and so cost four times + what GCM costs, for a mode that does less work than GCM. The offsets have to + be worked out in order but the block cipher calls under them do not depend on + each other, so eight go through ECB together. OCB-128 1130 to 3500 MB/s on + Apple silicon and 694 to 2173 on x86-64, the authenticated data 1141 to 5900 + and 692 to 2526. CCM is unchanged and stays that way: what is left there is + CBC-MAC, where each block waits for the one before it + [#159](https://github.com/kazu-yamamoto/crypton/pull/159) +* test(aes): run the XTS vectors, and add OCB and CCM at 192 and 256 bits. The + XTS known-answer tests never ran: the call was commented out and the test it + would have called did not compile, so vectors at both key sizes sat in the + tree unused. XTS is defined only for a 128-bit block, which the general KAT + runner cannot promise, so it gains a counterpart for a cipher that can. OCB + and CCM had vectors at 128 bits only. 2613 examples to 2679 + [#158](https://github.com/kazu-yamamoto/crypton/pull/158) +* perf(aes): build the AArch64 key schedule with the instructions rather than + the S-box table. The AArch64 path expanded a key by calling the generic + implementation and then inverting the round keys, so every schedule went + through sixteen lookups at addresses derived from the key -- a small thing + next to the per-block indexing the extensions exist to remove, but a key + schedule is what an attacker most wants out of a cache, and x86 has never + needed the table. AArch64 has no counterpart to AESKEYGENASSIST, but AESE + against a zero key is SubBytes and ShiftRows, and a word given to it in all + four columns comes back as SubWord in each of them. The words stay in + vector registers throughout, which is what makes it free: moving each one to + a general register for the instruction and back cost more than the + instruction did, 87 to 144 ns for an AES-128 schedule, where keeping them in + registers gives 81.4 + [#157](https://github.com/kazu-yamamoto/crypton/pull/157) +* perf(aes): AES-192 through the processor's AES instructions. Every 192-bit + slot in the branch table was left at the generic code, on x86 and on AArch64 + alike, so a 192-bit key got the table-driven software AES while 128 and 256 + got the instructions. It was 164 times slower for counter mode on the x86 + machine measured and 62 on Apple silicon, and it was also the only key size + whose data path indexes a table with bytes derived from the key -- a caller + who picks AES-192 over AES-128 for a wider margin was quietly given a weaker + one. Counter mode then GCM, before and after: Apple silicon 152.7 to 9452.0 + MB/s and 112.3 to 7049.3, x86-64 40.4 to 6635.1 and 39.9 to 2633.5. Both + implementations were already written once per key size, so this instantiates + them again at twelve rounds; x86 also needed the 192-bit schedule, which + does not fall into 128-bit pieces the way the other two do + [#156](https://github.com/kazu-yamamoto/crypton/pull/156) +* perf(sha256): use the Intel SHA extensions on x86-64, which is what issue + [#31](https://github.com/kazu-yamamoto/crypton/issues/31) reports -- SHA-256 + four to eight times slower than sha256sum and openssl, both of which use the + processor's instructions. AArch64 got its instructions in #104 and is at + parity with them; x86 had nothing. SHA256RNDS2 does two rounds at a time and + SHA256MSG1 and SHA256MSG2 help with the message schedule, so a block costs + four groups of sixteen instructions instead of sixty-four rounds of scalar + work. On an AMD EPYC 9V74: 338.3 to 1612.7 MB/s, against openssl's 1783.8 on + the same machine. The extensions arrived with Goldmont and Ice Lake at Intel + and with Zen at AMD, far later than AES-NI, so a processor without them is + ordinary rather than ancient: the code sits behind a target attribute and a + cpuid question, and the plain C stays for everything else + [#155](https://github.com/kazu-yamamoto/crypton/pull/155) +* perf(bcrypt): Blowfish, and the key setup bcrypt wraps it in, in C. bcrypt + is a cost parameter and a promise that the cost is paid, and what pays it is + the Blowfish key schedule; in Haskell that cost about twice what the usual + implementations charge, so a hash of a given length of time had to be asked + for with a lower cost than elsewhere. Cost 8 goes from 25.97 to 9.98 ms, + cost 10 from 102.01 to 39.79, cost 12 from 418.78 to 159.41, `bcrypt_pbkdf` + from 109.76 to 40.38, and Blowfish over 4 KiB from 0.05 to 0.01 -- at cost + 10 that is 39.8 ms against the 52 `htpasswd` takes on the same machine. The + Haskell cipher goes with it, so there is one implementation rather than two, + and nothing exposed changes + [#154](https://github.com/kazu-yamamoto/crypton/pull/154) +* perf(prime): fewer Miller-Rabin rounds for a candidate nobody chose. A + number handed over may have been built to pass, and against that the only + thing to go on is that a round catches three quarters of the composites + there are, so `isProbablyPrime`, `findPrimeFrom` and `findPrimeFromWith`, + which all take their number from the caller, keep their thirty rounds. A + candidate drawn here is the case Damgard, Landrock and Pomerance worked out + and Table 4.4 of the Handbook of Applied Cryptography tabulates: + `generatePrime` and `generateSafePrime` now use twice what it asks for one + chance in 2^80, capped at the thirty they had, which leaves the chance far + under one in 2^100 at every size. With the candidates held fixed, + `generatePrime 1024` goes from 28.5 to 16.9 ms and an RSA-2048 key from 52.9 + to 39.2 + [#153](https://github.com/kazu-yamamoto/crypton/pull/153) +* fix(rsa): work the private exponent out without the extended Euclidean + algorithm. The modulus is the secret there, so multiplying the value by a + random number hides nothing; what does is that `e` is public. Whatever `d` + is, `e * d = 1 + k * phi` for some `k` under `e`, and reading that modulo + `e` gives `k` as an inverse modulo a number of a handful of bits, which for + a prime `e` is Fermat; `d` is then an exact division. What phi touches is a + remainder and a division, and nothing in either follows it + [#152](https://github.com/kazu-yamamoto/crypton/pull/152) +* perf(f2m): ask aarch64 for its carry-less multiply as well. #148 used PMULL + only where the compiler had been told the machine has the crypto + extensions, which is so on Apple and not on a Linux built for the bare + ARMv8 baseline, though every processor that runs such a build has it. It is + now compiled behind an attribute and the machine asked at run time, through + the auxiliary vector on Linux and Android, elf_aux_info on FreeBSD and a + sysctl on Apple: sect283k1 421.2 to 168.9 us there, sect571r1 2289.2 to + 579.9 + [#151](https://github.com/kazu-yamamoto/crypton/pull/151) +* refactor(ecc): one multiplication for both of the curve APIs, and one place + for each buffer's size. Which path a point multiplication takes was written + out twice, and the copy in `Crypto.ECC.Simple.Prim` cannot be reached from + outside the library on a curve over a binary field, so the suite never ran + it; it moves to the internal module both already share, which makes the copy + nobody can call the same code everybody runs. The two buffers for a C call + that still had their size written out separately from the offsets into them + now take both from one list, as the one that was wrong in #141 does -- the + note there records that neither valgrind nor the debug RTS catches that + mistake, both having been tried + [#150](https://github.com/kazu-yamamoto/crypton/pull/150) +* perf(f2m): use the x86 carry-less multiply where the processor has it. + PCLMULQDQ is not part of the x86-64 baseline, so the cpuid the package + already runs for AES-NI reports one more bit and the multiplication that + uses the instruction sits behind an attribute. Measured through Rosetta, + which translates rather than runs it, so the ratio is what to read: + sect283k1 560.4 to 177.5 us, sect571r1 3049.6 to 623.9 + [#149](https://github.com/kazu-yamamoto/crypton/pull/149) +* perf(f2m): do the binary field arithmetic in C. The ladder of #142 spent + nearly all its time on one thing -- a carry-less multiplication, which + ordinary arithmetic does not give and which in Haskell was `Integer` shifts + and exclusive ors, about 4 us for a 283-bit multiplication. The field and + the ladder over it are now C, with the processor's instruction where there + is one and four interleaved groups of bits where there is not, folding for + the reduction and Fermat for the inverse. sect163k1 3431 to 67.4 us with + the instruction and 117.3 without, sect283k1 10181 to 157.6 and 386.2, + sect571r1 40469 to 521.6 and 2101.0. It is also constant time, which the + Haskell ladder was not + [#148](https://github.com/kazu-yamamoto/crypton/pull/148) +* perf(bignum): start the doubling for `R^2 mod m` at the highest power of two + under the modulus rather than at one, which for a modulus that fills its + limbs is half the steps. Two to three percent of a curve operation, and + every curve operation and every `expSafe` pays for it once. Folding instead + of Montgomery for the primes shaped `2^k - c` was written and measured + alongside it and is not here: it is slower in this representation, 87.8 ns + against 76.8 for a 521-bit multiplication, because the shift down by `k` + costs more than the reduction pass it replaces when `k` does not land on a + limb boundary + [#147](https://github.com/kazu-yamamoto/crypton/pull/147) +* perf(ecc): keep a table of the multiples of each curve's base point, which + is the point signing and making a key multiply and the only one worth a + table. A multiplication with it is one addition per four bits and no + doublings: secp256k1 211.1 to 59.8 us, secp384r1 519.3 to 144.2, secp521r1 + 1047.7 to 283.0, and ECDSA P-384 signing 556.4 to 179.9 on both elliptic + curve APIs, which share the table. A table is built when a curve is first + asked for one -- 2.8 ms for secp256k1, 5.5 for secp384r1, 10.6 for secp521r1 + -- and is 221 KB and 456 KB for the last two, so it pays for itself after + about fifteen multiplications + [#146](https://github.com/kazu-yamamoto/crypton/pull/146) +* perf(bignum): take the limbs four and two at a time as well as eight in the + loop every modular multiplication is built out of. Four and six limbs, which + is what most of the curves want, fell entirely to the one-at-a-time tail + before: a field multiplication at six limbs goes from about 58 to 49 ns, + secp384r1 scalar multiplication from 596.7 to 519.3 us and ECDSA P-384 + signing from 645.0 to 556.4. Specialising the sizes further, which is what a + generated implementation would do, measures about 4% more and is not here + [#145](https://github.com/kazu-yamamoto/crypton/pull/145) +* fix(rsa): keep the blinding factor out of the extended Euclidean algorithm. + The blinder is a random number and its inverse, and the inverse went through + an algorithm whose steps follow the number handed to it -- the number the + blinding rests on, and unlike the other inverses this one is worked out once + per operation rather than once per key. `n` being composite leaves no + Fermat to fall back on, so the algorithm is handed the factor multiplied by + sixteen fresh random bytes and its answer multiplied by them again, which + leaves the inverse wanted and shows the algorithm nothing to do with it. In + IO, where every draw of randomness goes to the system, `generateBlinder` + goes from 74 to about 120 us and a PKCS#1 v1.5 `signSafer` from 719 to about + 765; under a DRG the caller carries, 24.7 to 24.9 + [#144](https://github.com/kazu-yamamoto/crypton/pull/144) +* fix(rsa): work `qinv` out without the extended Euclidean algorithm. Making + a key inverts one prime modulo the other and both of them are the key + itself, so that inverse is now Fermat's little theorem through `expSafe`, + which the other prime being prime allows: 308.6 us against 10.1, on a key + that takes tens of milliseconds to make. Making a key cannot be constant + time -- the search for the primes takes as long as it takes -- but what that + leaks is about the search rather than about the primes it settles on, and + the haddock now says which is which + [#143](https://github.com/kazu-yamamoto/crypton/pull/143) +* perf(ecc): a ladder for the curves over a binary field. These were the last + multiplication whose cost followed the scalar: an affine double-and-add, one + addition for every bit that was set and none for the others, which on + sect283k1 ran from 9665 us for a scalar with two bits set to 17958 for one + with 270. It is now Montgomery's ladder, which carries the multiples of two + consecutive numbers -- their difference being the point is what lets it + carry only their x coordinates -- and spends one addition and one doubling + on every bit whichever way it goes, working the y out at the end from the + two x it is left with, so one division does for the whole multiplication + where the affine code had one per step. The multiplication is now flat, and + quicker: sect163k1 4428 to 3431 us, sect233r1 9304 to 6806, sect283k1 13797 + to 10181, sect409k1 30826 to 20584, sect571r1 61931 to 40469. Uniform is + not constant time -- these are `Integer` operations, whose cost follows the + values -- and the point with no x, which is its own negation, keeps the code + that was there + [#142](https://github.com/kazu-yamamoto/crypton/pull/142) +* perf(ecc): multiply points in C on curves over a prime field. P-256 has had + a C implementation all along; every other prime curve -- P-384, P-521, + secp256k1 and the rest -- multiplied points with `Integer` arithmetic, which + cannot be constant time, since what an `Integer` operation costs follows the + value it is given. The C walks four bits of scalar at a time, taking the + multiple to add from a table of sixteen that it reads by touching every + entry and keeping one with a mask, and its addition and doubling are the + complete formulas of Renes, Costello and Batina, which answer for every pair + of points with no case to choose between. A P-384 multiplication goes from + 1557 to 585 us and no longer follows the scalar, ECDSA P-384 signing from + 1700 to 636 us, P-521 from 1942 to 1123. Binary curves are unchanged, and a + point that is not on the curve keeps the answer the Haskell gives it + [#141](https://github.com/kazu-yamamoto/crypton/pull/141) +* fix(ecc): add at every bit in the prime-curve multiplication, which laziness + was skipping. The multiplication adds at every bit, set or not, so that its + cost follows the width of the curve's order rather than the scalar -- but + the addition was a binding only one branch of the following `if` used, so at + a bit that was not set it stayed a thunk and was never worked out. The cost + followed the number of bits set in the scalar, which is the nonce when + signing and the private key in ECDH: on P-384, 765.8 us for a scalar with + two bits set against 1671.5 for one with 383, in a straight line between. + Both copies of the multiplication had it, so both elliptic curve APIs were + affected on every prime curve but P-256 + [#140](https://github.com/kazu-yamamoto/crypton/pull/140) +* fix(ecdsa): keep the P-256 signature out of `Integer` arithmetic. The + scalar handed to the C implementation was reduced with `mod`, a division, + whose steps follow the number being divided -- the nonce when signing, the + private key in ECDH. Twice the order is more than 256 bits hold, so a + scalar that fits is brought under the order by one masked subtraction + instead. The second half of a signature, `kInv * (z + r * d)`, was + `Integer` arithmetic as well, and now goes through `scalarAdd` and + `scalarMul`, which on P-256 are the C implementation's fixed-width + arithmetic. What is left on that curve is the conversion between `Integer` + and fixed-width scalars, which is also what it costs: signing goes from 34.1 + to 39.3 us, and ECDH and the other curves are unchanged + [#139](https://github.com/kazu-yamamoto/crypton/pull/139) +* fix(dsa,ecdsa): invert the signing nonce without a side channel. Both + inverted it with the extended Euclidean algorithm, whose step count and + branches follow the bits of what it is given -- and a handful of signatures + whose nonces are partly known give the private key away, so the nonce is + worth as much as the key. `Crypto.Number.ModArithmetic.inverseSafe` works + the inverse out with Fermat's little theorem through `expSafe` instead, + falling back on `inverse` when the modulus turns out not to be prime, so + every answer is the one it was. On P-256 the C implementation does it. + Signing costs a little more: ECDSA P-256 30.6 to 34.1 us, ECDSA P-384 678.7 + to 703.4, DSA-2048 422.5 to 437.1. Verification inverts a value that + arrives in the signature and is left alone + [#138](https://github.com/kazu-yamamoto/crypton/pull/138) +* perf(number): square, and multiply, faster in `expSafe`. The product and + the Montgomery reduction are now a full product followed by a reduction + rather than interleaved, built out of one loop that takes its limbs eight at + a time, and squaring works out only the products on one side of the diagonal + and doubles their sum. At 2048 bits the constant-time exponentiation goes + from 3.59 to 2.15 ms, which is 1.4x GMP's own rather than 2.4x; RSA-2048 + signing goes from 0.80 to 0.63 ms and DH-2048 `getShared` from 2.43 to 1.67 + [#137](https://github.com/kazu-yamamoto/crypton/pull/137) +* fix(number): make `expSafe` hide the exponent again. It asked integer-gmp + for `powModSecInteger` and fell back on the ordinary `powModInteger` when + that was missing; since integer-gmp 1.1 it is always missing, so on every + GHC this package supports `expSafe` was the same windowed exponentiation as + `expFast`, table indexed by the exponent's bits, for RSA, DSA, DH, ElGamal + and Rabin alike. It now goes to C: four bits of exponent at a time, the + table of sixteen read by touching every entry and keeping one with a mask, + and a Montgomery multiplication whose final subtraction is masked too. The + exponent's length is still visible, rounded up to a whole 64-bit word, which + is what GMP's own `mpz_powm_sec` lets slip. Hiding the exponent costs 1.6x + at 512 bits and 2.4x at 2048: RSA-2048 signing goes from 0.46 to 0.80 ms and + DH-2048 `getShared` from 1.04 to 2.43 + [#136](https://github.com/kazu-yamamoto/crypton/pull/136) +* perf(prime): stop running a Fermat test that Miller-Rabin subsumes. Every + candidate was tested to base 2 before the Miller-Rabin rounds, which begin + with the same base and prove more; the primes it passed paid for it twice + and the composites it caught were nearly all caught by trial division first. + RSA-2048 key generation goes from 55.0 to 32.8 ms + [#135](https://github.com/kazu-yamamoto/crypton/pull/135) +* perf(f2m): reduce the binary field by folding the top back in rather than + taking a step per bit of excess, square a byte at a time through a table of + the patterns a byte spreads into, and take four bits of a multiplier at a + time rather than one. On the 283-bit field, squaring goes from 5440 to 2068 + ns and multiplication from 7526 to 4086. A scalar multiplication there is + still affine, so it inverts once per addition, which is where its time now + goes + [#134](https://github.com/kazu-yamamoto/crypton/pull/134) +* perf(ecc): fold instead of dividing in the generic prime-curve arithmetic, + and add the point being multiplied as the affine point it is. These primes + are `2^k - c` with `c` far smaller, so the top half of a product folds back + in with a shift, a multiplication and an addition, where dividing costs four + times as much -- above 256 bits, below which the folding costs more than it + saves. P-521 scalar multiplication goes from 892 to 492 us and P-384 from + 684 to 572 + [#133](https://github.com/kazu-yamamoto/crypton/pull/133) +* perf(ecc): route P-256 through the C implementation the library already had. + `Crypto.PubKey.ECDSA` reached `cbits/p256`; `Crypto.PubKey.ECC.*`, the older + and more widely used API, never did. ECDSA signing goes from 590 to 28.7 us, + verification from 726 to 91.4, and `getShared` from 1177 to 96. On P-256 + that multiplication is now constant time, where the generic code branches on + the scalar at every bit + [#132](https://github.com/kazu-yamamoto/crypton/pull/132) +* Breaking change: perf(camellia): put Camellia in C, 40 to 321 MiB/s. The + round function ran a byte at a time in Haskell; generating the tables that + take a byte straight to its contribution gained 14%, and the rest was the + language. Input that is not a whole number of blocks now raises, where the + tail of the answer used to be uninitialised memory + [#131](https://github.com/kazu-yamamoto/crypton/pull/131) +* Breaking change: perf(twofish): walk the blocks once and carry them in words + rather than appending each result to what came before and going through lists + per block. 2 MiB goes from 0.14 to 56 MiB/s, and the rate no longer falls as + the message grows. Input that is not a whole number of blocks now raises, + where it used to come back longer than it went in + [#130](https://github.com/kazu-yamamoto/crypton/pull/130) +* perf(modes): cut the message without copying the rest of it in the generic + block cipher modes, which every cipher but AES uses, and hand whole slices to + the cipher in the modes whose blocks do not depend on one another. Camellia + in CBC goes from 1.8 to 22.9 MiB/s at 1 MiB, DES CBC decryption from 0.5 to + 83, and every figure is now flat in the message length where it used to fall + [#129](https://github.com/kazu-yamamoto/crypton/pull/129) +* Breaking change: perf(des): put DES in C. It was carried over lists of + `Bool`, one cons cell per bit, with the key schedule recomputed for every + block: 0.04 MiB/s, and 3DES 0.013, against 105 and 41 for OpenSSL. They are + now 112 and 37. Input that is not a whole number of blocks now raises, where + the tail of the answer used to be uninitialised memory + [#128](https://github.com/kazu-yamamoto/crypton/pull/128) +* perf(cmac): slice the message rather than copying what is left of it once per + block, and chain through CBC, which is what CMAC's chaining is. A MAC over + 4 MiB goes from 0.36 to 1628 MiB/s, which is the speed of AES-CBC itself + [#127](https://github.com/kazu-yamamoto/crypton/pull/127) +* fix(rabin): decode OAEP without early exits, as + `Crypto.PubKey.RSA.OAEP.unpad` has since #91. The difference is not + measurable against the cost of mask generation, and is structural: the scan + across the padding no longer depends on the data + [#126](https://github.com/kazu-yamamoto/crypton/pull/126) +* Breaking change: fix(rabin): refuse a ciphertext or a signature that is not + below the modulus, and a ciphertext carrying a leading zero octet. Squaring + and the square roots that undo it work modulo n, so Basic and Rabin-Williams + decrypted `c + n` to whatever `c` decrypted to, and all three schemes verified + `s + n`, and `-s`, wherever they verified `s`. `Basic.signWith` also refuses a + padding whose first octet is zero, which the signature cannot carry: about one + signature in 256 was one its own `verify` rejected + [#125](https://github.com/kazu-yamamoto/crypton/pull/125) +* fix(prime): derive the Miller-Rabin witnesses from the number being tested and + from a secret drawn once per process. They came from one generator made once + and shared by every call, so the witnesses for one number were the witnesses + for every number, and testing a number again told the caller nothing it had + not already been told. This is the path every GHC since 9.0 takes, integer-gmp + 1.1 having no Miller-Rabin of its own + [#124](https://github.com/kazu-yamamoto/crypton/pull/124) +* docs(elgamal): say what `signWith` requires of its ephemeral value: the range + is 1 to p-2, not the "between 0 and p-1" the haddock claimed, and the value is + a private key that a signature discloses if it is reused or revealed + [#123](https://github.com/kazu-yamamoto/crypton/pull/123) +* Breaking change: fix(afis): give `split` and `merge` one answer for a parameter + they cannot use. They had four between them, including a division by zero for + an expand count of zero and, for a count of one, handing the diffused data back + as though it were the secret + [#122](https://github.com/kazu-yamamoto/crypton/pull/122) +* Breaking change: fix(rsa): refuse a ciphertext or a signature whose integer + representative is not below the modulus, which RFC 8017 requires in sections + 5.1.2 and 5.2.2. `PKCS15.decrypt` and `OAEP.decrypt` decrypted `c + n` to the + same message as `c`, and `PSS.verifyDigest` accepted `s + n` wherever it + accepted `s` + [#121](https://github.com/kazu-yamamoto/crypton/pull/121) +* fix(otp): search the HOTP resynchronization window without early exits. The + time taken read out both where in the window the client's counter was found + and how many of the submitted values were right -- the second of which the + answer itself does not give, being `Nothing` either way. A call now costs one + HMAC per counter in the window plus one per extra value, every time + [#120](https://github.com/kazu-yamamoto/crypton/pull/120) +* Breaking change: fix(kdf): report a refused parameter as a `CryptoError` rather + than as an `ErrorCall` carrying a string, with a `'`-suffixed variant of each + entry point returning `CryptoFailable`. PBKDF2 had no validation at all: a + negative output length reached `memSet` and killed the process with SIGBUS, and + an iteration count of zero returned 32 bytes of zeroes + [#119](https://github.com/kazu-yamamoto/crypton/pull/119) +* perf(xts): take eight blocks at a time on AArch64 and x86-64, and dispatch XTS + decryption through the branch table, which it had never used. AArch64 goes + from 1200 to 7742 MiB/s encrypting and 1166 to 7763 decrypting, x86-64 from + 1220 to 3464 and from 594 to 3461 + [#118](https://github.com/kazu-yamamoto/crypton/pull/118) +* perf(poly1305): take four blocks at a time with AVX2 on x86-64, folding the + lanes back together weighted by the powers of r. 1347 to 4137 MiB/s + [#117](https://github.com/kazu-yamamoto/crypton/pull/117) +* perf(ecc): work in Jacobian coordinates in both generic prime-field scalar + multiplications, and say in `Crypto.ECC` which curves branch on a secret + scalar. P-384 and P-521 ECDSA are 2.3x: signing goes from 3.36 to 1.46 ms and + from 5.96 to 2.61 ms. P-256, which has its own C implementation, is unaffected + [#116](https://github.com/kazu-yamamoto/crypton/pull/116) +* Breaking change: fix(padding): bound PKCS#7 padding by the block rather than by + the whole input, which had let a block of sixteen accept a claim of twenty, and + refuse a `ZERO` size of zero rather than dividing by it. What `ZERO` can and + cannot undo is now written down + [#115](https://github.com/kazu-yamamoto/crypton/pull/115) +* perf(gcm): give x86 its own GCM decryption loop. It fell to the generic one, + which calls the block function once per block, and ran at a quarter the speed + of encryption; both directions now take eight blocks at a time and fold their + GHASH into one reduction. AES-256-GCM decryption goes from 561 to 2733 MiB/s + and AES-128 from 667 to 3150 + [#114](https://github.com/kazu-yamamoto/crypton/pull/114) +* perf(chacha): take eight blocks at a time with AVX2 where the machine has it, + with the cpuid and XGETBV checks that decide. ChaCha20 on x86-64 goes from + 900 to 2074 MiB/s + [#113](https://github.com/kazu-yamamoto/crypton/pull/113) +* perf(chacha): do four blocks at a time with SSE2 on x86-64, where the cipher + had no vector code at all. ChaCha20 goes from 493 to 900 MiB/s + [#112](https://github.com/kazu-yamamoto/crypton/pull/112) +* perf(chacha): do four blocks at a time with NEON on AArch64. ChaCha20 goes + from 1025 to 1955 MiB/s + [#111](https://github.com/kazu-yamamoto/crypton/pull/111) +* feat(sha512): use the ARMv8.2 SHA-512 instructions on AArch64, which SHA-384 + and the truncated SHA-512/t variants share. Hashing 1 MiB goes from 1.53 ms + to 597 us. The extension is optional, so it is asked for at runtime on both + Apple and Linux rather than assumed + [#110](https://github.com/kazu-yamamoto/crypton/pull/110) +* perf(gcm): drive GCM from AArch64 rather than the generic loop, with a group + of eight blocks folding into a single GHASH reduction. AES-128-GCM goes from + 4030 to 8266 MiB/s and AES-256 from 4043 to 7172 + [#109](https://github.com/kazu-yamamoto/crypton/pull/109) +* perf(aes): specialise the AArch64 code by key size and interleave eight + blocks, and give CTR its own loop. AES-256 ECB goes from 3886 to 15991 + MiB/s, CTR from 2935 to 13567 and CBC decryption from 4366 to 15807 + [#108](https://github.com/kazu-yamamoto/crypton/pull/108) + +* perf(aes): build the AES-NI paths on Windows, which was missing from the list of + systems that compile them. Windows builds have been doing AES, and GHASH with it, + in the generic C + [#107](https://github.com/kazu-yamamoto/crypton/pull/107) +* fix(armv8): compile the AArch64 sources on a toolchain whose baseline lacks the + crypto extensions. They had not built with GCC on AArch64 Linux since #100; CI now + builds and tests there + [#106](https://github.com/kazu-yamamoto/crypton/pull/106) +* perf(gcm): fold four GHASH blocks into one reduction. AES-256-GCM is 1.6x at 1 KiB + and 2.6x at 64 KiB on Apple silicon, and the x86 paths gain the same structure + [#105](https://github.com/kazu-yamamoto/crypton/pull/105) +* perf(sha256): use the ARMv8 SHA-2 instructions on AArch64. SHA-256 and SHA-224 are + 5.5x + [#104](https://github.com/kazu-yamamoto/crypton/pull/104) +* ci: keep the macOS jobs from queueing behind each other, and supersede a branch's + earlier run + [#103](https://github.com/kazu-yamamoto/crypton/pull/103) +* perf(aes): use PMULL for GHASH on AArch64 + [#102](https://github.com/kazu-yamamoto/crypton/pull/102) +* ci: ask cabal where its caches live rather than assuming, and keep the build + products in the cache + [#101](https://github.com/kazu-yamamoto/crypton/pull/101) +* perf(aes): use the ARMv8 cryptographic extensions on AArch64. With the GHASH work + in #102 and #105, AES-256-ECB goes from 121 to 2992 MiB/s and AES-256-GCM from 92 to + 2318 MiB/s on Apple silicon + [#100](https://github.com/kazu-yamamoto/crypton/pull/100) +* build(bench): move the benchmarks from gauge, which is no longer maintained, to + tasty-bench, and let them resolve on a current GHC + [#99](https://github.com/kazu-yamamoto/crypton/pull/99) +* Breaking change: fix(padding): reject a `PKCS7` block size outside 1..255. `pad` + raises and `unpad` returns `Nothing`, where both previously narrowed the size to a + `Word8` and silently agreed on the wrong value + [#98](https://github.com/kazu-yamamoto/crypton/pull/98) +* feat(elgamal): fix `Crypto.PubKey.ElGamal` and expose it + [#97](https://github.com/kazu-yamamoto/crypton/pull/97) +* docs(bcrypt): say that only the first 72 bytes of a password count + [#96](https://github.com/kazu-yamamoto/crypton/pull/96) +* test: move the test suite from tasty to hspec, with hspec-discover. `cabal-version` + is now 2.0 + [#95](https://github.com/kazu-yamamoto/crypton/pull/95) + +* feat(aead): add `tryAeadSimpleDecrypt`, which takes the tag length as its own argument instead of reading it off the supplied tag + [#94](https://github.com/kazu-yamamoto/crypton/pull/94) +* Breaking change: feat(dh): add `tryGetShared` to `Crypto.PubKey.DH` and `Crypto.PubKey.ECC.DH`, reporting a rejected peer value as `CryptoFailable`; `getShared` is now defined in terms of it and so raises a `CryptoError` rather than an `ErrorCall` + [#93](https://github.com/kazu-yamamoto/crypton/pull/93) +* fix(otp): compare TOTP candidates without an early exit + [#92](https://github.com/kazu-yamamoto/crypton/pull/92) +* fix(rsa): drop the early exits from PKCS#1 v1.5 and OAEP unpadding + [#91](https://github.com/kazu-yamamoto/crypton/pull/91) +* Breaking change: fix(argon2): report invalid options as `CryptoFailed` rather than raising, adding `CryptoError_ParameterInvalid` to `CryptoError` + [#90](https://github.com/kazu-yamamoto/crypton/pull/90) +* Breaking change: fix(dh): validate the peer public number, and size the shared secret from `p` rather than `params_bits` + [#89](https://github.com/kazu-yamamoto/crypton/pull/89) +* fix(dsa): do not crash on values that are not invertible modulo `q` + [#88](https://github.com/kazu-yamamoto/crypton/pull/88) +* Breaking change: fix(ecdh): validate the peer point before the exchange + [#87](https://github.com/kazu-yamamoto/crypton/pull/87) +* Breaking change: fix(pkcs15): reject PKCS#1 v1.5 signatures of the wrong length or out of range + [#86](https://github.com/kazu-yamamoto/crypton/pull/86) +* Breaking change: fix(otp): require a digest long enough for RFC 4226 dynamic truncation, which was reading past the end of the MAC + [#85](https://github.com/kazu-yamamoto/crypton/pull/85) +* fix(ecc): accept zero-x P-256 shared secret + [#84](https://github.com/kazu-yamamoto/crypton/pull/84) +* fix(p256): accept valid edge-case points + [#83](https://github.com/kazu-yamamoto/crypton/pull/83) +* Breaking change: fix(hkdf): enforce output length limit + [#82](https://github.com/kazu-yamamoto/crypton/pull/82) +* Breaking change: fix(ed25519): reject non-canonical signatures + [#81](https://github.com/kazu-yamamoto/crypton/pull/81) +* Support GHC 9.14; `tested-with` now covers 9.10.2, 9.12.4 and 9.14.1 + [#74](https://github.com/kazu-yamamoto/crypton/pull/74) + +### API changes + +* New exports: `Crypto.OTP.minimumDigestSize`, `Crypto.PubKey.DH.tryGetShared`, + `Crypto.PubKey.ECC.DH.tryGetShared`, `Crypto.Cipher.Types.AEAD.tryAeadSimpleDecrypt`, + `Crypto.Number.ModArithmetic.inverseSafe`, `Crypto.PubKey.ECC.Prim.scalarInverse`, + `scalarAdd` and `scalarMul`, `Crypto.PubKey.ECC.P256.scalarReduce`, + and the whole of `Crypto.PubKey.ElGamal`, which was present but not exposed. + The variant of an entry point that reports a refusal rather than raising is + named `try` followed by the name it varies, `tryExpand` beside `expand`. A + trailing apostrophe was the obvious spelling and is what these were called + until shortly before release; it collides too easily, since a caller that + imports one of these modules unqualified and has its own `expand'` or + `split'` no longer compiles, and `tls` did. `Safe` was considered and set + aside: this library already uses that suffix for something else, in + `Crypto.Number.ModArithmetic.expSafe` and `inverseSafe` and in + `Crypto.PubKey.ECC.P256.scalarInvSafe`, where it means the value being + worked on stays out of the timing. + The KDFs gained a variant of each entry point that can refuse its parameters, + returning `CryptoFailable` instead of raising: `Crypto.KDF.Scrypt.tryGenerate`, + `Crypto.KDF.BCrypt.tryBcrypt`, `Crypto.KDF.BCryptPBKDF.tryGenerate` and + `tryHashInternal`, `Crypto.KDF.HKDF.tryExpand`, `Crypto.KDF.PBKDF2.tryGenerate` and + `tryFastPBKDF2_SHA1`, `tryFastPBKDF2_SHA256` and `tryFastPBKDF2_SHA512`, and + `Crypto.Data.AFIS.trySplit` and `tryMerge`. These are additions and break nothing. +* Breaking change: `CryptoError_ParameterInvalid` is added to `CryptoError`. It is + appended, so the `Enum` values of the existing constructors are unchanged, but an + exhaustive `case` without a wildcard will warn. Adding a constructor to an exported + datatype is what requires a major version bump under the PVP, which would have been + 1.2.0; this release goes to 2.0.0. Everything else below changes behaviour rather + than types. +* Breaking change: `getShared` in both DH modules raises a `CryptoError` where it + previously raised an `ErrorCall`, since it is now defined in terms of `tryGetShared`. + The same is now true of `Crypto.KDF.Scrypt.generate`, `Crypto.KDF.BCrypt.bcrypt`, + `Crypto.KDF.BCryptPBKDF.generate` and `hashInternal`, and `Crypto.Data.AFIS.split` + and `merge`, each of which is defined in terms of the variant above. +* Breaking change: input that used to be accepted is now rejected -- a digest shorter + than 20 bytes in `Crypto.OTP.hotp`, a signature of the wrong length or out of range + in `Crypto.PubKey.RSA.PKCS15.verify`, an off-curve peer point or a peer public number + outside `1 < y < p-1` in `getShared`, an output beyond 255 blocks in + `Crypto.KDF.HKDF.expand`, a non-canonical Ed25519 signature, and `Options` the + implementation refuses in `Crypto.KDF.Argon2.hash`. +* Breaking change: a value at or above the modulus is now rejected where it used to be + reduced and accepted -- a ciphertext in `Crypto.PubKey.RSA.PKCS15.decrypt` and + `Crypto.PubKey.RSA.OAEP.decrypt`, a signature in `Crypto.PubKey.RSA.PSS.verify`, and + both, along with a negated signature and a ciphertext with a leading zero octet, in + the three `Crypto.PubKey.Rabin.*` schemes. +* Breaking change: parameters that used to be accepted are now refused -- an iteration + count below one or a negative output length in `Crypto.KDF.PBKDF2`, an expand count + below two or a secret of no bytes in `Crypto.Data.AFIS`, a `PKCS7` claim longer than + the block and a `ZERO` size of zero in `Crypto.Data.Padding`, and a signature padding + whose first octet is zero in `Crypto.PubKey.Rabin.Basic.signWith`. +* Breaking change: DES, 3DES, Twofish and Camellia now raise on input that is not a + whole number of blocks, as AES already did. Before, DES and Camellia returned an + answer whose tail was never written -- uninitialised memory -- and Twofish returned + more than it was given, the missing bytes read as zero. +* Breaking change: `Crypto.Data.Padding.pad` raises on a `PKCS7` block size outside + 1..255, and `unpad` returns `Nothing` for one, where both used to narrow the size to + a `Word8` and hand back something other than what was padded. +* No exported function changed its signature. + ## 1.1.5 * fix(aead): reject undersized tags diff --git a/Crypto/Cipher/AES.hs b/Crypto/Cipher/AES.hs --- a/Crypto/Cipher/AES.hs +++ b/Crypto/Cipher/AES.hs @@ -7,6 +7,26 @@ -- Maintainer : Vincent Hanquez -- Stability : stable -- Portability : good +-- +-- AES, in the modes "Crypto.Cipher.Types" defines. +-- +-- == Which implementation runs +-- +-- Where the processor has instructions for AES -- AES-NI on x86-64, the +-- cryptographic extensions on AArch64 -- every key size and every mode here +-- goes through them, and a block costs the same whatever the key and the data +-- are. +-- +-- Where it does not, the fallback is the table-driven code in +-- @cbits\/aes\/generic.c@, which indexes a 256-byte table with bytes derived +-- from the key and from the block. That is the cache-timing exposure the +-- instructions exist to remove, and on such a machine AES here is not +-- constant time. Every x86-64 part since about 2010 and every AArch64 one in +-- ordinary use has the instructions. +-- +-- 'Crypto.System.CPU.processorOptions' says which of the two a given machine +-- got: @AESNI@ in that list means the processor's AES instructions, on either +-- architecture. module Crypto.Cipher.AES ( AES128, AES192, diff --git a/Crypto/Cipher/AES/Primitive.hs b/Crypto/Cipher/AES/Primitive.hs --- a/Crypto/Cipher/AES/Primitive.hs +++ b/Crypto/Cipher/AES/Primitive.hs @@ -21,8 +21,6 @@ initAES, -- * Miscellanea - genCTR, - genCounter, -- * Encryption encryptECB, @@ -172,12 +170,6 @@ ivToPtr :: ByteArrayAccess iv => iv -> (Ptr Word8 -> IO a) -> IO a ivToPtr iv f = withByteArray iv (f . castPtr) -ivCopyPtr :: IV AES -> (Ptr Word8 -> IO a) -> IO (a, IV AES) -ivCopyPtr (IV iv) f = (\(x, y) -> (x, IV y)) `fmap` copyAndModify iv f - where - copyAndModify :: ByteArray ba => ba -> (Ptr Word8 -> IO a) -> IO (a, ba) - copyAndModify ba f' = B.copyRet ba f' - withKeyAndIV :: ByteArrayAccess iv => AES -> iv -> (Ptr AES -> Ptr Word8 -> IO a) -> IO a withKeyAndIV ctx iv f = keyToPtr ctx $ \kptr -> ivToPtr iv $ \ivp -> f kptr ivp @@ -249,61 +241,6 @@ -- ^ ciphertext encryptCBC = doCBC c_aes_encrypt_cbc --- | generate a counter mode pad. this is generally xor-ed to an input --- to make the standard counter mode block operations. --- --- if the length requested is not a multiple of the block cipher size, --- more data will be returned, so that the returned bytearray is --- a multiple of the block cipher size. -{-# NOINLINE genCTR #-} -genCTR - :: ByteArray ba - => AES - -- ^ Cipher Key. - -> IV AES - -- ^ usually a 128 bit integer. - -> Int - -- ^ length of bytes required. - -> ba -genCTR ctx (IV iv) len - | len <= 0 = B.empty - | otherwise = B.allocAndFreeze (nbBlocks * 16) generate - where - generate o = withKeyAndIV ctx iv $ \k i -> c_aes_gen_ctr (castPtr o) k i (fromIntegral nbBlocks) - (nbBlocks', r) = len `quotRem` 16 - nbBlocks = if r == 0 then nbBlocks' else nbBlocks' + 1 - --- | generate a counter mode pad. this is generally xor-ed to an input --- to make the standard counter mode block operations. --- --- if the length requested is not a multiple of the block cipher size, --- more data will be returned, so that the returned bytearray is --- a multiple of the block cipher size. --- --- Similiar to 'genCTR' but also return the next IV for continuation -{-# NOINLINE genCounter #-} -genCounter - :: ByteArray ba - => AES - -> IV AES - -> Int - -> (ba, IV AES) -genCounter ctx iv len - | len <= 0 = (B.empty, iv) - | otherwise = unsafeDoIO $ - keyToPtr ctx $ \k -> - ivCopyPtr iv $ \i -> - B.alloc outputLength $ \o -> do - c_aes_gen_ctr_cont (castPtr o) k i (fromIntegral nbBlocks) - where - (nbBlocks', r) = len `quotRem` 16 - nbBlocks = if r == 0 then nbBlocks' else nbBlocks' + 1 - outputLength = nbBlocks * 16 - -{- TODO: when genCTR has same AESIV requirements for IV, add the following rules: - - RULES "snd . genCounter" forall ctx iv len . snd (genCounter ctx iv len) = genCTR ctx iv len - -} - -- | encrypt using Counter mode (CTR) -- -- in CTR mode encryption and decryption is the same operation. @@ -737,12 +674,6 @@ foreign import ccall "crypton_aes.h crypton_aes_decrypt_xts" c_aes_decrypt_xts :: CString -> Ptr AES -> Ptr AES -> Ptr Word8 -> CUInt -> CString -> CUInt -> IO () - -foreign import ccall "crypton_aes.h crypton_aes_gen_ctr" - c_aes_gen_ctr :: CString -> Ptr AES -> Ptr Word8 -> CUInt -> IO () - -foreign import ccall unsafe "crypton_aes.h crypton_aes_gen_ctr_cont" - c_aes_gen_ctr_cont :: CString -> Ptr AES -> Ptr Word8 -> CUInt -> IO () foreign import ccall "crypton_aes.h crypton_aes_encrypt_ctr" c_aes_encrypt_ctr diff --git a/Crypto/Cipher/Blowfish/Box.hs b/Crypto/Cipher/Blowfish/Box.hs deleted file mode 100644 --- a/Crypto/Cipher/Blowfish/Box.hs +++ /dev/null @@ -1,303 +0,0 @@ -{-# LANGUAGE MagicHash #-} - --- | --- Module : Crypto.Cipher.Blowfish.Box --- License : BSD-style --- Stability : experimental --- Portability : Good -module Crypto.Cipher.Blowfish.Box ( - KeySchedule (..), - createKeySchedule, - copyKeySchedule, -) where - -import Crypto.Internal.WordArray ( - MutableArray32, - mutableArray32FromAddrBE, - mutableArrayRead32, - mutableArrayWrite32, - ) - -newtype KeySchedule = KeySchedule MutableArray32 - --- | Copy the state of one key schedule into the other. --- The first parameter is the destination and the second the source. -copyKeySchedule :: KeySchedule -> KeySchedule -> IO () -copyKeySchedule (KeySchedule dst) (KeySchedule src) = loop 0 - where - loop 1042 = return () - loop i = do - w32 <- mutableArrayRead32 src i - mutableArrayWrite32 dst i w32 - loop (i + 1) - --- | Create a key schedule mutable array of the pbox followed by --- all the sboxes. -createKeySchedule :: IO KeySchedule -createKeySchedule = - KeySchedule - `fmap` mutableArray32FromAddrBE - 1042 - "\ - \\x24\x3f\x6a\x88\x85\xa3\x08\xd3\x13\x19\x8a\x2e\x03\x70\x73\x44\ - \\xa4\x09\x38\x22\x29\x9f\x31\xd0\x08\x2e\xfa\x98\xec\x4e\x6c\x89\ - \\x45\x28\x21\xe6\x38\xd0\x13\x77\xbe\x54\x66\xcf\x34\xe9\x0c\x6c\ - \\xc0\xac\x29\xb7\xc9\x7c\x50\xdd\x3f\x84\xd5\xb5\xb5\x47\x09\x17\ - \\x92\x16\xd5\xd9\x89\x79\xfb\x1b\ - \\xd1\x31\x0b\xa6\x98\xdf\xb5\xac\x2f\xfd\x72\xdb\xd0\x1a\xdf\xb7\ - \\xb8\xe1\xaf\xed\x6a\x26\x7e\x96\xba\x7c\x90\x45\xf1\x2c\x7f\x99\ - \\x24\xa1\x99\x47\xb3\x91\x6c\xf7\x08\x01\xf2\xe2\x85\x8e\xfc\x16\ - \\x63\x69\x20\xd8\x71\x57\x4e\x69\xa4\x58\xfe\xa3\xf4\x93\x3d\x7e\ - \\x0d\x95\x74\x8f\x72\x8e\xb6\x58\x71\x8b\xcd\x58\x82\x15\x4a\xee\ - \\x7b\x54\xa4\x1d\xc2\x5a\x59\xb5\x9c\x30\xd5\x39\x2a\xf2\x60\x13\ - \\xc5\xd1\xb0\x23\x28\x60\x85\xf0\xca\x41\x79\x18\xb8\xdb\x38\xef\ - \\x8e\x79\xdc\xb0\x60\x3a\x18\x0e\x6c\x9e\x0e\x8b\xb0\x1e\x8a\x3e\ - \\xd7\x15\x77\xc1\xbd\x31\x4b\x27\x78\xaf\x2f\xda\x55\x60\x5c\x60\ - \\xe6\x55\x25\xf3\xaa\x55\xab\x94\x57\x48\x98\x62\x63\xe8\x14\x40\ - \\x55\xca\x39\x6a\x2a\xab\x10\xb6\xb4\xcc\x5c\x34\x11\x41\xe8\xce\ - \\xa1\x54\x86\xaf\x7c\x72\xe9\x93\xb3\xee\x14\x11\x63\x6f\xbc\x2a\ - \\x2b\xa9\xc5\x5d\x74\x18\x31\xf6\xce\x5c\x3e\x16\x9b\x87\x93\x1e\ - \\xaf\xd6\xba\x33\x6c\x24\xcf\x5c\x7a\x32\x53\x81\x28\x95\x86\x77\ - \\x3b\x8f\x48\x98\x6b\x4b\xb9\xaf\xc4\xbf\xe8\x1b\x66\x28\x21\x93\ - \\x61\xd8\x09\xcc\xfb\x21\xa9\x91\x48\x7c\xac\x60\x5d\xec\x80\x32\ - \\xef\x84\x5d\x5d\xe9\x85\x75\xb1\xdc\x26\x23\x02\xeb\x65\x1b\x88\ - \\x23\x89\x3e\x81\xd3\x96\xac\xc5\x0f\x6d\x6f\xf3\x83\xf4\x42\x39\ - \\x2e\x0b\x44\x82\xa4\x84\x20\x04\x69\xc8\xf0\x4a\x9e\x1f\x9b\x5e\ - \\x21\xc6\x68\x42\xf6\xe9\x6c\x9a\x67\x0c\x9c\x61\xab\xd3\x88\xf0\ - \\x6a\x51\xa0\xd2\xd8\x54\x2f\x68\x96\x0f\xa7\x28\xab\x51\x33\xa3\ - \\x6e\xef\x0b\x6c\x13\x7a\x3b\xe4\xba\x3b\xf0\x50\x7e\xfb\x2a\x98\ - \\xa1\xf1\x65\x1d\x39\xaf\x01\x76\x66\xca\x59\x3e\x82\x43\x0e\x88\ - \\x8c\xee\x86\x19\x45\x6f\x9f\xb4\x7d\x84\xa5\xc3\x3b\x8b\x5e\xbe\ - \\xe0\x6f\x75\xd8\x85\xc1\x20\x73\x40\x1a\x44\x9f\x56\xc1\x6a\xa6\ - \\x4e\xd3\xaa\x62\x36\x3f\x77\x06\x1b\xfe\xdf\x72\x42\x9b\x02\x3d\ - \\x37\xd0\xd7\x24\xd0\x0a\x12\x48\xdb\x0f\xea\xd3\x49\xf1\xc0\x9b\ - \\x07\x53\x72\xc9\x80\x99\x1b\x7b\x25\xd4\x79\xd8\xf6\xe8\xde\xf7\ - \\xe3\xfe\x50\x1a\xb6\x79\x4c\x3b\x97\x6c\xe0\xbd\x04\xc0\x06\xba\ - \\xc1\xa9\x4f\xb6\x40\x9f\x60\xc4\x5e\x5c\x9e\xc2\x19\x6a\x24\x63\ - \\x68\xfb\x6f\xaf\x3e\x6c\x53\xb5\x13\x39\xb2\xeb\x3b\x52\xec\x6f\ - \\x6d\xfc\x51\x1f\x9b\x30\x95\x2c\xcc\x81\x45\x44\xaf\x5e\xbd\x09\ - \\xbe\xe3\xd0\x04\xde\x33\x4a\xfd\x66\x0f\x28\x07\x19\x2e\x4b\xb3\ - \\xc0\xcb\xa8\x57\x45\xc8\x74\x0f\xd2\x0b\x5f\x39\xb9\xd3\xfb\xdb\ - \\x55\x79\xc0\xbd\x1a\x60\x32\x0a\xd6\xa1\x00\xc6\x40\x2c\x72\x79\ - \\x67\x9f\x25\xfe\xfb\x1f\xa3\xcc\x8e\xa5\xe9\xf8\xdb\x32\x22\xf8\ - \\x3c\x75\x16\xdf\xfd\x61\x6b\x15\x2f\x50\x1e\xc8\xad\x05\x52\xab\ - \\x32\x3d\xb5\xfa\xfd\x23\x87\x60\x53\x31\x7b\x48\x3e\x00\xdf\x82\ - \\x9e\x5c\x57\xbb\xca\x6f\x8c\xa0\x1a\x87\x56\x2e\xdf\x17\x69\xdb\ - \\xd5\x42\xa8\xf6\x28\x7e\xff\xc3\xac\x67\x32\xc6\x8c\x4f\x55\x73\ - \\x69\x5b\x27\xb0\xbb\xca\x58\xc8\xe1\xff\xa3\x5d\xb8\xf0\x11\xa0\ - \\x10\xfa\x3d\x98\xfd\x21\x83\xb8\x4a\xfc\xb5\x6c\x2d\xd1\xd3\x5b\ - \\x9a\x53\xe4\x79\xb6\xf8\x45\x65\xd2\x8e\x49\xbc\x4b\xfb\x97\x90\ - \\xe1\xdd\xf2\xda\xa4\xcb\x7e\x33\x62\xfb\x13\x41\xce\xe4\xc6\xe8\ - \\xef\x20\xca\xda\x36\x77\x4c\x01\xd0\x7e\x9e\xfe\x2b\xf1\x1f\xb4\ - \\x95\xdb\xda\x4d\xae\x90\x91\x98\xea\xad\x8e\x71\x6b\x93\xd5\xa0\ - \\xd0\x8e\xd1\xd0\xaf\xc7\x25\xe0\x8e\x3c\x5b\x2f\x8e\x75\x94\xb7\ - \\x8f\xf6\xe2\xfb\xf2\x12\x2b\x64\x88\x88\xb8\x12\x90\x0d\xf0\x1c\ - \\x4f\xad\x5e\xa0\x68\x8f\xc3\x1c\xd1\xcf\xf1\x91\xb3\xa8\xc1\xad\ - \\x2f\x2f\x22\x18\xbe\x0e\x17\x77\xea\x75\x2d\xfe\x8b\x02\x1f\xa1\ - \\xe5\xa0\xcc\x0f\xb5\x6f\x74\xe8\x18\xac\xf3\xd6\xce\x89\xe2\x99\ - \\xb4\xa8\x4f\xe0\xfd\x13\xe0\xb7\x7c\xc4\x3b\x81\xd2\xad\xa8\xd9\ - \\x16\x5f\xa2\x66\x80\x95\x77\x05\x93\xcc\x73\x14\x21\x1a\x14\x77\ - \\xe6\xad\x20\x65\x77\xb5\xfa\x86\xc7\x54\x42\xf5\xfb\x9d\x35\xcf\ - \\xeb\xcd\xaf\x0c\x7b\x3e\x89\xa0\xd6\x41\x1b\xd3\xae\x1e\x7e\x49\ - \\x00\x25\x0e\x2d\x20\x71\xb3\x5e\x22\x68\x00\xbb\x57\xb8\xe0\xaf\ - \\x24\x64\x36\x9b\xf0\x09\xb9\x1e\x55\x63\x91\x1d\x59\xdf\xa6\xaa\ - \\x78\xc1\x43\x89\xd9\x5a\x53\x7f\x20\x7d\x5b\xa2\x02\xe5\xb9\xc5\ - \\x83\x26\x03\x76\x62\x95\xcf\xa9\x11\xc8\x19\x68\x4e\x73\x4a\x41\ - \\xb3\x47\x2d\xca\x7b\x14\xa9\x4a\x1b\x51\x00\x52\x9a\x53\x29\x15\ - \\xd6\x0f\x57\x3f\xbc\x9b\xc6\xe4\x2b\x60\xa4\x76\x81\xe6\x74\x00\ - \\x08\xba\x6f\xb5\x57\x1b\xe9\x1f\xf2\x96\xec\x6b\x2a\x0d\xd9\x15\ - \\xb6\x63\x65\x21\xe7\xb9\xf9\xb6\xff\x34\x05\x2e\xc5\x85\x56\x64\ - \\x53\xb0\x2d\x5d\xa9\x9f\x8f\xa1\x08\xba\x47\x99\x6e\x85\x07\x6a\ - \\x4b\x7a\x70\xe9\xb5\xb3\x29\x44\xdb\x75\x09\x2e\xc4\x19\x26\x23\ - \\xad\x6e\xa6\xb0\x49\xa7\xdf\x7d\x9c\xee\x60\xb8\x8f\xed\xb2\x66\ - \\xec\xaa\x8c\x71\x69\x9a\x17\xff\x56\x64\x52\x6c\xc2\xb1\x9e\xe1\ - \\x19\x36\x02\xa5\x75\x09\x4c\x29\xa0\x59\x13\x40\xe4\x18\x3a\x3e\ - \\x3f\x54\x98\x9a\x5b\x42\x9d\x65\x6b\x8f\xe4\xd6\x99\xf7\x3f\xd6\ - \\xa1\xd2\x9c\x07\xef\xe8\x30\xf5\x4d\x2d\x38\xe6\xf0\x25\x5d\xc1\ - \\x4c\xdd\x20\x86\x84\x70\xeb\x26\x63\x82\xe9\xc6\x02\x1e\xcc\x5e\ - \\x09\x68\x6b\x3f\x3e\xba\xef\xc9\x3c\x97\x18\x14\x6b\x6a\x70\xa1\ - \\x68\x7f\x35\x84\x52\xa0\xe2\x86\xb7\x9c\x53\x05\xaa\x50\x07\x37\ - \\x3e\x07\x84\x1c\x7f\xde\xae\x5c\x8e\x7d\x44\xec\x57\x16\xf2\xb8\ - \\xb0\x3a\xda\x37\xf0\x50\x0c\x0d\xf0\x1c\x1f\x04\x02\x00\xb3\xff\ - \\xae\x0c\xf5\x1a\x3c\xb5\x74\xb2\x25\x83\x7a\x58\xdc\x09\x21\xbd\ - \\xd1\x91\x13\xf9\x7c\xa9\x2f\xf6\x94\x32\x47\x73\x22\xf5\x47\x01\ - \\x3a\xe5\xe5\x81\x37\xc2\xda\xdc\xc8\xb5\x76\x34\x9a\xf3\xdd\xa7\ - \\xa9\x44\x61\x46\x0f\xd0\x03\x0e\xec\xc8\xc7\x3e\xa4\x75\x1e\x41\ - \\xe2\x38\xcd\x99\x3b\xea\x0e\x2f\x32\x80\xbb\xa1\x18\x3e\xb3\x31\ - \\x4e\x54\x8b\x38\x4f\x6d\xb9\x08\x6f\x42\x0d\x03\xf6\x0a\x04\xbf\ - \\x2c\xb8\x12\x90\x24\x97\x7c\x79\x56\x79\xb0\x72\xbc\xaf\x89\xaf\ - \\xde\x9a\x77\x1f\xd9\x93\x08\x10\xb3\x8b\xae\x12\xdc\xcf\x3f\x2e\ - \\x55\x12\x72\x1f\x2e\x6b\x71\x24\x50\x1a\xdd\xe6\x9f\x84\xcd\x87\ - \\x7a\x58\x47\x18\x74\x08\xda\x17\xbc\x9f\x9a\xbc\xe9\x4b\x7d\x8c\ - \\xec\x7a\xec\x3a\xdb\x85\x1d\xfa\x63\x09\x43\x66\xc4\x64\xc3\xd2\ - \\xef\x1c\x18\x47\x32\x15\xd9\x08\xdd\x43\x3b\x37\x24\xc2\xba\x16\ - \\x12\xa1\x4d\x43\x2a\x65\xc4\x51\x50\x94\x00\x02\x13\x3a\xe4\xdd\ - \\x71\xdf\xf8\x9e\x10\x31\x4e\x55\x81\xac\x77\xd6\x5f\x11\x19\x9b\ - \\x04\x35\x56\xf1\xd7\xa3\xc7\x6b\x3c\x11\x18\x3b\x59\x24\xa5\x09\ - \\xf2\x8f\xe6\xed\x97\xf1\xfb\xfa\x9e\xba\xbf\x2c\x1e\x15\x3c\x6e\ - \\x86\xe3\x45\x70\xea\xe9\x6f\xb1\x86\x0e\x5e\x0a\x5a\x3e\x2a\xb3\ - \\x77\x1f\xe7\x1c\x4e\x3d\x06\xfa\x29\x65\xdc\xb9\x99\xe7\x1d\x0f\ - \\x80\x3e\x89\xd6\x52\x66\xc8\x25\x2e\x4c\xc9\x78\x9c\x10\xb3\x6a\ - \\xc6\x15\x0e\xba\x94\xe2\xea\x78\xa5\xfc\x3c\x53\x1e\x0a\x2d\xf4\ - \\xf2\xf7\x4e\xa7\x36\x1d\x2b\x3d\x19\x39\x26\x0f\x19\xc2\x79\x60\ - \\x52\x23\xa7\x08\xf7\x13\x12\xb6\xeb\xad\xfe\x6e\xea\xc3\x1f\x66\ - \\xe3\xbc\x45\x95\xa6\x7b\xc8\x83\xb1\x7f\x37\xd1\x01\x8c\xff\x28\ - \\xc3\x32\xdd\xef\xbe\x6c\x5a\xa5\x65\x58\x21\x85\x68\xab\x98\x02\ - \\xee\xce\xa5\x0f\xdb\x2f\x95\x3b\x2a\xef\x7d\xad\x5b\x6e\x2f\x84\ - \\x15\x21\xb6\x28\x29\x07\x61\x70\xec\xdd\x47\x75\x61\x9f\x15\x10\ - \\x13\xcc\xa8\x30\xeb\x61\xbd\x96\x03\x34\xfe\x1e\xaa\x03\x63\xcf\ - \\xb5\x73\x5c\x90\x4c\x70\xa2\x39\xd5\x9e\x9e\x0b\xcb\xaa\xde\x14\ - \\xee\xcc\x86\xbc\x60\x62\x2c\xa7\x9c\xab\x5c\xab\xb2\xf3\x84\x6e\ - \\x64\x8b\x1e\xaf\x19\xbd\xf0\xca\xa0\x23\x69\xb9\x65\x5a\xbb\x50\ - \\x40\x68\x5a\x32\x3c\x2a\xb4\xb3\x31\x9e\xe9\xd5\xc0\x21\xb8\xf7\ - \\x9b\x54\x0b\x19\x87\x5f\xa0\x99\x95\xf7\x99\x7e\x62\x3d\x7d\xa8\ - \\xf8\x37\x88\x9a\x97\xe3\x2d\x77\x11\xed\x93\x5f\x16\x68\x12\x81\ - \\x0e\x35\x88\x29\xc7\xe6\x1f\xd6\x96\xde\xdf\xa1\x78\x58\xba\x99\ - \\x57\xf5\x84\xa5\x1b\x22\x72\x63\x9b\x83\xc3\xff\x1a\xc2\x46\x96\ - \\xcd\xb3\x0a\xeb\x53\x2e\x30\x54\x8f\xd9\x48\xe4\x6d\xbc\x31\x28\ - \\x58\xeb\xf2\xef\x34\xc6\xff\xea\xfe\x28\xed\x61\xee\x7c\x3c\x73\ - \\x5d\x4a\x14\xd9\xe8\x64\xb7\xe3\x42\x10\x5d\x14\x20\x3e\x13\xe0\ - \\x45\xee\xe2\xb6\xa3\xaa\xab\xea\xdb\x6c\x4f\x15\xfa\xcb\x4f\xd0\ - \\xc7\x42\xf4\x42\xef\x6a\xbb\xb5\x65\x4f\x3b\x1d\x41\xcd\x21\x05\ - \\xd8\x1e\x79\x9e\x86\x85\x4d\xc7\xe4\x4b\x47\x6a\x3d\x81\x62\x50\ - \\xcf\x62\xa1\xf2\x5b\x8d\x26\x46\xfc\x88\x83\xa0\xc1\xc7\xb6\xa3\ - \\x7f\x15\x24\xc3\x69\xcb\x74\x92\x47\x84\x8a\x0b\x56\x92\xb2\x85\ - \\x09\x5b\xbf\x00\xad\x19\x48\x9d\x14\x62\xb1\x74\x23\x82\x0e\x00\ - \\x58\x42\x8d\x2a\x0c\x55\xf5\xea\x1d\xad\xf4\x3e\x23\x3f\x70\x61\ - \\x33\x72\xf0\x92\x8d\x93\x7e\x41\xd6\x5f\xec\xf1\x6c\x22\x3b\xdb\ - \\x7c\xde\x37\x59\xcb\xee\x74\x60\x40\x85\xf2\xa7\xce\x77\x32\x6e\ - \\xa6\x07\x80\x84\x19\xf8\x50\x9e\xe8\xef\xd8\x55\x61\xd9\x97\x35\ - \\xa9\x69\xa7\xaa\xc5\x0c\x06\xc2\x5a\x04\xab\xfc\x80\x0b\xca\xdc\ - \\x9e\x44\x7a\x2e\xc3\x45\x34\x84\xfd\xd5\x67\x05\x0e\x1e\x9e\xc9\ - \\xdb\x73\xdb\xd3\x10\x55\x88\xcd\x67\x5f\xda\x79\xe3\x67\x43\x40\ - \\xc5\xc4\x34\x65\x71\x3e\x38\xd8\x3d\x28\xf8\x9e\xf1\x6d\xff\x20\ - \\x15\x3e\x21\xe7\x8f\xb0\x3d\x4a\xe6\xe3\x9f\x2b\xdb\x83\xad\xf7\ - \\xe9\x3d\x5a\x68\x94\x81\x40\xf7\xf6\x4c\x26\x1c\x94\x69\x29\x34\ - \\x41\x15\x20\xf7\x76\x02\xd4\xf7\xbc\xf4\x6b\x2e\xd4\xa2\x00\x68\ - \\xd4\x08\x24\x71\x33\x20\xf4\x6a\x43\xb7\xd4\xb7\x50\x00\x61\xaf\ - \\x1e\x39\xf6\x2e\x97\x24\x45\x46\x14\x21\x4f\x74\xbf\x8b\x88\x40\ - \\x4d\x95\xfc\x1d\x96\xb5\x91\xaf\x70\xf4\xdd\xd3\x66\xa0\x2f\x45\ - \\xbf\xbc\x09\xec\x03\xbd\x97\x85\x7f\xac\x6d\xd0\x31\xcb\x85\x04\ - \\x96\xeb\x27\xb3\x55\xfd\x39\x41\xda\x25\x47\xe6\xab\xca\x0a\x9a\ - \\x28\x50\x78\x25\x53\x04\x29\xf4\x0a\x2c\x86\xda\xe9\xb6\x6d\xfb\ - \\x68\xdc\x14\x62\xd7\x48\x69\x00\x68\x0e\xc0\xa4\x27\xa1\x8d\xee\ - \\x4f\x3f\xfe\xa2\xe8\x87\xad\x8c\xb5\x8c\xe0\x06\x7a\xf4\xd6\xb6\ - \\xaa\xce\x1e\x7c\xd3\x37\x5f\xec\xce\x78\xa3\x99\x40\x6b\x2a\x42\ - \\x20\xfe\x9e\x35\xd9\xf3\x85\xb9\xee\x39\xd7\xab\x3b\x12\x4e\x8b\ - \\x1d\xc9\xfa\xf7\x4b\x6d\x18\x56\x26\xa3\x66\x31\xea\xe3\x97\xb2\ - \\x3a\x6e\xfa\x74\xdd\x5b\x43\x32\x68\x41\xe7\xf7\xca\x78\x20\xfb\ - \\xfb\x0a\xf5\x4e\xd8\xfe\xb3\x97\x45\x40\x56\xac\xba\x48\x95\x27\ - \\x55\x53\x3a\x3a\x20\x83\x8d\x87\xfe\x6b\xa9\xb7\xd0\x96\x95\x4b\ - \\x55\xa8\x67\xbc\xa1\x15\x9a\x58\xcc\xa9\x29\x63\x99\xe1\xdb\x33\ - \\xa6\x2a\x4a\x56\x3f\x31\x25\xf9\x5e\xf4\x7e\x1c\x90\x29\x31\x7c\ - \\xfd\xf8\xe8\x02\x04\x27\x2f\x70\x80\xbb\x15\x5c\x05\x28\x2c\xe3\ - \\x95\xc1\x15\x48\xe4\xc6\x6d\x22\x48\xc1\x13\x3f\xc7\x0f\x86\xdc\ - \\x07\xf9\xc9\xee\x41\x04\x1f\x0f\x40\x47\x79\xa4\x5d\x88\x6e\x17\ - \\x32\x5f\x51\xeb\xd5\x9b\xc0\xd1\xf2\xbc\xc1\x8f\x41\x11\x35\x64\ - \\x25\x7b\x78\x34\x60\x2a\x9c\x60\xdf\xf8\xe8\xa3\x1f\x63\x6c\x1b\ - \\x0e\x12\xb4\xc2\x02\xe1\x32\x9e\xaf\x66\x4f\xd1\xca\xd1\x81\x15\ - \\x6b\x23\x95\xe0\x33\x3e\x92\xe1\x3b\x24\x0b\x62\xee\xbe\xb9\x22\ - \\x85\xb2\xa2\x0e\xe6\xba\x0d\x99\xde\x72\x0c\x8c\x2d\xa2\xf7\x28\ - \\xd0\x12\x78\x45\x95\xb7\x94\xfd\x64\x7d\x08\x62\xe7\xcc\xf5\xf0\ - \\x54\x49\xa3\x6f\x87\x7d\x48\xfa\xc3\x9d\xfd\x27\xf3\x3e\x8d\x1e\ - \\x0a\x47\x63\x41\x99\x2e\xff\x74\x3a\x6f\x6e\xab\xf4\xf8\xfd\x37\ - \\xa8\x12\xdc\x60\xa1\xeb\xdd\xf8\x99\x1b\xe1\x4c\xdb\x6e\x6b\x0d\ - \\xc6\x7b\x55\x10\x6d\x67\x2c\x37\x27\x65\xd4\x3b\xdc\xd0\xe8\x04\ - \\xf1\x29\x0d\xc7\xcc\x00\xff\xa3\xb5\x39\x0f\x92\x69\x0f\xed\x0b\ - \\x66\x7b\x9f\xfb\xce\xdb\x7d\x9c\xa0\x91\xcf\x0b\xd9\x15\x5e\xa3\ - \\xbb\x13\x2f\x88\x51\x5b\xad\x24\x7b\x94\x79\xbf\x76\x3b\xd6\xeb\ - \\x37\x39\x2e\xb3\xcc\x11\x59\x79\x80\x26\xe2\x97\xf4\x2e\x31\x2d\ - \\x68\x42\xad\xa7\xc6\x6a\x2b\x3b\x12\x75\x4c\xcc\x78\x2e\xf1\x1c\ - \\x6a\x12\x42\x37\xb7\x92\x51\xe7\x06\xa1\xbb\xe6\x4b\xfb\x63\x50\ - \\x1a\x6b\x10\x18\x11\xca\xed\xfa\x3d\x25\xbd\xd8\xe2\xe1\xc3\xc9\ - \\x44\x42\x16\x59\x0a\x12\x13\x86\xd9\x0c\xec\x6e\xd5\xab\xea\x2a\ - \\x64\xaf\x67\x4e\xda\x86\xa8\x5f\xbe\xbf\xe9\x88\x64\xe4\xc3\xfe\ - \\x9d\xbc\x80\x57\xf0\xf7\xc0\x86\x60\x78\x7b\xf8\x60\x03\x60\x4d\ - \\xd1\xfd\x83\x46\xf6\x38\x1f\xb0\x77\x45\xae\x04\xd7\x36\xfc\xcc\ - \\x83\x42\x6b\x33\xf0\x1e\xab\x71\xb0\x80\x41\x87\x3c\x00\x5e\x5f\ - \\x77\xa0\x57\xbe\xbd\xe8\xae\x24\x55\x46\x42\x99\xbf\x58\x2e\x61\ - \\x4e\x58\xf4\x8f\xf2\xdd\xfd\xa2\xf4\x74\xef\x38\x87\x89\xbd\xc2\ - \\x53\x66\xf9\xc3\xc8\xb3\x8e\x74\xb4\x75\xf2\x55\x46\xfc\xd9\xb9\ - \\x7a\xeb\x26\x61\x8b\x1d\xdf\x84\x84\x6a\x0e\x79\x91\x5f\x95\xe2\ - \\x46\x6e\x59\x8e\x20\xb4\x57\x70\x8c\xd5\x55\x91\xc9\x02\xde\x4c\ - \\xb9\x0b\xac\xe1\xbb\x82\x05\xd0\x11\xa8\x62\x48\x75\x74\xa9\x9e\ - \\xb7\x7f\x19\xb6\xe0\xa9\xdc\x09\x66\x2d\x09\xa1\xc4\x32\x46\x33\ - \\xe8\x5a\x1f\x02\x09\xf0\xbe\x8c\x4a\x99\xa0\x25\x1d\x6e\xfe\x10\ - \\x1a\xb9\x3d\x1d\x0b\xa5\xa4\xdf\xa1\x86\xf2\x0f\x28\x68\xf1\x69\ - \\xdc\xb7\xda\x83\x57\x39\x06\xfe\xa1\xe2\xce\x9b\x4f\xcd\x7f\x52\ - \\x50\x11\x5e\x01\xa7\x06\x83\xfa\xa0\x02\xb5\xc4\x0d\xe6\xd0\x27\ - \\x9a\xf8\x8c\x27\x77\x3f\x86\x41\xc3\x60\x4c\x06\x61\xa8\x06\xb5\ - \\xf0\x17\x7a\x28\xc0\xf5\x86\xe0\x00\x60\x58\xaa\x30\xdc\x7d\x62\ - \\x11\xe6\x9e\xd7\x23\x38\xea\x63\x53\xc2\xdd\x94\xc2\xc2\x16\x34\ - \\xbb\xcb\xee\x56\x90\xbc\xb6\xde\xeb\xfc\x7d\xa1\xce\x59\x1d\x76\ - \\x6f\x05\xe4\x09\x4b\x7c\x01\x88\x39\x72\x0a\x3d\x7c\x92\x7c\x24\ - \\x86\xe3\x72\x5f\x72\x4d\x9d\xb9\x1a\xc1\x5b\xb4\xd3\x9e\xb8\xfc\ - \\xed\x54\x55\x78\x08\xfc\xa5\xb5\xd8\x3d\x7c\xd3\x4d\xad\x0f\xc4\ - \\x1e\x50\xef\x5e\xb1\x61\xe6\xf8\xa2\x85\x14\xd9\x6c\x51\x13\x3c\ - \\x6f\xd5\xc7\xe7\x56\xe1\x4e\xc4\x36\x2a\xbf\xce\xdd\xc6\xc8\x37\ - \\xd7\x9a\x32\x34\x92\x63\x82\x12\x67\x0e\xfa\x8e\x40\x60\x00\xe0\ - \\x3a\x39\xce\x37\xd3\xfa\xf5\xcf\xab\xc2\x77\x37\x5a\xc5\x2d\x1b\ - \\x5c\xb0\x67\x9e\x4f\xa3\x37\x42\xd3\x82\x27\x40\x99\xbc\x9b\xbe\ - \\xd5\x11\x8e\x9d\xbf\x0f\x73\x15\xd6\x2d\x1c\x7e\xc7\x00\xc4\x7b\ - \\xb7\x8c\x1b\x6b\x21\xa1\x90\x45\xb2\x6e\xb1\xbe\x6a\x36\x6e\xb4\ - \\x57\x48\xab\x2f\xbc\x94\x6e\x79\xc6\xa3\x76\xd2\x65\x49\xc2\xc8\ - \\x53\x0f\xf8\xee\x46\x8d\xde\x7d\xd5\x73\x0a\x1d\x4c\xd0\x4d\xc6\ - \\x29\x39\xbb\xdb\xa9\xba\x46\x50\xac\x95\x26\xe8\xbe\x5e\xe3\x04\ - \\xa1\xfa\xd5\xf0\x6a\x2d\x51\x9a\x63\xef\x8c\xe2\x9a\x86\xee\x22\ - \\xc0\x89\xc2\xb8\x43\x24\x2e\xf6\xa5\x1e\x03\xaa\x9c\xf2\xd0\xa4\ - \\x83\xc0\x61\xba\x9b\xe9\x6a\x4d\x8f\xe5\x15\x50\xba\x64\x5b\xd6\ - \\x28\x26\xa2\xf9\xa7\x3a\x3a\xe1\x4b\xa9\x95\x86\xef\x55\x62\xe9\ - \\xc7\x2f\xef\xd3\xf7\x52\xf7\xda\x3f\x04\x6f\x69\x77\xfa\x0a\x59\ - \\x80\xe4\xa9\x15\x87\xb0\x86\x01\x9b\x09\xe6\xad\x3b\x3e\xe5\x93\ - \\xe9\x90\xfd\x5a\x9e\x34\xd7\x97\x2c\xf0\xb7\xd9\x02\x2b\x8b\x51\ - \\x96\xd5\xac\x3a\x01\x7d\xa6\x7d\xd1\xcf\x3e\xd6\x7c\x7d\x2d\x28\ - \\x1f\x9f\x25\xcf\xad\xf2\xb8\x9b\x5a\xd6\xb4\x72\x5a\x88\xf5\x4c\ - \\xe0\x29\xac\x71\xe0\x19\xa5\xe6\x47\xb0\xac\xfd\xed\x93\xfa\x9b\ - \\xe8\xd3\xc4\x8d\x28\x3b\x57\xcc\xf8\xd5\x66\x29\x79\x13\x2e\x28\ - \\x78\x5f\x01\x91\xed\x75\x60\x55\xf7\x96\x0e\x44\xe3\xd3\x5e\x8c\ - \\x15\x05\x6d\xd4\x88\xf4\x6d\xba\x03\xa1\x61\x25\x05\x64\xf0\xbd\ - \\xc3\xeb\x9e\x15\x3c\x90\x57\xa2\x97\x27\x1a\xec\xa9\x3a\x07\x2a\ - \\x1b\x3f\x6d\x9b\x1e\x63\x21\xf5\xf5\x9c\x66\xfb\x26\xdc\xf3\x19\ - \\x75\x33\xd9\x28\xb1\x55\xfd\xf5\x03\x56\x34\x82\x8a\xba\x3c\xbb\ - \\x28\x51\x77\x11\xc2\x0a\xd9\xf8\xab\xcc\x51\x67\xcc\xad\x92\x5f\ - \\x4d\xe8\x17\x51\x38\x30\xdc\x8e\x37\x9d\x58\x62\x93\x20\xf9\x91\ - \\xea\x7a\x90\xc2\xfb\x3e\x7b\xce\x51\x21\xce\x64\x77\x4f\xbe\x32\ - \\xa8\xb6\xe3\x7e\xc3\x29\x3d\x46\x48\xde\x53\x69\x64\x13\xe6\x80\ - \\xa2\xae\x08\x10\xdd\x6d\xb2\x24\x69\x85\x2d\xfd\x09\x07\x21\x66\ - \\xb3\x9a\x46\x0a\x64\x45\xc0\xdd\x58\x6c\xde\xcf\x1c\x20\xc8\xae\ - \\x5b\xbe\xf7\xdd\x1b\x58\x8d\x40\xcc\xd2\x01\x7f\x6b\xb4\xe3\xbb\ - \\xdd\xa2\x6a\x7e\x3a\x59\xff\x45\x3e\x35\x0a\x44\xbc\xb4\xcd\xd5\ - \\x72\xea\xce\xa8\xfa\x64\x84\xbb\x8d\x66\x12\xae\xbf\x3c\x6f\x47\ - \\xd2\x9b\xe4\x63\x54\x2f\x5d\x9e\xae\xc2\x77\x1b\xf6\x4e\x63\x70\ - \\x74\x0e\x0d\x8d\xe7\x5b\x13\x57\xf8\x72\x16\x71\xaf\x53\x7d\x5d\ - \\x40\x40\xcb\x08\x4e\xb4\xe2\xcc\x34\xd2\x46\x6a\x01\x15\xaf\x84\ - \\xe1\xb0\x04\x28\x95\x98\x3a\x1d\x06\xb8\x9f\xb4\xce\x6e\xa0\x48\ - \\x6f\x3f\x3b\x82\x35\x20\xab\x82\x01\x1a\x1d\x4b\x27\x72\x27\xf8\ - \\x61\x15\x60\xb1\xe7\x93\x3f\xdc\xbb\x3a\x79\x2b\x34\x45\x25\xbd\ - \\xa0\x88\x39\xe1\x51\xce\x79\x4b\x2f\x32\xc9\xb7\xa0\x1f\xba\xc9\ - \\xe0\x1c\xc8\x7e\xbc\xc7\xd1\xf6\xcf\x01\x11\xc3\xa1\xe8\xaa\xc7\ - \\x1a\x90\x87\x49\xd4\x4f\xbd\x9a\xd0\xda\xde\xcb\xd5\x0a\xda\x38\ - \\x03\x39\xc3\x2a\xc6\x91\x36\x67\x8d\xf9\x31\x7c\xe0\xb1\x2b\x4f\ - \\xf7\x9e\x59\xb7\x43\xf5\xbb\x3a\xf2\xd5\x19\xff\x27\xd9\x45\x9c\ - \\xbf\x97\x22\x2c\x15\xe6\xfc\x2a\x0f\x91\xfc\x71\x9b\x94\x15\x25\ - \\xfa\xe5\x93\x61\xce\xb6\x9c\xeb\xc2\xa8\x64\x59\x12\xba\xa8\xd1\ - \\xb6\xc1\x07\x5e\xe3\x05\x6a\x0c\x10\xd2\x50\x65\xcb\x03\xa4\x42\ - \\xe0\xec\x6e\x0e\x16\x98\xdb\x3b\x4c\x98\xa0\xbe\x32\x78\xe9\x64\ - \\x9f\x1f\x95\x32\xe0\xd3\x92\xdf\xd3\xa0\x34\x2b\x89\x71\xf2\x1e\ - \\x1b\x0a\x74\x41\x4b\xa3\x34\x8c\xc5\xbe\x71\x20\xc3\x76\x32\xd8\ - \\xdf\x35\x9f\x8d\x9b\x99\x2f\x2e\xe6\x0b\x6f\x47\x0f\xe3\xf1\x1d\ - \\xe5\x4c\xda\x54\x1e\xda\xd8\x91\xce\x62\x79\xcf\xcd\x3e\x7e\x6f\ - \\x16\x18\xb1\x66\xfd\x2c\x1d\x05\x84\x8f\xd2\xc5\xf6\xfb\x22\x99\ - \\xf5\x23\xf3\x57\xa6\x32\x76\x23\x93\xa8\x35\x31\x56\xcc\xcd\x02\ - \\xac\xf0\x81\x62\x5a\x75\xeb\xb5\x6e\x16\x36\x97\x88\xd2\x73\xcc\ - \\xde\x96\x62\x92\x81\xb9\x49\xd0\x4c\x50\x90\x1b\x71\xc6\x56\x14\ - \\xe6\xc6\xc7\xbd\x32\x7a\x14\x0a\x45\xe1\xd0\x06\xc3\xf2\x7b\x9a\ - \\xc9\xaa\x53\xfd\x62\xa8\x0f\x00\xbb\x25\xbf\xe2\x35\xbd\xd2\xf6\ - \\x71\x12\x69\x05\xb2\x04\x02\x22\xb6\xcb\xcf\x7c\xcd\x76\x9c\x2b\ - \\x53\x11\x3e\xc0\x16\x40\xe3\xd3\x38\xab\xbd\x60\x25\x47\xad\xf0\ - \\xba\x38\x20\x9c\xf7\x46\xce\x76\x77\xaf\xa1\xc5\x20\x75\x60\x60\ - \\x85\xcb\xfe\x4e\x8a\xe8\x8d\xd8\x7a\xaa\xf9\xb0\x4c\xf9\xaa\x7e\ - \\x19\x48\xc2\x5c\x02\xfb\x8a\x8c\x01\xc3\x6a\xe4\xd6\xeb\xe1\xf9\ - \\x90\xd4\xf8\x69\xa6\x5c\xde\xa0\x3f\x09\x25\x2d\xc2\x08\xe6\x9f\ - \\xb7\x4e\x61\x32\xce\x77\xe2\x5b\x57\x8f\xdf\xe3\x3a\xc3\x72\xe6\ - \"# diff --git a/Crypto/Cipher/Blowfish/Primitive.hs b/Crypto/Cipher/Blowfish/Primitive.hs --- a/Crypto/Cipher/Blowfish/Primitive.hs +++ b/Crypto/Cipher/Blowfish/Primitive.hs @@ -12,273 +12,139 @@ -- License : BSD-style -- Stability : experimental -- Portability : Good +-- +-- The cipher itself is in C, as is the key setup bcrypt wraps around it: +-- what the schedule costs is the whole of what bcrypt is for, and in Haskell +-- it cost about twice what the usual implementations do. module Crypto.Cipher.Blowfish.Primitive ( Context, initBlowfish, encrypt, decrypt, - KeySchedule, - createKeySchedule, - freezeKeySchedule, - expandKey, - expandKeyWithSalt, - cipherBlockMutable, + bcryptHash, + bcryptPbkdfHash, ) where -import Control.Monad (when) -import Data.Bits -import Data.Memory.Endian -import Data.Word - -import Crypto.Cipher.Blowfish.Box import Crypto.Error -import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) +import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, ScrubbedBytes) import qualified Crypto.Internal.ByteArray as B import Crypto.Internal.Compat import Crypto.Internal.Imports -import Crypto.Internal.WordArray +import Foreign.C.Types (CInt (..)) +import Foreign.Ptr (Ptr) -newtype Context = Context Array32 +-- | The key schedule: the P array and the four S boxes, as the C keeps them. +newtype Context = Context ScrubbedBytes instance NFData Context where rnf a = a `seq` () +-- | How many bytes of schedule the C wants: eighteen words and four boxes of +-- two hundred and fifty-six. +contextSize :: Int +contextSize = (18 + 4 * 256) * 4 + -- | Initialize a new Blowfish context from a key. -- -- key needs to be between 0 and 448 bits. initBlowfish :: ByteArrayAccess key => key -> CryptoFailable Context initBlowfish key | B.length key > (448 `div` 8) = CryptoFailed CryptoError_KeySizeInvalid - | otherwise = CryptoPassed $ unsafeDoIO $ do - ks <- createKeySchedule - expandKey ks key - freezeKeySchedule ks - --- | Get an immutable Blowfish context by freezing a mutable key schedule. -freezeKeySchedule :: KeySchedule -> IO Context -freezeKeySchedule (KeySchedule ma) = Context `fmap` mutableArray32Freeze ma - -expandKey :: ByteArrayAccess key => KeySchedule -> key -> IO () -expandKey ks@(KeySchedule ma) key = do - when (B.length key > 0) $ iterKeyStream key 0 0 $ \i l r a0 a1 cont -> do - mutableArrayWriteXor32 ma i l - mutableArrayWriteXor32 ma (i + 1) r - when (i + 2 < 18) (cont a0 a1) - loop 0 0 0 - where - loop i l r = do - n <- cipherBlockMutable ks (fromIntegral l `shiftL` 32 .|. fromIntegral r) - let nl = fromIntegral (n `shiftR` 32) - nr = fromIntegral (n .&. 0xffffffff) - mutableArrayWrite32 ma i nl - mutableArrayWrite32 ma (i + 1) nr - when (i < 18 + 1024) (loop (i + 2) nl nr) - -expandKeyWithSalt - :: (ByteArrayAccess key, ByteArrayAccess salt) - => KeySchedule - -> key - -> salt - -> IO () -expandKeyWithSalt ks key salt - | B.length salt == 16 = - expandKeyWithSalt128 - ks - key - (fromBE $ B.toW64BE salt 0) - (fromBE $ B.toW64BE salt 8) - | otherwise = expandKeyWithSaltAny ks key salt - -expandKeyWithSaltAny - :: (ByteArrayAccess key, ByteArrayAccess salt) - => KeySchedule - -- ^ The key schedule - -> key - -- ^ The key - -> salt - -- ^ The salt - -> IO () -expandKeyWithSaltAny ks@(KeySchedule ma) key salt = do - when (B.length key > 0) $ iterKeyStream key 0 0 $ \i l r a0 a1 cont -> do - mutableArrayWriteXor32 ma i l - mutableArrayWriteXor32 ma (i + 1) r - when (i + 2 < 18) (cont a0 a1) - -- Go through the entire key schedule overwriting the P-Array and S-Boxes - when (B.length salt > 0) $ iterKeyStream salt 0 0 $ \i l r a0 a1 cont -> do - let l' = xor l a0 - let r' = xor r a1 - n <- cipherBlockMutable ks (fromIntegral l' `shiftL` 32 .|. fromIntegral r') - let nl = fromIntegral (n `shiftR` 32) - nr = fromIntegral (n .&. 0xffffffff) - mutableArrayWrite32 ma i nl - mutableArrayWrite32 ma (i + 1) nr - when (i + 2 < 18 + 1024) (cont nl nr) - -expandKeyWithSalt128 - :: ByteArrayAccess ba - => KeySchedule - -- ^ The key schedule - -> ba - -- ^ The key - -> Word64 - -- ^ First word of the salt - -> Word64 - -- ^ Second word of the salt - -> IO () -expandKeyWithSalt128 ks@(KeySchedule ma) key salt1 salt2 = do - when (B.length key > 0) $ iterKeyStream key 0 0 $ \i l r a0 a1 cont -> do - mutableArrayWriteXor32 ma i l - mutableArrayWriteXor32 ma (i + 1) r - when (i + 2 < 18) (cont a0 a1) - -- Go through the entire key schedule overwriting the P-Array and S-Boxes - loop 0 salt1 salt1 salt2 - where - loop i input slt1 slt2 - | i == 1042 = return () - | otherwise = do - n <- cipherBlockMutable ks input - let nl = fromIntegral (n `shiftR` 32) - nr = fromIntegral (n .&. 0xffffffff) - mutableArrayWrite32 ma i nl - mutableArrayWrite32 ma (i + 1) nr - loop (i + 2) (n `xor` slt2) slt2 slt1 + | otherwise = CryptoPassed $ + unsafeDoIO $ + fmap Context $ + B.alloc contextSize $ \ctx -> + B.withByteArray key $ \k -> + c_blowfish_init ctx k (fromIntegral (B.length key)) -- | Encrypt blocks -- -- Input need to be a multiple of 8 bytes encrypt :: ByteArray ba => Context -> ba -> ba -encrypt ctx ba - | B.length ba == 0 = B.empty - | B.length ba `mod` 8 /= 0 = error "invalid data length" - | otherwise = B.mapAsWord64 (cipherBlock ctx False) ba +encrypt = through c_blowfish_encrypt -- | Decrypt blocks -- -- Input need to be a multiple of 8 bytes decrypt :: ByteArray ba => Context -> ba -> ba -decrypt ctx ba - | B.length ba == 0 = B.empty - | B.length ba `mod` 8 /= 0 = error "invalid data length" - | otherwise = B.mapAsWord64 (cipherBlock ctx True) ba +decrypt = through c_blowfish_decrypt --- | Encrypt or decrypt a single block of 64 bits. --- --- The inverse argument decides whether to encrypt or decrypt. -cipherBlock :: Context -> Bool -> Word64 -> Word64 -cipherBlock (Context ar) inverse input = doRound input 0 +through + :: ByteArray ba + => (Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO ()) + -> Context + -> ba + -> ba +through f (Context ctx) input + | len `mod` 8 /= 0 = + error "Crypto.Cipher.Blowfish: input length must be a multiple of 8" + | otherwise = unsafeDoIO $ + B.alloc len $ \out -> + B.withByteArray ctx $ \c -> + B.withByteArray input $ \i -> f c out i (fromIntegral len) where - -- \| Transform the input over 16 rounds - doRound :: Word64 -> Int -> Word64 - doRound !i roundIndex - | roundIndex == 16 = - let final = (fromIntegral (p 16) `shiftL` 32) .|. fromIntegral (p 17) - in rotateL (i `xor` final) 32 - | otherwise = - let newr = fromIntegral (i `shiftR` 32) `xor` p roundIndex - newi = ((i `shiftL` 32) `xor` f newr) .|. fromIntegral newr - in doRound newi (roundIndex + 1) + len = B.length input - -- \| The Blowfish Feistel function F - f :: Word32 -> Word64 - f t = - let a = s0 (0xff .&. (t `shiftR` 24)) - b = s1 (0xff .&. (t `shiftR` 16)) - c = s2 (0xff .&. (t `shiftR` 8)) - d = s3 (0xff .&. t) - in fromIntegral (((a + b) `xor` c) + d) `shiftL` 32 +-- | What bcrypt does with Blowfish: the key setup that costs what the cost +-- says, and then the sixty-four encryptions. The answer is 24 bytes, of +-- which bcrypt keeps 23. +-- +-- The salt has to be 16 bytes and the key 1 to 73, which is a password of at +-- most 72 with the zero byte the original implementation appends. 'Nothing' +-- means it was given something else. +bcryptHash + :: (ByteArrayAccess salt, ByteArrayAccess key, ByteArray output) + => Int + -- ^ the cost, between 4 and 31 + -> salt + -> key + -> Maybe output +bcryptHash cost salt key + | cost < 4 || cost > 31 = Nothing + | B.length salt /= 16 = Nothing + | B.length key < 1 || B.length key > 73 = Nothing + | otherwise = unsafeDoIO $ do + (r, out) <- B.allocRet 24 $ \o -> + B.withByteArray salt $ \s -> + B.withByteArray key $ \k -> + c_bcrypt o (fromIntegral cost) s k (fromIntegral (B.length key)) + return $ if r == 0 then Just out else Nothing - -- \| S-Box arrays, each containing 256 32-bit words - -- The first 18 words contain the P-Array of subkeys - s0, s1, s2, s3 :: Word32 -> Word32 - s0 i = arrayRead32 ar (fromIntegral i + 18) - s1 i = arrayRead32 ar (fromIntegral i + 274) - s2 i = arrayRead32 ar (fromIntegral i + 530) - s3 i = arrayRead32 ar (fromIntegral i + 786) - p :: Int -> Word32 - p i - | inverse = arrayRead32 ar (17 - i) - | otherwise = arrayRead32 ar i +foreign import ccall unsafe "crypton_blowfish_init" + c_blowfish_init :: Ptr Word8 -> Ptr Word8 -> Word32 -> IO () --- | Blowfish encrypt a Word using the current state of the key schedule -cipherBlockMutable :: KeySchedule -> Word64 -> IO Word64 -cipherBlockMutable (KeySchedule ma) input = doRound input 0 - where - -- \| Transform the input over 16 rounds - doRound !i roundIndex - | roundIndex == 16 = do - pVal1 <- mutableArrayRead32 ma 16 - pVal2 <- mutableArrayRead32 ma 17 - let final = (fromIntegral pVal1 `shiftL` 32) .|. fromIntegral pVal2 - return $ rotateL (i `xor` final) 32 - | otherwise = do - pVal <- mutableArrayRead32 ma roundIndex - let newr = fromIntegral (i `shiftR` 32) `xor` pVal - newr' <- f newr - let newi = ((i `shiftL` 32) `xor` newr') .|. fromIntegral newr - doRound newi (roundIndex + 1) +foreign import ccall unsafe "crypton_blowfish_encrypt" + c_blowfish_encrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () - -- \| The Blowfish Feistel function F - f :: Word32 -> IO Word64 - f t = do - a <- s0 (0xff .&. (t `shiftR` 24)) - b <- s1 (0xff .&. (t `shiftR` 16)) - c <- s2 (0xff .&. (t `shiftR` 8)) - d <- s3 (0xff .&. t) - return (fromIntegral (((a + b) `xor` c) + d) `shiftL` 32) +foreign import ccall unsafe "crypton_blowfish_decrypt" + c_blowfish_decrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () - -- \| S-Box arrays, each containing 256 32-bit words - -- The first 18 words contain the P-Array of subkeys - s0, s1, s2, s3 :: Word32 -> IO Word32 - s0 i = mutableArrayRead32 ma (fromIntegral i + 18) - s1 i = mutableArrayRead32 ma (fromIntegral i + 274) - s2 i = mutableArrayRead32 ma (fromIntegral i + 530) - s3 i = mutableArrayRead32 ma (fromIntegral i + 786) +-- the work is what the cost says, so this one may take a while: it is a safe +-- call, which lets the other capabilities carry on while it does +foreign import ccall safe "crypton_bcrypt" + c_bcrypt :: Ptr Word8 -> Word32 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO CInt -iterKeyStream - :: ByteArrayAccess x - => x - -> Word32 - -> Word32 - -> ( Int - -> Word32 - -> Word32 - -> Word32 - -> Word32 - -> (Word32 -> Word32 -> IO ()) - -> IO () - ) +-- | What bcrypt_pbkdf does with Blowfish: the same key setup sixty-four times +-- over, and then the four blocks of its own magic. Writes 32 bytes where it +-- is pointed, which is what the caller of this one wants. +bcryptPbkdfHash + :: (ByteArrayAccess pass, ByteArrayAccess salt) + => pass + -> salt + -> Ptr Word8 -> IO () -iterKeyStream x a0 a1 g = f 0 0 a0 a1 - where - len = B.length x - -- Avoiding the modulo operation when interating over the ring - -- buffer is assumed to be more efficient here. All other - -- implementations do this, too. The branch prediction shall prefer - -- the branch with the increment. - n j = if j + 1 >= len then 0 else j + 1 - f i j0 b0 b1 = g i l r b0 b1 (f (i + 2) j8) - where - j1 = n j0 - j2 = n j1 - j3 = n j2 - j4 = n j3 - j5 = n j4 - j6 = n j5 - j7 = n j6 - j8 = n j7 - x0 = fromIntegral (B.index x j0) - x1 = fromIntegral (B.index x j1) - x2 = fromIntegral (B.index x j2) - x3 = fromIntegral (B.index x j3) - x4 = fromIntegral (B.index x j4) - x5 = fromIntegral (B.index x j5) - x6 = fromIntegral (B.index x j6) - x7 = fromIntegral (B.index x j7) - l = shiftL x0 24 .|. shiftL x1 16 .|. shiftL x2 8 .|. x3 - r = shiftL x4 24 .|. shiftL x5 16 .|. shiftL x6 8 .|. x7 -{-# INLINE iterKeyStream #-} +bcryptPbkdfHash pass salt out = + B.withByteArray pass $ \p -> + B.withByteArray salt $ \s -> do + _ <- + c_bcrypt_pbkdf_hash + out + p + (fromIntegral (B.length pass)) + s + (fromIntegral (B.length salt)) + return () --- Benchmarking shows that GHC considers this function too big to inline --- although forcing inlining causes an actual improvement. --- It is assumed that all function calls (especially the continuation) --- collapse into a tight loop after inlining. +foreign import ccall safe "crypton_bcrypt_pbkdf_hash" + c_bcrypt_pbkdf_hash + :: Ptr Word8 -> Ptr Word8 -> Word32 -> Ptr Word8 -> Word32 -> IO CInt diff --git a/Crypto/Cipher/Camellia/Primitive.hs b/Crypto/Cipher/Camellia/Primitive.hs --- a/Crypto/Cipher/Camellia/Primitive.hs +++ b/Crypto/Cipher/Camellia/Primitive.hs @@ -1,4 +1,4 @@ -{-# LANGUAGE MagicHash #-} +{-# LANGUAGE ForeignFunctionInterface #-} -- | -- Module : Crypto.Cipher.Camellia.Primitive @@ -7,6 +7,8 @@ -- Stability : experimental -- Portability : Good -- +-- Camellia with a 128-bit key, over the C in @cbits/crypton_camellia.c@. +-- -- This only cover Camellia 128 bits for now. The API will change once -- 192 and 256 mode are implemented too. module Crypto.Cipher.Camellia.Primitive ( @@ -16,296 +18,65 @@ decrypt, ) where -import Data.Bits -import Data.Word - import Crypto.Error -import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) +import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B -import Crypto.Internal.WordArray -import Crypto.Internal.Words -import Data.Memory.Endian - -data Mode = Decrypt | Encrypt - -w64tow128 :: (Word64, Word64) -> Word128 -w64tow128 (x1, x2) = Word128 x1 x2 - -w64tow8 :: Word64 -> (Word8, Word8, Word8, Word8, Word8, Word8, Word8, Word8) -w64tow8 x = (t1, t2, t3, t4, t5, t6, t7, t8) - where - t1 = fromIntegral (x `shiftR` 56) - t2 = fromIntegral (x `shiftR` 48) - t3 = fromIntegral (x `shiftR` 40) - t4 = fromIntegral (x `shiftR` 32) - t5 = fromIntegral (x `shiftR` 24) - t6 = fromIntegral (x `shiftR` 16) - t7 = fromIntegral (x `shiftR` 8) - t8 = fromIntegral (x) - -w8tow64 :: (Word8, Word8, Word8, Word8, Word8, Word8, Word8, Word8) -> Word64 -w8tow64 (t1, t2, t3, t4, t5, t6, t7, t8) = - (fromIntegral t1 `shiftL` 56) - .|. (fromIntegral t2 `shiftL` 48) - .|. (fromIntegral t3 `shiftL` 40) - .|. (fromIntegral t4 `shiftL` 32) - .|. (fromIntegral t5 `shiftL` 24) - .|. (fromIntegral t6 `shiftL` 16) - .|. (fromIntegral t7 `shiftL` 8) - .|. (fromIntegral t8) - -sbox :: Int -> Word8 -sbox = arrayRead8 t - where - t = - array8 - "\x70\x82\x2c\xec\xb3\x27\xc0\xe5\xe4\x85\x57\x35\xea\x0c\xae\x41\ - \\x23\xef\x6b\x93\x45\x19\xa5\x21\xed\x0e\x4f\x4e\x1d\x65\x92\xbd\ - \\x86\xb8\xaf\x8f\x7c\xeb\x1f\xce\x3e\x30\xdc\x5f\x5e\xc5\x0b\x1a\ - \\xa6\xe1\x39\xca\xd5\x47\x5d\x3d\xd9\x01\x5a\xd6\x51\x56\x6c\x4d\ - \\x8b\x0d\x9a\x66\xfb\xcc\xb0\x2d\x74\x12\x2b\x20\xf0\xb1\x84\x99\ - \\xdf\x4c\xcb\xc2\x34\x7e\x76\x05\x6d\xb7\xa9\x31\xd1\x17\x04\xd7\ - \\x14\x58\x3a\x61\xde\x1b\x11\x1c\x32\x0f\x9c\x16\x53\x18\xf2\x22\ - \\xfe\x44\xcf\xb2\xc3\xb5\x7a\x91\x24\x08\xe8\xa8\x60\xfc\x69\x50\ - \\xaa\xd0\xa0\x7d\xa1\x89\x62\x97\x54\x5b\x1e\x95\xe0\xff\x64\xd2\ - \\x10\xc4\x00\x48\xa3\xf7\x75\xdb\x8a\x03\xe6\xda\x09\x3f\xdd\x94\ - \\x87\x5c\x83\x02\xcd\x4a\x90\x33\x73\x67\xf6\xf3\x9d\x7f\xbf\xe2\ - \\x52\x9b\xd8\x26\xc8\x37\xc6\x3b\x81\x96\x6f\x4b\x13\xbe\x63\x2e\ - \\xe9\x79\xa7\x8c\x9f\x6e\xbc\x8e\x29\xf5\xf9\xb6\x2f\xfd\xb4\x59\ - \\x78\x98\x06\x6a\xe7\x46\x71\xba\xd4\x25\xab\x42\x88\xa2\x8d\xfa\ - \\x72\x07\xb9\x55\xf8\xee\xac\x0a\x36\x49\x2a\x68\x3c\x38\xf1\xa4\ - \\x40\x28\xd3\x7b\xbb\xc9\x43\xc1\x15\xe3\xad\xf4\x77\xc7\x80\x9e"# - -sbox1 :: Word8 -> Word8 -sbox1 x = sbox (fromIntegral x) - -sbox2 :: Word8 -> Word8 -sbox2 x = sbox1 x `rotateL` 1 - -sbox3 :: Word8 -> Word8 -sbox3 x = sbox1 x `rotateL` 7 - -sbox4 :: Word8 -> Word8 -sbox4 x = sbox1 (x `rotateL` 1) - -sigma1, sigma2, sigma3, sigma4, sigma5, sigma6 :: Word64 -sigma1 = 0xA09E667F3BCC908B -sigma2 = 0xB67AE8584CAA73B2 -sigma3 = 0xC6EF372FE94F82BE -sigma4 = 0x54FF53A5F1D36F1C -sigma5 = 0x10E527FADE682D1D -sigma6 = 0xB05688C2B3E6C1FD - -rotl128 :: Word128 -> Int -> Word128 -rotl128 v 0 = v -rotl128 (Word128 x1 x2) 64 = Word128 x2 x1 -rotl128 v@(Word128 x1 x2) w - | w > 64 = (v `rotl128` 64) `rotl128` (w - 64) - | otherwise = Word128 (x1high .|. x2low) (x2high .|. x1low) - where - splitBits i = (i .&. complement x, i .&. x) - where - x = 2 ^ w - 1 - (x1high, x1low) = splitBits (x1 `rotateL` w) - (x2high, x2low) = splitBits (x2 `rotateL` w) - --- | Camellia context -data Camellia = Camellia - { k :: Array64 - , kw :: Array64 - , ke :: Array64 - } +import Crypto.Internal.Compat (unsafeDoIO) +import Data.Word +import Foreign.Ptr (Ptr) -setKeyInterim - :: ByteArrayAccess key => key -> (Word128, Word128, Word128, Word128) -setKeyInterim keyseed = (w64tow128 kL, w64tow128 kR, w64tow128 kA, w64tow128 kB) - where - kL = (fromBE $ B.toW64BE keyseed 0, fromBE $ B.toW64BE keyseed 8) - kR = (0, 0) +-- | The subkeys of RFC 3713 section 2.2: kw, k and ke, as 26 64-bit words. +newtype Camellia = Camellia Bytes + deriving (Eq) - kA = - let d1 = (fst kL `xor` fst kR) - d2 = (snd kL `xor` snd kR) - d3 = d2 `xor` feistel d1 sigma1 - d4 = d1 `xor` feistel d3 sigma2 - d5 = d4 `xor` (fst kL) - d6 = d3 `xor` (snd kL) - d7 = d6 `xor` feistel d5 sigma3 - d8 = d5 `xor` feistel d7 sigma4 - in (d8, d7) +scheduleSize :: Int +scheduleSize = 26 * 8 - kB = - let d1 = (fst kA `xor` fst kR) - d2 = (snd kA `xor` snd kR) - d3 = d2 `xor` feistel d1 sigma5 - d4 = d1 `xor` feistel d3 sigma6 - in (d4, d3) +blockBytes :: Int +blockBytes = 16 --- | Initialize a 128-bit key --- --- Return the initialized key or a error message if the given --- keyseed was not 16-bytes in length. -initCamellia - :: ByteArray key - => key - -- ^ The key to create the camellia context - -> CryptoFailable Camellia +-- | Initialize a 128-bit key. +initCamellia :: ByteArrayAccess key => key -> CryptoFailable Camellia initCamellia key - | B.length key /= 16 = CryptoFailed $ CryptoError_KeySizeInvalid + | B.length key /= 16 = CryptoFailed CryptoError_KeySizeInvalid | otherwise = - let (kL, _, kA, _) = setKeyInterim key - in let (Word128 kw1 kw2) = (kL `rotl128` 0) - in let (Word128 k1 k2) = (kA `rotl128` 0) - in let (Word128 k3 k4) = (kL `rotl128` 15) - in let (Word128 k5 k6) = (kA `rotl128` 15) - in let (Word128 ke1 ke2) = (kA `rotl128` 30) -- ke1 = (KA <<< 30) >> 64; ke2 = (KA <<< 30) & MASK64; - in let (Word128 k7 k8) = (kL `rotl128` 45) -- k7 = (KL <<< 45) >> 64; k8 = (KL <<< 45) & MASK64; - in let (Word128 k9 _) = (kA `rotl128` 45) -- k9 = (KA <<< 45) >> 64; - in let (Word128 _ k10) = (kL `rotl128` 60) - in let (Word128 k11 k12) = (kA `rotl128` 60) - in let (Word128 ke3 ke4) = (kL `rotl128` 77) - in let (Word128 k13 k14) = (kL `rotl128` 94) - in let (Word128 k15 k16) = (kA `rotl128` 94) - in let (Word128 k17 k18) = (kL `rotl128` 111) - in let (Word128 kw3 kw4) = (kA `rotl128` 111) - in CryptoPassed $ - Camellia - { kw = array64 4 [kw1, kw2, kw3, kw4] - , ke = array64 4 [ke1, ke2, ke3, ke4] - , k = - array64 - 18 - [ k1 - , k2 - , k3 - , k4 - , k5 - , k6 - , k7 - , k8 - , k9 - , k10 - , k11 - , k12 - , k13 - , k14 - , k15 - , k16 - , k17 - , k18 - ] - } - -feistel :: Word64 -> Word64 -> Word64 -feistel fin sk = - let x = fin `xor` sk - in let (t1, t2, t3, t4, t5, t6, t7, t8) = w64tow8 x - in let t1' = sbox1 t1 - in let t2' = sbox2 t2 - in let t3' = sbox3 t3 - in let t4' = sbox4 t4 - in let t5' = sbox2 t5 - in let t6' = sbox3 t6 - in let t7' = sbox4 t7 - in let t8' = sbox1 t8 - in let y1 = t1' `xor` t3' `xor` t4' `xor` t6' `xor` t7' `xor` t8' - in let y2 = t1' `xor` t2' `xor` t4' `xor` t5' `xor` t7' `xor` t8' - in let y3 = t1' `xor` t2' `xor` t3' `xor` t5' `xor` t6' `xor` t8' - in let y4 = t2' `xor` t3' `xor` t4' `xor` t5' `xor` t6' `xor` t7' - in let y5 = t1' `xor` t2' `xor` t6' `xor` t7' `xor` t8' - in let y6 = t2' `xor` t3' `xor` t5' `xor` t7' `xor` t8' - in let y7 = t3' `xor` t4' `xor` t5' `xor` t6' `xor` t8' - in let y8 = t1' `xor` t4' `xor` t5' `xor` t6' `xor` t7' - in w8tow64 (y1, y2, y3, y4, y5, y6, y7, y8) - -fl :: Word64 -> Word64 -> Word64 -fl fin sk = - let (x1, x2) = w64to32 fin - in let (k1, k2) = w64to32 sk - in let y2 = x2 `xor` ((x1 .&. k1) `rotateL` 1) - in let y1 = x1 `xor` (y2 .|. k2) - in w32to64 (y1, y2) - -flinv :: Word64 -> Word64 -> Word64 -flinv fin sk = - let (y1, y2) = w64to32 fin - in let (k1, k2) = w64to32 sk - in let x1 = y1 `xor` (y2 .|. k2) - in let x2 = y2 `xor` ((x1 .&. k1) `rotateL` 1) - in w32to64 (x1, x2) - -{- in decrypt mode 0->17 1->16 ... -} -getKeyK :: Mode -> Camellia -> Int -> Word64 -getKeyK Encrypt key i = k key `arrayRead64` i -getKeyK Decrypt key i = k key `arrayRead64` (17 - i) - -{- in decrypt mode 0->3 1->2 2->1 3->0 -} -getKeyKe :: Mode -> Camellia -> Int -> Word64 -getKeyKe Encrypt key i = ke key `arrayRead64` i -getKeyKe Decrypt key i = ke key `arrayRead64` (3 - i) - -{- in decrypt mode 0->2 1->3 2->0 3->1 -} -getKeyKw :: Mode -> Camellia -> Int -> Word64 -getKeyKw Encrypt key i = (kw key) `arrayRead64` i -getKeyKw Decrypt key i = (kw key) `arrayRead64` ((i + 2) `mod` 4) - -{- perform the following - D2 = D2 ^ F(D1, k1); // Round 1 - D1 = D1 ^ F(D2, k2); // Round 2 - D2 = D2 ^ F(D1, k3); // Round 3 - D1 = D1 ^ F(D2, k4); // Round 4 - D2 = D2 ^ F(D1, k5); // Round 5 - D1 = D1 ^ F(D2, k6); // Round 6 - -} -doBlockRound :: Mode -> Camellia -> Word64 -> Word64 -> Int -> (Word64, Word64) -doBlockRound mode key d1 d2 i = - let r1 = d2 `xor` feistel d1 (getKeyK mode key (0 + i {- Round 1+i -})) - in let r2 = d1 `xor` feistel r1 (getKeyK mode key (1 + i {- Round 2+i -})) - in let r3 = r1 `xor` feistel r2 (getKeyK mode key (2 + i {- Round 3+i -})) - in let r4 = r2 `xor` feistel r3 (getKeyK mode key (3 + i {- Round 4+i -})) - in let r5 = r3 `xor` feistel r4 (getKeyK mode key (4 + i {- Round 5+i -})) - in let r6 = r4 `xor` feistel r5 (getKeyK mode key (5 + i {- Round 6+i -})) - in (r6, r5) - -doBlock :: Mode -> Camellia -> Word128 -> Word128 -doBlock mode key (Word128 d1 d2) = - let d1a = d1 `xor` (getKeyKw mode key 0 {- Prewhitening -}) - in let d2a = d2 `xor` (getKeyKw mode key 1) - in let (d1b, d2b) = doBlockRound mode key d1a d2a 0 - in let d1c = fl d1b (getKeyKe mode key 0 {- FL -}) - in let d2c = flinv d2b (getKeyKe mode key 1 {- FLINV -}) - in let (d1d, d2d) = doBlockRound mode key d1c d2c 6 - in let d1e = fl d1d (getKeyKe mode key 2 {- FL -}) - in let d2e = flinv d2d (getKeyKe mode key 3 {- FLINV -}) - in let (d1f, d2f) = doBlockRound mode key d1e d2e 12 - in let d2g = d2f `xor` (getKeyKw mode key 2 {- Postwhitening -}) - in let d1g = d1f `xor` (getKeyKw mode key 3) - in w64tow128 (d2g, d1g) + CryptoPassed $ + Camellia $ + B.allocAndFreeze scheduleSize $ \ks -> + B.withByteArray key $ \k -> c_camellia_init ks k -{- encryption for 128 bits blocks -} -encryptBlock :: Camellia -> Word128 -> Word128 -encryptBlock = doBlock Encrypt +-- | Encrypt the given input, which has to be a whole number of blocks. +encrypt :: ByteArray ba => Camellia -> ba -> ba +encrypt = run c_camellia_encrypt -{- decryption for 128 bits blocks -} -decryptBlock :: Camellia -> Word128 -> Word128 -decryptBlock = doBlock Decrypt +-- | Decrypt the given input, which has to be a whole number of blocks. +decrypt :: ByteArray ba => Camellia -> ba -> ba +decrypt = run c_camellia_decrypt --- | Encrypts the given ByteString using the given Key -encrypt +run :: ByteArray ba - => Camellia - -- ^ The key to use + => (Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO ()) + -> Camellia -> ba - -- ^ The data to encrypt -> ba -encrypt key = B.mapAsWord128 (encryptBlock key) +run f (Camellia sched) input + | len `mod` blockBytes /= 0 = + error $ + "Crypto.Cipher.Camellia: input length must be a multiple of block size (16). Its length is: " + ++ show len + | otherwise = unsafeDoIO $ + B.alloc len $ \out -> + B.withByteArray sched $ \ks -> + B.withByteArray input $ \inp -> + f out ks inp (fromIntegral (len `div` blockBytes)) + where + len = B.length input --- | Decrypts the given ByteString using the given Key -decrypt - :: ByteArray ba - => Camellia - -- ^ The key to use - -> ba - -- ^ The data to decrypt - -> ba -decrypt key = B.mapAsWord128 (decryptBlock key) +foreign import ccall unsafe "crypton_camellia.h crypton_camellia_init" + c_camellia_init :: Ptr Word8 -> Ptr Word8 -> IO () + +foreign import ccall unsafe "crypton_camellia.h crypton_camellia_encrypt" + c_camellia_encrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () + +foreign import ccall unsafe "crypton_camellia.h crypton_camellia_decrypt" + c_camellia_decrypt :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () diff --git a/Crypto/Cipher/ChaChaPoly1305.hs b/Crypto/Cipher/ChaChaPoly1305.hs --- a/Crypto/Cipher/ChaChaPoly1305.hs +++ b/Crypto/Cipher/ChaChaPoly1305.hs @@ -193,8 +193,12 @@ initFromRootState rootState = State encState polyState 0 0 where (polyKey, encState) = ChaCha.generate rootState 64 + -- 64 bytes are generated so the ChaCha state advances a whole block; the + -- first 32 of them are the key, so the length is right by construction polyState = - throwCryptoError $ Poly1305.initialize (B.take 32 polyKey :: ScrubbedBytes) + Poly1305.initialize $ + throwCryptoError $ + Poly1305.key (B.take 32 polyKey :: ScrubbedBytes) -- | Initialize a new XChaChaPoly1305 State -- diff --git a/Crypto/Cipher/DES.hs b/Crypto/Cipher/DES.hs --- a/Crypto/Cipher/DES.hs +++ b/Crypto/Cipher/DES.hs @@ -4,6 +4,9 @@ -- Maintainer : Vincent Hanquez -- Stability : stable -- Portability : good +-- +-- DES, which is here because callers still meet it rather than because it +-- should be chosen: its 56-bit key is exhaustible. Prefer "Crypto.Cipher.AES". module Crypto.Cipher.DES ( DES, ) where @@ -13,11 +16,9 @@ import Crypto.Error import Crypto.Internal.ByteArray (ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B -import Data.Memory.Endian -import Data.Word -- | DES Context -data DES = DES Word64 +data DES = DES Schedule Schedule deriving (Eq) instance Cipher DES where @@ -27,13 +28,11 @@ instance BlockCipher DES where blockSize _ = 8 - ecbEncrypt (DES key) = B.mapAsWord64 (unBlock . encrypt key . Block) - ecbDecrypt (DES key) = B.mapAsWord64 (unBlock . decrypt key . Block) + ecbEncrypt (DES enc _) = ecb enc + ecbDecrypt (DES _ dec) = ecb dec initDES :: ByteArrayAccess key => key -> CryptoFailable DES initDES k - | len == 8 = CryptoPassed $ DES key - | otherwise = CryptoFailed $ CryptoError_KeySizeInvalid - where - len = B.length k - key = fromBE $ B.toW64BE k 0 + | B.length k == 8 = + CryptoPassed $ DES (schedule [(Encrypt, k)]) (schedule [(Decrypt, k)]) + | otherwise = CryptoFailed CryptoError_KeySizeInvalid diff --git a/Crypto/Cipher/DES/Primitive.hs b/Crypto/Cipher/DES/Primitive.hs --- a/Crypto/Cipher/DES/Primitive.hs +++ b/Crypto/Cipher/DES/Primitive.hs @@ -1,570 +1,83 @@ -{-# LANGUAGE FlexibleInstances #-} - ------------------------------------------------------------------------------ - ------------------------------------------------------------------------------ +{-# LANGUAGE ForeignFunctionInterface #-} -- | --- Module : Crypto.Cipher.DES.Primitive --- License : BSD-style +-- Module : Crypto.Cipher.DES.Primitive +-- License : BSD-style +-- Stability : experimental +-- Portability : Good -- --- This module is copy of DES module from Crypto package. --- http://hackage.haskell.org/package/Crypto +-- The DES block operation, as FIPS 46-3 defines it, over the C in +-- @cbits/crypton_des.c@. +-- +-- A t'Schedule' holds the round keys of one or more stages in the order they +-- are applied, which is what lets single DES and the three stage constructions +-- share one entry point. module Crypto.Cipher.DES.Primitive ( - encrypt, - decrypt, - Block (..), + Schedule, + Direction (..), + schedule, + ecb, ) where -import Data.Bits +import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) +import qualified Crypto.Internal.ByteArray as B +import Crypto.Internal.Compat (unsafeDoIO) import Data.Word - --- | a DES block (64 bits) -newtype Block = Block {unBlock :: Word64} - -type Rotation = Int -type Key = Word64 - -type Bits4 = [Bool] -type Bits6 = [Bool] -type Bits32 = [Bool] -type Bits48 = [Bool] -type Bits56 = [Bool] -type Bits64 = [Bool] - -desXor :: [Bool] -> [Bool] -> [Bool] -desXor a b = zipWith (/=) a b - -desRotate :: [Bool] -> Int -> [Bool] -desRotate bits rot = drop rot' bits ++ take rot' bits - where - rot' = rot `mod` length bits - -bitify :: Word64 -> Bits64 -bitify w = map (\b -> w .&. (shiftL 1 b) /= 0) [63, 62 .. 0] - -unbitify :: Bits64 -> Word64 -unbitify bs = foldl (\i b -> if b then 1 + shiftL i 1 else shiftL i 1) 0 bs - -initial_permutation :: Bits64 -> Bits64 -initial_permutation mb = map ((!!) mb) i - where - i = - [ 57 - , 49 - , 41 - , 33 - , 25 - , 17 - , 9 - , 1 - , 59 - , 51 - , 43 - , 35 - , 27 - , 19 - , 11 - , 3 - , 61 - , 53 - , 45 - , 37 - , 29 - , 21 - , 13 - , 5 - , 63 - , 55 - , 47 - , 39 - , 31 - , 23 - , 15 - , 7 - , 56 - , 48 - , 40 - , 32 - , 24 - , 16 - , 8 - , 0 - , 58 - , 50 - , 42 - , 34 - , 26 - , 18 - , 10 - , 2 - , 60 - , 52 - , 44 - , 36 - , 28 - , 20 - , 12 - , 4 - , 62 - , 54 - , 46 - , 38 - , 30 - , 22 - , 14 - , 6 - ] - -{- -"\x39\x31\x29\x21\x19\x11\x09\x01\x3b\x33\x2b\x23\x1b\x13\ -\\x0b\x03\x3d\x35\x2d\x25\x1d\x15\x0d\x05\x3f\x37\x2f\x27\ -\\x1f\x17\x0f\x07\x38\x30\x28\x20\x18\x10\x08\x00\x3a\x32\ -\\x2a\x22\x1a\x12\x0a\x02\x3c\x34\x2c\x24\x1c\x14\x0c\x04\ -\\x3e\x36\x2e\x26\x1e\x16\x0e\x06" --} - -key_transformation :: Bits64 -> Bits56 -key_transformation kb = map ((!!) kb) i - where - i = - [ 56 - , 48 - , 40 - , 32 - , 24 - , 16 - , 8 - , 0 - , 57 - , 49 - , 41 - , 33 - , 25 - , 17 - , 9 - , 1 - , 58 - , 50 - , 42 - , 34 - , 26 - , 18 - , 10 - , 2 - , 59 - , 51 - , 43 - , 35 - , 62 - , 54 - , 46 - , 38 - , 30 - , 22 - , 14 - , 6 - , 61 - , 53 - , 45 - , 37 - , 29 - , 21 - , 13 - , 5 - , 60 - , 52 - , 44 - , 36 - , 28 - , 20 - , 12 - , 4 - , 27 - , 19 - , 11 - , 3 - ] - -{- -"\x38\x30\x28\x20\x18\x10\x08\x00\x39\x31\x29\x21\x19\x11\ -\\x09\x01\x3a\x32\x2a\x22\x1a\x12\x0a\x02\x3b\x33\x2b\x23\ -\\x3e\x36\x2e\x26\x1e\x16\x0e\x06\x3d\x35\x2d\x25\x1d\x15\ -\\x0d\x05\x3c\x34\x2c\x24\x1c\x14\x0c\x04\x1b\x13\x0b\x03" --} - -des_enc :: Block -> Key -> Block -des_enc = do_des [1, 2, 4, 6, 8, 10, 12, 14, 15, 17, 19, 21, 23, 25, 27, 28] - -des_dec :: Block -> Key -> Block -des_dec = do_des [28, 27, 25, 23, 21, 19, 17, 15, 14, 12, 10, 8, 6, 4, 2, 1] - -do_des :: [Rotation] -> Block -> Key -> Block -do_des rots (Block m) k = Block $ des_work rots (takeDrop 32 mb) kb - where - kb = key_transformation $ bitify k - mb = initial_permutation $ bitify m - -des_work :: [Rotation] -> (Bits32, Bits32) -> Bits56 -> Word64 -des_work [] (ml, mr) _ = unbitify $ final_perm $ (mr ++ ml) -des_work (r : rs) mb kb = des_work rs mb' kb - where - mb' = do_round r mb kb - -do_round :: Rotation -> (Bits32, Bits32) -> Bits56 -> (Bits32, Bits32) -do_round r (ml, mr) kb = (mr, m') - where - kb' = get_key kb r - comp_kb = compression_permutation kb' - expa_mr = expansion_permutation mr - res = comp_kb `desXor` expa_mr - res' = drop 1 $ iterate (trans 6) ([], res) - trans n (_, b) = (take n b, drop n b) - res_s = - concat $ - zipWith - (\f (x, _) -> f x) - [ s_box_1 - , s_box_2 - , s_box_3 - , s_box_4 - , s_box_5 - , s_box_6 - , s_box_7 - , s_box_8 - ] - res' - res_p = p_box res_s - m' = res_p `desXor` ml - -get_key :: Bits56 -> Rotation -> Bits56 -get_key kb r = kb' - where - (kl, kr) = takeDrop 28 kb - kb' = desRotate kl r ++ desRotate kr r - -compression_permutation :: Bits56 -> Bits48 -compression_permutation kb = map ((!!) kb) i - where - i = - [ 13 - , 16 - , 10 - , 23 - , 0 - , 4 - , 2 - , 27 - , 14 - , 5 - , 20 - , 9 - , 22 - , 18 - , 11 - , 3 - , 25 - , 7 - , 15 - , 6 - , 26 - , 19 - , 12 - , 1 - , 40 - , 51 - , 30 - , 36 - , 46 - , 54 - , 29 - , 39 - , 50 - , 44 - , 32 - , 47 - , 43 - , 48 - , 38 - , 55 - , 33 - , 52 - , 45 - , 41 - , 49 - , 35 - , 28 - , 31 - ] - -expansion_permutation :: Bits32 -> Bits48 -expansion_permutation mb = map ((!!) mb) i - where - i = - [ 31 - , 0 - , 1 - , 2 - , 3 - , 4 - , 3 - , 4 - , 5 - , 6 - , 7 - , 8 - , 7 - , 8 - , 9 - , 10 - , 11 - , 12 - , 11 - , 12 - , 13 - , 14 - , 15 - , 16 - , 15 - , 16 - , 17 - , 18 - , 19 - , 20 - , 19 - , 20 - , 21 - , 22 - , 23 - , 24 - , 23 - , 24 - , 25 - , 26 - , 27 - , 28 - , 27 - , 28 - , 29 - , 30 - , 31 - , 0 - ] - -s_box :: [[Word8]] -> Bits6 -> Bits4 -s_box s [a, b, c, d, e, f] = to_bool 4 $ (s !! row) !! col - where - row = sum $ zipWith numericise [a, f] [1, 0] - col = sum $ zipWith numericise [b, c, d, e] [3, 2, 1, 0] - numericise :: Bool -> Int -> Int - numericise = (\x y -> if x then 2 ^ y else 0) - - to_bool :: Int -> Word8 -> [Bool] - to_bool 0 _ = [] - to_bool n i = ((i .&. 8) == 8) : to_bool (n - 1) (shiftL i 1) -s_box _ _ = error "DES: internal error bits6 more than 6 elements" - -s_box_1 :: Bits6 -> Bits4 -s_box_1 = s_box i - where - i = - [ [14, 4, 13, 1, 2, 15, 11, 8, 3, 10, 6, 12, 5, 9, 0, 7] - , [0, 15, 7, 4, 14, 2, 13, 1, 10, 6, 12, 11, 9, 5, 3, 8] - , [4, 1, 14, 8, 13, 6, 2, 11, 15, 12, 9, 7, 3, 10, 5, 0] - , [15, 12, 8, 2, 4, 9, 1, 7, 5, 11, 3, 14, 10, 0, 6, 13] - ] - -s_box_2 :: Bits6 -> Bits4 -s_box_2 = s_box i - where - i = - [ [15, 1, 8, 14, 6, 11, 3, 4, 9, 7, 2, 13, 12, 0, 5, 10] - , [3, 13, 4, 7, 15, 2, 8, 14, 12, 0, 1, 10, 6, 9, 11, 5] - , [0, 14, 7, 11, 10, 4, 13, 1, 5, 8, 12, 6, 9, 3, 2, 15] - , [13, 8, 10, 1, 3, 15, 4, 2, 11, 6, 7, 12, 0, 5, 14, 9] - ] - -s_box_3 :: Bits6 -> Bits4 -s_box_3 = s_box i - where - i = - [ [10, 0, 9, 14, 6, 3, 15, 5, 1, 13, 12, 7, 11, 4, 2, 8] - , [13, 7, 0, 9, 3, 4, 6, 10, 2, 8, 5, 14, 12, 11, 15, 1] - , [13, 6, 4, 9, 8, 15, 3, 0, 11, 1, 2, 12, 5, 10, 14, 7] - , [1, 10, 13, 0, 6, 9, 8, 7, 4, 15, 14, 3, 11, 5, 2, 12] - ] - -s_box_4 :: Bits6 -> Bits4 -s_box_4 = s_box i - where - i = - [ [7, 13, 14, 3, 0, 6, 9, 10, 1, 2, 8, 5, 11, 12, 4, 15] - , [13, 8, 11, 5, 6, 15, 0, 3, 4, 7, 2, 12, 1, 10, 14, 9] - , [10, 6, 9, 0, 12, 11, 7, 13, 15, 1, 3, 14, 5, 2, 8, 4] - , [3, 15, 0, 6, 10, 1, 13, 8, 9, 4, 5, 11, 12, 7, 2, 14] - ] - -s_box_5 :: Bits6 -> Bits4 -s_box_5 = s_box i - where - i = - [ [2, 12, 4, 1, 7, 10, 11, 6, 8, 5, 3, 15, 13, 0, 14, 9] - , [14, 11, 2, 12, 4, 7, 13, 1, 5, 0, 15, 10, 3, 9, 8, 6] - , [4, 2, 1, 11, 10, 13, 7, 8, 15, 9, 12, 5, 6, 3, 0, 14] - , [11, 8, 12, 7, 1, 14, 2, 13, 6, 15, 0, 9, 10, 4, 5, 3] - ] +import Foreign.C.Types (CInt (..)) +import Foreign.Ptr (Ptr, plusPtr) -s_box_6 :: Bits6 -> Bits4 -s_box_6 = s_box i - where - i = - [ [12, 1, 10, 15, 9, 2, 6, 8, 0, 13, 3, 4, 14, 7, 5, 11] - , [10, 15, 4, 2, 7, 12, 9, 5, 6, 1, 13, 14, 0, 11, 3, 8] - , [9, 14, 15, 5, 2, 8, 12, 3, 7, 0, 4, 10, 1, 13, 11, 6] - , [4, 3, 2, 12, 9, 5, 15, 10, 11, 14, 1, 7, 6, 0, 8, 13] - ] +-- | Which way a stage runs. +data Direction = Encrypt | Decrypt + deriving (Show, Eq) -s_box_7 :: Bits6 -> Bits4 -s_box_7 = s_box i - where - i = - [ [4, 11, 2, 14, 15, 0, 8, 13, 3, 12, 9, 7, 5, 10, 6, 1] - , [13, 0, 11, 7, 4, 9, 1, 10, 14, 3, 5, 12, 2, 15, 8, 6] - , [1, 4, 11, 13, 12, 3, 7, 14, 10, 15, 6, 8, 0, 5, 9, 2] - , [6, 11, 13, 8, 1, 4, 10, 7, 9, 5, 0, 15, 14, 2, 3, 12] - ] +-- | The round keys of one or more stages, in the order they are applied. +newtype Schedule = Schedule Bytes + deriving (Eq) -s_box_8 :: Bits6 -> Bits4 -s_box_8 = s_box i - where - i = - [ [13, 2, 8, 4, 6, 15, 11, 1, 10, 9, 3, 14, 5, 0, 12, 7] - , [1, 15, 13, 8, 10, 3, 7, 4, 12, 5, 6, 11, 0, 14, 9, 2] - , [7, 11, 4, 1, 9, 12, 14, 2, 0, 6, 10, 13, 15, 3, 5, 8] - , [2, 1, 14, 7, 4, 10, 8, 13, 15, 12, 9, 0, 3, 5, 6, 11] - ] +-- | Bytes per stage: sixteen rounds of eight six-bit values. +stageSize :: Int +stageSize = 16 * 8 -p_box :: Bits32 -> Bits32 -p_box kb = map ((!!) kb) i - where - i = - [ 15 - , 6 - , 19 - , 20 - , 28 - , 11 - , 27 - , 16 - , 0 - , 14 - , 22 - , 25 - , 4 - , 17 - , 30 - , 9 - , 1 - , 7 - , 23 - , 13 - , 31 - , 26 - , 2 - , 8 - , 18 - , 12 - , 29 - , 5 - , 21 - , 10 - , 3 - , 24 - ] +-- | The block size DES works in. +blockBytes :: Int +blockBytes = 8 -final_perm :: Bits64 -> Bits64 -final_perm kb = map ((!!) kb) i +-- | Build the schedule for a sequence of stages, each an eight byte key and +-- the direction that stage runs in. Shorter keys are rejected by the callers, +-- which know their own size; the bytes past the eighth are not read. +schedule :: ByteArrayAccess key => [(Direction, key)] -> Schedule +schedule stages = + Schedule $ B.allocAndFreeze (stageSize * length stages) $ \dst -> + mapM_ (uncurry (one dst)) (zip [0 ..] stages) where - i = - [ 39 - , 7 - , 47 - , 15 - , 55 - , 23 - , 63 - , 31 - , 38 - , 6 - , 46 - , 14 - , 54 - , 22 - , 62 - , 30 - , 37 - , 5 - , 45 - , 13 - , 53 - , 21 - , 61 - , 29 - , 36 - , 4 - , 44 - , 12 - , 52 - , 20 - , 60 - , 28 - , 35 - , 3 - , 43 - , 11 - , 51 - , 19 - , 59 - , 27 - , 34 - , 2 - , 42 - , 10 - , 50 - , 18 - , 58 - , 26 - , 33 - , 1 - , 41 - , 9 - , 49 - , 17 - , 57 - , 25 - , 32 - , 0 - , 40 - , 8 - , 48 - , 16 - , 56 - , 24 - ] + one dst i (dir, key) = + B.withByteArray key $ \k -> + c_des_init (dst `plusPtr` (i * stageSize)) k (reverseFlag dir) + reverseFlag Encrypt = 0 + reverseFlag Decrypt = 1 -takeDrop :: Int -> [a] -> ([a], [a]) -takeDrop _ [] = ([], []) -takeDrop 0 xs = ([], xs) -takeDrop n (x : xs) = (x : ys, zs) +-- | Apply every stage of the schedule, in order, to each block of the input. +ecb :: ByteArray ba => Schedule -> ba -> ba +ecb (Schedule sched) input + | len `mod` blockBytes /= 0 = + error $ + "Crypto.Cipher.DES: input length must be a multiple of block size (8). Its length is: " + ++ show len + | otherwise = unsafeDoIO $ + B.alloc len $ \out -> + B.withByteArray sched $ \ks -> + B.withByteArray input $ \inp -> + c_des_ecb + out + ks + (fromIntegral (B.length sched `div` stageSize)) + inp + (fromIntegral (len `div` blockBytes)) where - (ys, zs) = takeDrop (n - 1) xs + len = B.length input --- | Basic DES encryption which takes a key and a block of plaintext --- and returns the encrypted block of ciphertext according to the standard. -encrypt :: Word64 -> Block -> Block -encrypt = flip des_enc +foreign import ccall unsafe "crypton_des.h crypton_des_init" + c_des_init :: Ptr Word8 -> Ptr Word8 -> CInt -> IO () --- | Basic DES decryption which takes a key and a block of ciphertext and --- returns the decrypted block of plaintext according to the standard. -decrypt :: Word64 -> Block -> Block -decrypt = flip des_dec +foreign import ccall unsafe "crypton_des.h crypton_des_ecb" + c_des_ecb :: Ptr Word8 -> Ptr Word8 -> Word32 -> Ptr Word8 -> Word32 -> IO () diff --git a/Crypto/Cipher/TripleDES.hs b/Crypto/Cipher/TripleDES.hs --- a/Crypto/Cipher/TripleDES.hs +++ b/Crypto/Cipher/TripleDES.hs @@ -13,82 +13,104 @@ import Crypto.Cipher.DES.Primitive import Crypto.Cipher.Types import Crypto.Error -import Crypto.Internal.ByteArray (ByteArrayAccess) +import Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes) import qualified Crypto.Internal.ByteArray as B -import Data.Memory.Endian -import Data.Word -- | 3DES with 3 different keys used all in the same direction -data DES_EEE3 = DES_EEE3 Word64 Word64 Word64 +data DES_EEE3 = DES_EEE3 Schedule Schedule deriving (Eq) -- | 3DES with 3 different keys used in alternative direction -data DES_EDE3 = DES_EDE3 Word64 Word64 Word64 +data DES_EDE3 = DES_EDE3 Schedule Schedule deriving (Eq) -- | 3DES where the first and third keys are equal, used in the same direction -data DES_EEE2 = DES_EEE2 Word64 Word64 -- key1 and key3 are equal +data DES_EEE2 = DES_EEE2 Schedule Schedule deriving (Eq) -- | 3DES where the first and third keys are equal, used in alternative direction -data DES_EDE2 = DES_EDE2 Word64 Word64 -- key1 and key3 are equal +data DES_EDE2 = DES_EDE2 Schedule Schedule deriving (Eq) instance Cipher DES_EEE3 where cipherName _ = "3DES_EEE" cipherKeySize _ = KeySizeFixed 24 - cipherInit k = init3DES DES_EEE3 k + cipherInit k = init3DES DES_EEE3 Encrypt k instance Cipher DES_EDE3 where cipherName _ = "3DES_EDE" cipherKeySize _ = KeySizeFixed 24 - cipherInit k = init3DES DES_EDE3 k + cipherInit k = init3DES DES_EDE3 Decrypt k instance Cipher DES_EDE2 where cipherName _ = "2DES_EDE" cipherKeySize _ = KeySizeFixed 16 - cipherInit k = init2DES DES_EDE2 k + cipherInit k = init2DES DES_EDE2 Decrypt k instance Cipher DES_EEE2 where cipherName _ = "2DES_EEE" cipherKeySize _ = KeySizeFixed 16 - cipherInit k = init2DES DES_EEE2 k + cipherInit k = init2DES DES_EEE2 Encrypt k instance BlockCipher DES_EEE3 where blockSize _ = 8 - ecbEncrypt (DES_EEE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (encrypt k3 . encrypt k2 . encrypt k1) . Block) - ecbDecrypt (DES_EEE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (decrypt k1 . decrypt k2 . decrypt k3) . Block) + ecbEncrypt (DES_EEE3 enc _) = ecb enc + ecbDecrypt (DES_EEE3 _ dec) = ecb dec instance BlockCipher DES_EDE3 where blockSize _ = 8 - ecbEncrypt (DES_EDE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (encrypt k3 . decrypt k2 . encrypt k1) . Block) - ecbDecrypt (DES_EDE3 k1 k2 k3) = B.mapAsWord64 (unBlock . (decrypt k1 . encrypt k2 . decrypt k3) . Block) + ecbEncrypt (DES_EDE3 enc _) = ecb enc + ecbDecrypt (DES_EDE3 _ dec) = ecb dec instance BlockCipher DES_EEE2 where blockSize _ = 8 - ecbEncrypt (DES_EEE2 k1 k2) = B.mapAsWord64 (unBlock . (encrypt k1 . encrypt k2 . encrypt k1) . Block) - ecbDecrypt (DES_EEE2 k1 k2) = B.mapAsWord64 (unBlock . (decrypt k1 . decrypt k2 . decrypt k1) . Block) + ecbEncrypt (DES_EEE2 enc _) = ecb enc + ecbDecrypt (DES_EEE2 _ dec) = ecb dec instance BlockCipher DES_EDE2 where blockSize _ = 8 - ecbEncrypt (DES_EDE2 k1 k2) = B.mapAsWord64 (unBlock . (encrypt k1 . decrypt k2 . encrypt k1) . Block) - ecbDecrypt (DES_EDE2 k1 k2) = B.mapAsWord64 (unBlock . (decrypt k1 . encrypt k2 . decrypt k1) . Block) + ecbEncrypt (DES_EDE2 enc _) = ecb enc + ecbDecrypt (DES_EDE2 _ dec) = ecb dec +-- | The schedules of a three stage cipher, for both directions. +-- +-- The outer stages encrypt and the middle one goes whichever way the +-- construction says; decrypting is the same three stages in the opposite +-- order, each the other way round. +stages + :: ByteArrayAccess key + => Direction + -- ^ the direction of the middle stage when encrypting + -> (key, key, key) + -> (Schedule, Schedule) +stages mid (k1, k2, k3) = + ( schedule [(Encrypt, k1), (mid, k2), (Encrypt, k3)] + , schedule [(Decrypt, k3), (opposite mid, k2), (Decrypt, k1)] + ) + where + opposite Encrypt = Decrypt + opposite Decrypt = Encrypt + init3DES :: ByteArrayAccess key - => (Word64 -> Word64 -> Word64 -> a) -> key -> CryptoFailable a -init3DES constr k - | len == 24 = CryptoPassed $ constr k1 k2 k3 + => (Schedule -> Schedule -> a) -> Direction -> key -> CryptoFailable a +init3DES constr mid k + | B.length k == 24 = + CryptoPassed $ uncurry constr $ stages mid (part 0, part 8, part 16) | otherwise = CryptoFailed CryptoError_KeySizeInvalid where - len = B.length k - (k1, k2, k3) = (fromBE $ B.toW64BE k 0, fromBE $ B.toW64BE k 8, fromBE $ B.toW64BE k 16) + part = keyPart k init2DES - :: ByteArrayAccess key => (Word64 -> Word64 -> a) -> key -> CryptoFailable a -init2DES constr k - | len == 16 = CryptoPassed $ constr k1 k2 + :: ByteArrayAccess key + => (Schedule -> Schedule -> a) -> Direction -> key -> CryptoFailable a +init2DES constr mid k + | B.length k == 16 = + CryptoPassed $ uncurry constr $ stages mid (part 0, part 8, part 0) | otherwise = CryptoFailed CryptoError_KeySizeInvalid where - len = B.length k - (k1, k2) = (fromBE $ B.toW64BE k 0, fromBE $ B.toW64BE k 8) + part = keyPart k + +-- | The eight bytes of a key that start at the given offset. +keyPart :: ByteArrayAccess key => key -> Int -> ScrubbedBytes +keyPart k i = B.take 8 $ B.drop i (B.convert k :: ScrubbedBytes) diff --git a/Crypto/Cipher/Twofish/Primitive.hs b/Crypto/Cipher/Twofish/Primitive.hs --- a/Crypto/Cipher/Twofish/Primitive.hs +++ b/Crypto/Cipher/Twofish/Primitive.hs @@ -13,6 +13,7 @@ import Crypto.Internal.ByteArray (ByteArray) import qualified Crypto.Internal.ByteArray as B import Crypto.Internal.WordArray +import Crypto.Internal.Words (Word128 (..)) import Data.Bits import Data.List (foldl') import Data.Word @@ -65,13 +66,34 @@ generatedK = array32 40 $ genK keyPackage generatedS = genSboxes keyPackage $ sWords key -mapBlocks :: ByteArray ba => (ba -> ba) -> ba -> ba +-- | Run a block operation over every block of the input. +-- +-- 'B.mapAsWord128' walks the input and the output once each, where taking a +-- block off the front and appending the result copied the whole of both, once +-- per block. +mapBlocks :: ByteArray ba => (Word128 -> Word128) -> ba -> ba mapBlocks operation input - | B.null rest = blockOutput - | otherwise = blockOutput `B.append` mapBlocks operation rest + | B.length input `mod` blockSize /= 0 = + error $ + "Crypto.Cipher.Twofish: input length must be a multiple of block size (16). Its length is: " + ++ show (B.length input) + | otherwise = B.mapAsWord128 operation input + +-- | The four little-endian words of a block, from the two big-endian words +-- t'Word128' is read as. +load32ls :: Word128 -> (Word32, Word32, Word32, Word32) +load32ls (Word128 hi lo) = + ( byteSwap32 (fromIntegral (hi `shiftR` 32)) + , byteSwap32 (fromIntegral hi) + , byteSwap32 (fromIntegral (lo `shiftR` 32)) + , byteSwap32 (fromIntegral lo) + ) + +store32ls :: (Word32, Word32, Word32, Word32) -> Word128 +store32ls (a, b, c, d) = Word128 (pair a b) (pair c d) where - (block, rest) = B.splitAt blockSize input - blockOutput = operation block + pair x y = + (fromIntegral (byteSwap32 x) `shiftL` 32) .|. fromIntegral (byteSwap32 y) -- | Encrypts the given ByteString using the given Key encrypt @@ -83,7 +105,7 @@ -> ba encrypt cipher = mapBlocks (encryptBlock cipher) -encryptBlock :: ByteArray ba => Twofish -> ba -> ba +encryptBlock :: Twofish -> Word128 -> Word128 encryptBlock Twofish{s = (s1, s2, s3, s4), k = ks} message = store32ls ts where (a, b, c, d) = load32ls message @@ -150,7 +172,7 @@ decrypt cipher = mapBlocks (decryptBlock cipher) {- decryption for 128 bits blocks -} -decryptBlock :: ByteArray ba => Twofish -> ba -> ba +decryptBlock :: Twofish -> Word128 -> Word128 decryptBlock Twofish{s = (s1, s2, s3, s4), k = ks} message = store32ls ixs where (a, b, c, d) = load32ls message @@ -251,26 +273,6 @@ , [0x02, 0xA1, 0xFC, 0xC1, 0x47, 0xAE, 0x3D, 0x19] , [0xA4, 0x55, 0x87, 0x5A, 0x58, 0xDB, 0x9E, 0x03] ] - -load32ls :: ByteArray ba => ba -> (Word32, Word32, Word32, Word32) -load32ls message = (intify q1, intify q2, intify q3, intify q4) - where - (half1, half2) = B.splitAt 8 message - (q1, q2) = B.splitAt 4 half1 - (q3, q4) = B.splitAt 4 half2 - - intify :: ByteArray ba => ba -> Word32 - intify bytes = - foldl' - (\int (!word, !ind) -> int .|. shiftL (fromIntegral word) (ind * 8)) - 0 - (zip (B.unpack bytes) [0 ..]) - -store32ls :: ByteArray ba => (Word32, Word32, Word32, Word32) -> ba -store32ls (a, b, c, d) = B.pack $ concatMap splitWordl [a, b, c, d] - where - splitWordl :: Word32 -> [Word8] - splitWordl w = fmap (\ind -> fromIntegral $ shiftR w (8 * ind)) [0 .. 3] -- Create S words sWords :: ByteArray ba => ba -> [Word8] diff --git a/Crypto/Cipher/Types/AEAD.hs b/Crypto/Cipher/Types/AEAD.hs --- a/Crypto/Cipher/Types/AEAD.hs +++ b/Crypto/Cipher/Types/AEAD.hs @@ -105,3 +105,37 @@ aead = aeadAppendHeader aeadIni header (output, aeadFinal) = aeadDecrypt aead input tag = aeadFinalize aeadFinal (B.length authTag) + +-- | Simple AEAD decryption with the tag length given by the caller. +-- +-- 'aeadSimpleDecrypt' authenticates as many octets as the tag it is handed is +-- long. That is the caller's choice only for as long as the tag is: one read +-- off the wire is the peer's, and an attacker who truncates it picks how much +-- of it gets verified, down to 'minimumTagLength'. +-- +-- Here the length is a separate argument and a tag that is not exactly that +-- long is refused before anything is compared, so the peer cannot weaken the +-- check. Prefer this wherever the tag is attacker reachable. +tryAeadSimpleDecrypt + :: (ByteArrayAccess aad, ByteArray ba) + => AEAD a + -- ^ An AEAD Context + -> aad + -- ^ Associated\/additional data + -> ba + -- ^ Ciphertext + -> Int + -- ^ The tag length to authenticate, which the tag must match + -> AuthTag + -- ^ The authentication tag + -> Maybe ba + -- ^ Plaintext +tryAeadSimpleDecrypt aeadIni header input taglen authTag + | taglen < minimumTagLength = Nothing + | B.length authTag /= taglen = Nothing + | tag == authTag = Just output + | otherwise = Nothing + where + aead = aeadAppendHeader aeadIni header + (output, aeadFinal) = aeadDecrypt aead input + tag = aeadFinalize aeadFinal taglen diff --git a/Crypto/Cipher/Types/Block.hs b/Crypto/Cipher/Types/Block.hs --- a/Crypto/Cipher/Types/Block.hs +++ b/Crypto/Cipher/Types/Block.hs @@ -43,7 +43,6 @@ import Crypto.Cipher.Types.AEAD import Crypto.Cipher.Types.Base import Crypto.Cipher.Types.GF -import Crypto.Cipher.Types.Utils import Crypto.Error import Data.Word @@ -54,7 +53,10 @@ withByteArray, ) import qualified Crypto.Internal.ByteArray as B +import Data.ByteString (ByteString) +import qualified Data.ByteString as S +import Foreign.Marshal.Utils (copyBytes) import Foreign.Ptr import Foreign.Storable @@ -211,49 +213,131 @@ cbcEncryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba -cbcEncryptGeneric cipher ivini input = mconcat $ doEnc ivini $ chunk (blockSize cipher) input +cbcEncryptGeneric cipher ivini input = + B.concat $ doEnc ivini $ slices (blockSize cipher) input where + -- the blocks of the message as shared slices rather than copies: each + -- block already costs an exclusive or and a call into the cipher, both of + -- which allocate, and the chain makes it one block at a time doEnc _ [] = [] doEnc iv (i : is) = - let o = ecbEncrypt cipher $ B.xor iv i + let o = ecbEncrypt cipher (B.bxor iv i) `asTypeOf` input in o : doEnc (IV o) is +-- | How many blocks to hand the cipher at a time in the modes whose blocks do +-- not depend on one another. Enough that the cost of a call disappears, few +-- enough that what it copies stays in cache. +blocksPerCall :: Int +blocksPerCall = 2048 + +-- | The input in slices of that many blocks. A ByteString shares where +-- 'B.splitAt' copies the rest of the message, once per slice. +slices :: ByteArray ba => Int -> ba -> [ByteString] +slices bytes input = go (B.convert input) + where + go bs + | S.null bs = [] + | otherwise = let (hd, tl) = S.splitAt bytes bs in hd : go tl + +-- | The previous ciphertext block of every block in a slice: the incoming IV, +-- and then the slice itself one block short. +shiftedBy :: BlockCipher cipher => Int -> IV cipher -> ByteString -> ByteString +shiftedBy bsz iv c = S.append (B.convert iv) (S.take (S.length c - bsz) c) + +-- | The last whole block of a slice, which is where the next one carries on +-- from. +lastBlockOf :: Int -> ByteString -> IV cipher +lastBlockOf bsz c = IV (B.convert (S.drop (S.length c - bsz) c) :: Bytes) + +-- | Decryption does not chain: @P_i@ is @D(C_i)@ exclusive-ored with +-- @C_(i-1)@, so a whole slice is decrypted in one call and exclusive-ored with +-- the ciphertext moved along by a block. cbcDecryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba -cbcDecryptGeneric cipher ivini input = mconcat $ doDec ivini $ chunk (blockSize cipher) input +cbcDecryptGeneric cipher ivini input = + B.concat $ doDec ivini $ slices (blocksPerCall * bsz) input where + bsz = blockSize cipher + conv x = B.convert x `asTypeOf` input + xorB a b = B.bxor a b `asTypeOf` input doDec _ [] = [] - doDec iv (i : is) = - let o = B.xor iv $ ecbDecrypt cipher i - in o : doDec (IV i) is + doDec iv (c : cs) = + xorB (ecbDecrypt cipher (conv c)) (conv (shiftedBy bsz iv c)) + : doDec (lastBlockOf bsz c) cs cfbEncryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba -cfbEncryptGeneric cipher ivini input = mconcat $ doEnc ivini $ chunk (blockSize cipher) input +cfbEncryptGeneric cipher ivini input = + B.concat $ doEnc ivini $ slices (blockSize cipher) input where doEnc _ [] = [] doEnc (IV iv) (i : is) = - let o = B.xor i $ ecbEncrypt cipher iv + let o = B.bxor i (ecbEncrypt cipher iv) `asTypeOf` input in o : doEnc (IV o) is +-- | Nor does this one: @P_i@ is @C_i@ exclusive-ored with @E(C_(i-1))@, and +-- what gets encrypted is again the ciphertext moved along by a block. cfbDecryptGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba -cfbDecryptGeneric cipher ivini input = mconcat $ doDec ivini $ chunk (blockSize cipher) input +cfbDecryptGeneric cipher ivini input = + B.concat $ doDec ivini $ slices (blocksPerCall * bsz) input where + bsz = blockSize cipher + conv x = B.convert x `asTypeOf` input + xorB a b = B.bxor a b `asTypeOf` input doDec _ [] = [] - doDec (IV iv) (i : is) = - let o = B.xor i $ ecbEncrypt cipher iv - in o : doDec (IV i) is + doDec iv (c : cs) = + xorB (conv c) (ecbEncrypt cipher (conv (shiftedBy bsz iv c))) + : doDec (lastBlockOf bsz c) cs +-- | The counters do not depend on the message at all, so a slice of them is +-- built and encrypted in one call. ctrCombineGeneric :: (ByteArray ba, BlockCipher cipher) => cipher -> IV cipher -> ba -> ba -ctrCombineGeneric cipher ivini input = mconcat $ doCnt ivini $ chunk (blockSize cipher) input +ctrCombineGeneric cipher ivini input = + B.concat $ doCnt ivini $ slices (blocksPerCall * bsz) input where + bsz = blockSize cipher + conv x = B.convert x `asTypeOf` input + xorB a b = B.bxor a b `asTypeOf` input doCnt _ [] = [] - doCnt iv@(IV ivd) (i : is) = - let ivEnc = ecbEncrypt cipher ivd - in B.xor i ivEnc : doCnt (ivAdd iv 1) is + doCnt iv (m : ms) = + xorB (conv m) (ecbEncrypt cipher (counters iv n `asTypeOf` input)) + : doCnt (ivAdd iv n) ms + where + n = (S.length m + bsz - 1) `div` bsz +-- | The counters for a slice: the given one, then each next as the one before +-- it plus one. +-- +-- One buffer, filled in place. Asking 'ivAdd' for each of them separately +-- allocated a block per block and walked the whole width of the counter from +-- the original every time, which cost more than the cipher did: counter mode +-- ran at a quarter of what the same cipher managed in ECB, and at an eighth +-- for Blowfish. +counters :: (ByteArray ba, BlockCipher cipher) => IV cipher -> Int -> ba +counters iv n = B.allocAndFreeze (n * bsz) fill + where + bsz = B.length iv + + fill p = do + B.copyByteArrayToPtr iv p + let go k prev + | k >= n = return () + | otherwise = do + let this = prev `plusPtr` bsz + copyBytes this prev bsz + increment this (bsz - 1) + go (k + 1) this + go 1 p + + increment p ofs + | ofs < 0 = return () + | otherwise = do + v <- peek (p `plusPtr` ofs) :: IO Word8 + poke (p `plusPtr` ofs) (v + 1) + if v == 0xff then increment p (ofs - 1) else return () + xtsEncryptGeneric :: (ByteArray ba, BlockCipher128 cipher) => XTS ba cipher xtsEncryptGeneric = xtsGeneric ecbEncrypt @@ -269,13 +353,13 @@ -> ba -> ba xtsGeneric f (cipher, tweakCipher) (IV iv) sPoint input = - mconcat $ doXts iniTweak $ chunk (blockSize cipher) input + B.concat $ doXts iniTweak $ slices (blockSize cipher) input where encTweak = ecbEncrypt tweakCipher iv iniTweak = iterate xtsGFMul encTweak !! fromIntegral sPoint doXts _ [] = [] doXts tweak (i : is) = - let o = B.xor (f cipher $ B.xor i tweak) tweak + let o = B.bxor (f cipher (B.bxor i tweak)) tweak `asTypeOf` input in o : doXts (xtsGFMul tweak) is {- diff --git a/Crypto/Cipher/Types/Utils.hs b/Crypto/Cipher/Types/Utils.hs --- a/Crypto/Cipher/Types/Utils.hs +++ b/Crypto/Cipher/Types/Utils.hs @@ -10,13 +10,19 @@ import Crypto.Internal.ByteArray (ByteArray) import qualified Crypto.Internal.ByteArray as B +import Data.ByteString (ByteString) +import qualified Data.ByteString as S -- | Chunk some input byte array into @sz byte list of byte array. +-- +-- The input is held as a 'ByteString' while it is cut up, because +-- 'Crypto.Internal.ByteArray.splitAt' copies both halves whatever the type +-- underneath: cutting a block off the front that way copies the rest of the +-- message, once per block, and so the message about n/2 times. A ByteString +-- shares instead, and only the blocks themselves are copied out. chunk :: ByteArray b => Int -> b -> [b] -chunk sz bs = split bs +chunk sz bs = map B.convert (split (B.convert bs :: ByteString)) where split b - | B.length b <= sz = [b] - | otherwise = - let (b1, b2) = B.splitAt sz b - in b1 : split b2 + | S.length b <= sz = [b] + | otherwise = let (b1, b2) = S.splitAt sz b in b1 : split b2 diff --git a/Crypto/ConstructHash/MiyaguchiPreneel.hs b/Crypto/ConstructHash/MiyaguchiPreneel.hs --- a/Crypto/ConstructHash/MiyaguchiPreneel.hs +++ b/Crypto/ConstructHash/MiyaguchiPreneel.hs @@ -19,6 +19,7 @@ import Prelude hiding (foldl') import Crypto.Cipher.Types +import Crypto.Cipher.Types.Utils (chunk) import Crypto.Data.Padding (Format (ZERO), pad) import Crypto.Error (throwCryptoError) import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) @@ -43,11 +44,11 @@ MP . foldl' (step $ g) (B.replicate bsz 0) . chunks . pad (ZERO bsz) . B.convert where bsz = blockSize (g B.empty {- dummy to get block size -}) + -- 'chunk' slices rather than splitting the message, which copied whatever + -- was left of it once per block chunks msg | B.null msg = [] - | otherwise = (hd :: Bytes) : chunks tl - where - (hd, tl) = B.splitAt bsz msg + | otherwise = chunk bsz (msg :: Bytes) -- | Compute Miyaguchi-Preneel one way compress using the inferred block cipher. -- Only safe when KEY-SIZE equals to BLOCK-SIZE. @@ -74,4 +75,4 @@ k = g iv bxor :: ByteArray ba => ba -> ba -> ba -bxor = B.xor +bxor = B.bxor diff --git a/Crypto/Data/AFIS.hs b/Crypto/Data/AFIS.hs --- a/Crypto/Data/AFIS.hs +++ b/Crypto/Data/AFIS.hs @@ -15,10 +15,13 @@ -- destroy a key stored on disk. module Crypto.Data.AFIS ( split, + trySplit, merge, + tryMerge, ) where import Control.Monad (foldM, forM_) +import Crypto.Error import Crypto.Hash import Crypto.Internal.Compat import Crypto.Random.Types @@ -49,6 +52,10 @@ -- -- where acc is : -- acc(n+1) = hash (n ++ rand(n)) ^ acc(n) +-- +-- The data has to be at least one byte long and the number of times to diffuse +-- it at least two; anything else raises 'CryptoError_ParameterInvalid', which +-- 'trySplit' reports as 'CryptoFailed' instead. split :: (ByteArray ba, HashAlgorithm hash, DRG rng) => hash @@ -61,10 +68,31 @@ -- ^ original data to diffuse. -> (ba, rng) -- ^ The diffused data -{-# NOINLINE split #-} -split hashAlg rng expandTimes src - | expandTimes <= 1 = error "invalid expandTimes value" - | otherwise = unsafeDoIO $ do +split hashAlg rng expandTimes src = + throwCryptoError (trySplit hashAlg rng expandTimes src) + +-- | Split data to diffused data, reporting parameters the splitter cannot work +-- with rather than raising. +-- +-- See 'split'. +trySplit + :: (ByteArray ba, HashAlgorithm hash, DRG rng) + => hash + -- ^ Hash algorithm to use as diffuser + -> rng + -- ^ Random generator to use + -> Int + -- ^ Number of times to diffuse the data. + -> ba + -- ^ original data to diffuse. + -> CryptoFailable (ba, rng) + -- ^ The diffused data +{-# NOINLINE trySplit #-} +trySplit hashAlg rng expandTimes src + | expandTimes < 2 = CryptoFailed CryptoError_ParameterInvalid + -- an empty secret splits into nothing at all, which merge cannot undo + | blockSize == 0 = CryptoFailed CryptoError_ParameterInvalid + | otherwise = CryptoPassed $ unsafeDoIO $ do (rng', bs) <- B.allocRet diffusedLen runOp return (bs, rng') where @@ -87,6 +115,11 @@ return g' -- | Merge previously diffused data back to the original data. +-- +-- The diffused data has to be a non-empty multiple of the number of times it +-- was diffused, and that number at least two -- the same values 'split' +-- accepts. Anything else raises 'CryptoError_ParameterInvalid', which +-- 'tryMerge' reports as 'CryptoFailed' instead. merge :: (ByteArray ba, HashAlgorithm hash) => hash @@ -97,11 +130,31 @@ -- ^ Diffused data -> ba -- ^ Original data -{-# NOINLINE merge #-} -merge hashAlg expandTimes bs - | r /= 0 = error "diffused data not a multiple of expandTimes" - | originalSize <= 0 = error "diffused data null" - | otherwise = B.allocAndFreeze originalSize $ \dstPtr -> +merge hashAlg expandTimes bs = + throwCryptoError (tryMerge hashAlg expandTimes bs) + +-- | Merge previously diffused data back to the original data, reporting +-- parameters the merger cannot work with rather than raising. +-- +-- See 'merge'. +tryMerge + :: (ByteArray ba, HashAlgorithm hash) + => hash + -- ^ Hash algorithm used as diffuser + -> Int + -- ^ Number of times to un-diffuse the data + -> ba + -- ^ Diffused data + -> CryptoFailable ba + -- ^ Original data +{-# NOINLINE tryMerge #-} +tryMerge hashAlg expandTimes bs + -- guards the quotRem below, which for zero would divide by zero; a count + -- of one would return the diffused data itself as the secret + | expandTimes < 2 = CryptoFailed CryptoError_ParameterInvalid + | r /= 0 = CryptoFailed CryptoError_ParameterInvalid + | originalSize <= 0 = CryptoFailed CryptoError_ParameterInvalid + | otherwise = CryptoPassed $ B.allocAndFreeze originalSize $ \dstPtr -> B.withByteArray bs $ \srcPtr -> do memSet dstPtr 0 originalSize forM_ [0 .. (expandTimes - 2)] $ \i -> do diff --git a/Crypto/Data/Padding.hs b/Crypto/Data/Padding.hs --- a/Crypto/Data/Padding.hs +++ b/Crypto/Data/Padding.hs @@ -22,18 +22,54 @@ PKCS5 | -- | PKCS7 with padding size between 1 and 255 PKCS7 Int - | -- | zero padding with block size + | -- | Zero padding with block size, which must be at least 1. + -- + -- Zero padding does not say how much of it there is, so 'unpad' cannot + -- undo 'pad': see 'unpad'. ZERO Int deriving (Show, Eq) +-- | Is this a block size PKCS7 can describe? +-- +-- The padding octet carries the number of octets added, so it cannot describe +-- a block longer than 255, and a block of zero has nothing to describe. +-- Outside that range the octet would be computed as an 'Int' and then narrowed +-- to a 'Data.Word.Word8', which wraps: 'pad' and 'unpad' would agree on the +-- wrapped value and hand back something other than what was padded. +pkcs7SizeValid :: Int -> Bool +pkcs7SizeValid sz = sz >= 1 && sz <= 255 + +-- | Is this a block size 'ZERO' can use? +-- +-- Nothing is written into the padding, so there is no upper bound to match +-- the one 'PKCS7' has; but a block of zero or fewer octets is not a block, +-- and the length is taken modulo it. +zeroSizeValid :: Int -> Bool +zeroSizeValid sz = sz >= 1 + -- | Apply some pad to a bytearray +-- +-- A 'PKCS7' block size outside 1..255, or a 'ZERO' block size below 1, raises +-- an 'error'; 'unpad' reports the same condition as 'Nothing'. pad :: ByteArray byteArray => Format -> byteArray -> byteArray pad PKCS5 bin = pad (PKCS7 8) bin -pad (PKCS7 sz) bin = bin `B.append` paddingString +pad (PKCS7 sz) bin + | not (pkcs7SizeValid sz) = + error $ + "Crypto.Data.Padding: PKCS7 block size " + ++ show sz + ++ " is not between 1 and 255" + | otherwise = bin `B.append` paddingString where paddingString = B.replicate paddingByte (fromIntegral paddingByte) paddingByte = sz - (B.length bin `mod` sz) -pad (ZERO sz) bin = bin `B.append` paddingString +pad (ZERO sz) bin + | not (zeroSizeValid sz) = + error $ + "Crypto.Data.Padding: ZERO block size " + ++ show sz + ++ " is not at least 1" + | otherwise = bin `B.append` paddingString where paddingString = B.replicate paddingSz 0 paddingSz @@ -44,12 +80,25 @@ len = B.length bin -- | Try to remove some padding from a bytearray. +-- +-- 'PKCS7' padding says how long it is, so this undoes 'pad' exactly. +-- +-- 'ZERO' padding says nothing, and 'pad' adds none at all when the input is +-- already a multiple of the block size, so there is no way to tell padding +-- from data that happens to end in zero octets. This therefore does not undo +-- 'pad': it returns the input unchanged when the last octet is not zero, and +-- 'Nothing' when it is, rather than guess and hand back less than it was +-- given. Zero padding is only usable where the original length is known by +-- other means. unpad :: ByteArray byteArray => Format -> byteArray -> Maybe byteArray unpad PKCS5 bin = unpad (PKCS7 8) bin unpad (PKCS7 sz) bin + | not (pkcs7SizeValid sz) = Nothing | len == 0 = Nothing | (len `mod` sz) /= 0 = Nothing - | paddingSz < 1 || paddingSz > len = Nothing + -- the padded length is a multiple of the block size and the padding is + -- what was added to reach it, so it is never more than one block + | paddingSz < 1 || paddingSz > sz = Nothing | paddingWitness `B.constEq` padding = Just content | otherwise = Nothing where @@ -59,6 +108,7 @@ (content, padding) = B.splitAt (len - paddingSz) bin paddingWitness = B.replicate paddingSz paddingByte :: Bytes unpad (ZERO sz) bin + | not (zeroSizeValid sz) = Nothing | len == 0 = Nothing | (len `mod` sz) /= 0 = Nothing | B.index bin (len - 1) /= 0 = Just bin diff --git a/Crypto/Debug.hs b/Crypto/Debug.hs new file mode 100644 --- /dev/null +++ b/Crypto/Debug.hs @@ -0,0 +1,52 @@ +-- | +-- Module : Crypto.Debug +-- License : BSD-style +-- Maintainer : Kazu Yamamoto +-- Stability : experimental +-- Portability : unknown +-- +-- Printing secret key material, on purpose. +-- +-- The 'Show' instance of a type that holds a secret does not print it. That +-- is deliberate: 'Show' is what @print@, a message built with @error@, an +-- exception and a test framework's failure output all reach for, and a +-- private key reaching a log or a bug report that way is an accident nobody +-- asked for. Those instances render the public part and write @\@ +-- for the rest. +-- +-- This module is how you print one when printing it is what you mean. What +-- 'debugShow' returns is what the derived 'Show' used to return, so for the +-- types that still have a 'Read' instance +-- +-- > read (debugShow k) == k +-- +-- and a call site that was serializing a key through @show@ moves by one +-- word. +-- +-- Needing 'debugShow' in scope is the record of the intent: nothing here is +-- exported anywhere else, so a search for this module finds every place a key +-- can be revealed. Do not leave a call to it where production code runs. +module Crypto.Debug ( + DebugShow (..), + debugShowBytes, +) where + +import Data.Bits (shiftR, (.&.)) +import qualified Data.ByteArray as BA +import Data.Word (Word8) + +-- | Rendering a value with its secret in place. +class DebugShow a where + -- | Render the value, secret included. + debugShow :: a -> String + +-- | Render a secret that is held as bytes, in hexadecimal. The secret keys +-- that keep theirs in a @ScrubbedBytes@ never had a 'Show' that printed it, +-- so unlike the rest of this module what comes back is for reading and not +-- for 'Prelude.read'. +debugShowBytes :: BA.ByteArrayAccess ba => String -> ba -> String +debugShowBytes con b = con ++ (' ' : concatMap hex (BA.unpack b)) + where + hex :: Word8 -> String + hex w = [digit (w `shiftR` 4), digit (w .&. 0x0f)] + digit n = "0123456789abcdef" !! fromIntegral n diff --git a/Crypto/ECC.hs b/Crypto/ECC.hs --- a/Crypto/ECC.hs +++ b/Crypto/ECC.hs @@ -12,6 +12,24 @@ -- Portability : unknown -- -- Elliptic Curve Cryptography +-- +-- == Timing +-- +-- t'Curve_P256R1' reaches a dedicated implementation whose scalar +-- multiplication does not branch on the scalar. t'Curve_P384R1' and +-- t'Curve_P521R1' do not: they are built on "Crypto.ECC.Simple.Prim", whose +-- scalar multiplication is a double-and-add over @Integer@ and is +-- documented there as vulnerable to timing attacks. +-- +-- That matters wherever the scalar is secret, which is both operations that +-- have one: 'ecdh', which multiplies by the private key, and ECDSA signing, +-- which multiplies by the secret nonce. Verification and public-key +-- derivation work on values an attacker already has, so they are unaffected. +-- +-- Note also that @Integer@ arithmetic is variable-time underneath, so no +-- curve built on "Crypto.ECC.Simple.Prim" can be made constant-time without +-- leaving it. Where that matters, use t'Curve_P256R1', t'Curve_X25519', +-- t'Curve_X448' or t'Curve_Edwards25519'. module Crypto.ECC ( Curve_P256R1 (..), Curve_P384R1 (..), @@ -204,8 +222,17 @@ instance EllipticCurveDH Curve_P256R1 where ecdhRaw _ s p = SharedSecret $ P256.pointDh s p - ecdh prx s p = checkNonZeroDH (ecdhRaw prx s p) + -- An all-zero x-coordinate can be valid. Since P-256's group has prime + -- order n, s * P is the identity only when P is the identity or the + -- 256-bit scalar s is zero or n. + ecdh _ s p + | P256.pointIsAtInfinity p + || P256.scalarIsZero s + || P256.scalarCmp s P256.scalarN == EQ = + CryptoFailed CryptoError_ScalarMultiplicationInvalid + | otherwise = CryptoPassed $ SharedSecret $ P256.pointDh s p + instance EllipticCurveBasepointArith Curve_P256R1 where curveOrderBits _ = 256 pointBaseSmul _ = P256.toPoint @@ -215,6 +242,10 @@ scalarAdd _ = P256.scalarAdd scalarMul _ = P256.scalarMul +-- | NIST P-384. +-- +-- Scalar multiplication branches on the scalar; see the note on timing +-- at the head of this module. data Curve_P384R1 = Curve_P384R1 deriving (Show, Data) @@ -251,6 +282,10 @@ scalarAdd _ = ecScalarAdd scalarMul _ = ecScalarMul +-- | NIST P-521. +-- +-- Scalar multiplication branches on the scalar; see the note on timing +-- at the head of this module. data Curve_P521R1 = Curve_P521R1 deriving (Show, Data) diff --git a/Crypto/ECC/Simple/Prim.hs b/Crypto/ECC/Simple/Prim.hs --- a/Crypto/ECC/Simple/Prim.hs +++ b/Crypto/ECC/Simple/Prim.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE BangPatterns #-} {-# LANGUAGE ScopedTypeVariables #-} -- | Elliptic Curve Arithmetic. @@ -15,14 +16,19 @@ pointFromIntegers, isPointAtInfinity, isPointValid, + isPointInSubgroup, ) where import Crypto.ECC.Simple.Types import Crypto.Error +import Crypto.Internal.ECC (CurveField (..), MulResult (..), curveMul) +import Crypto.Number.Basic (numBits) import Crypto.Number.F2m import Crypto.Number.Generate (generateBetween) import Crypto.Number.ModArithmetic import Crypto.Random +import Data.Bits (shiftL, shiftR, testBit, (.&.)) + import Data.Maybe import Data.Proxy @@ -125,41 +131,231 @@ pointBaseMul :: Curve curve => Scalar curve -> Point curve pointBaseMul n = pointMul n (curveEccG $ curveParameters (Proxy :: Proxy curve)) --- | Elliptic curve point multiplication (double and add algorithm). +-- | Elliptic curve point multiplication. -- --- /WARNING:/ Vulnerable to timing attacks. -pointMul :: Curve curve => Scalar curve -> Point curve -> Point curve +-- Over a prime field this goes to C, four bits of scalar at a time, with the +-- multiple to add taken from a table read by touching every entry of it. +-- Over a binary field it also goes to C, as Montgomery's ladder: it carries +-- the x coordinates of two consecutive multiples -- their difference being +-- the point is what lets it carry no more than that -- and spends one +-- addition and one doubling on every bit whichever way the bit goes, with the +-- two exchanged by a mask rather than chosen by a branch. Either way the work +-- follows the width of the curve's order and not the scalar. +-- +-- What falls back on the 'Integer' arithmetic below is a point that is not on +-- the curve, the one point of a binary curve that has no x, and a prime the C +-- will not take. +-- +-- Multiplying the base point of a curve over a prime field -- which is what +-- signing and making a key do, and nothing else does -- goes through a table +-- of its multiples, built when that curve is first asked for one and kept +-- afterwards. The build is a few milliseconds and the table a few hundred +-- kilobytes, and a multiplication that uses it takes about a third of what +-- one without it takes. +-- +-- /WARNING:/ What is left of the 'Integer' arithmetic below -- a point off +-- the curve, the one point of a binary curve with no x, a prime or a +-- polynomial the C will not take -- has uniform operation counts at best, and +-- uniform operation counts are not constant time: those operations cost what +-- the values they are given cost. See the note in +-- "Crypto.ECC". +pointMul + :: forall curve. Curve curve => Scalar curve -> Point curve -> Point curve pointMul _ PointO = PointO pointMul (Scalar n) p | n == 0 = PointO - | n == 1 = p - | odd n = pointAdd p (pointMul (Scalar (n - 1)) p) - | otherwise = pointMul (Scalar (n `div` 2)) (pointDouble p) + | n < 0 = pointNegate (pointMul (Scalar (negate n) :: Scalar curve) p) + | otherwise = + case curveType (Proxy :: Proxy curve) of + CurvePrime (CurvePrimeParam pr) -> primeMul pr + CurveBinary (CurveBinaryParam fx) -> binaryMul fx + where + cc = curveParameters (Proxy :: Proxy curve) + a = curveEccA cc + -- Count to the width of the order, which is public, so a scalar in range + -- -- which is every secret one -- takes the same number of steps whatever + -- it is. A scalar may still be given out of range, and then the count has + -- to follow it or the high bits would be dropped. + bits = max (integerBits n) (integerBits (curveEccN cc)) --- | Elliptic curve double-scalar multiplication (uses Shamir's trick). + -- The C answers for a point on the curve; anything else keeps the + -- answers it has always had from the code below. + primeMul pr = case p of + Point px py + | isPointValid (Proxy :: Proxy curve) px py -> + answer slow $ + curveMul + (Prime pr a (curveEccB cc)) + (curveEccN cc) + n + px + py + (p == curveEccG cc) + _ -> slow + where + slow = jacobianMul pr a bits n p + + -- The ladder answers for a point on the curve that has an x; the one + -- point with no x, and anything off the curve, keep what they had. + binaryMul fx = case p of + Point px py + | isPointValid (Proxy :: Proxy curve) px py -> + answer (affineMul n p) $ + curveMul (Binary fx (curveEccB cc)) (curveEccN cc) n px py False + _ -> affineMul n p + + -- what the C could not take goes back to the code that was here before + answer fallback r = case r of + MulPoint x y -> Point x y + MulInfinity -> PointO + MulUnsupported -> fallback + + affineMul k q + | k == 0 = PointO + | k == 1 = q + | odd k = pointAdd q (affineMul (k - 1) q) + | otherwise = affineMul (k `div` 2) (pointDouble q) + +-- | Number of bits needed to write n, for n > 0. +integerBits :: Integer -> Int +integerBits = go 0 + where + go acc 0 = acc + go acc k = go (acc + 1) (k `div` 2) + +-- | A point in Jacobian coordinates: @(X, Y, Z)@ stands for the affine +-- @(X\/Z^2, Y\/Z^3)@, and @JPointO@ for the point at infinity. Only ever +-- used inside this module, since t'Point' is what the curve exposes. +data JPoint = JPointO | JPoint !Integer !Integer !Integer + +-- | The prime, the width to fold at, and what to fold back in. A @c@ of zero +-- says to divide instead, either because the prime has no such shape or +-- because it is too small for folding to pay: @c@ has to be under half the +-- width, or folding would not shrink the number, and below 256 bits the +-- handful of 'Integer' operations folding takes costs more than the division +-- it saves -- measured on P-192, where folding is 14% slower. -- +-- Most curve primes are @2^k - c@ with @c@ far smaller than the prime, and +-- then reducing is a shift, a multiplication by @c@ and an addition, where +-- dividing a number twice the width costs about four times as much. +data Field = Field !Integer !Int !Integer + +mkField :: Integer -> Field +mkField p + | p > 0 && c > 0 && 2 * numBits c <= k && k >= 256 = Field p k c + | otherwise = Field p 0 0 + where + k = numBits p + c = (1 `shiftL` k) - p + +fieldPrime :: Field -> Integer +fieldPrime (Field p _ _) = p + +fieldReduce :: Field -> Integer -> Integer +fieldReduce (Field p k c) x + | c == 0 || x < 0 = x `mod` p + | otherwise = trim (fold x) + where + mask = (1 `shiftL` k) - 1 + fold v + | v > mask = fold ((v `shiftR` k) * c + (v .&. mask)) + | otherwise = v + trim v + | v >= p = trim (v - p) + | otherwise = v +{-# INLINE fieldReduce #-} + +jacobianMul + :: Integer -> Integer -> Int -> Integer -> Point curve -> Point curve +jacobianMul _ _ _ _ PointO = PointO +jacobianMul pr a bits n (Point px py) = fromJacobian f (go (bits - 1) JPointO) + where + f = mkField pr + + -- The bangs are what make the addition happen at every bit. Without + -- them the one that is not taken stays a thunk and is never worked out, + -- so the multiplication costs a step for every bit that is set rather + -- than for every bit there is, and a single measurement tells an attacker + -- how many bits of the scalar are set. + go i acc + | i < 0 = acc + | otherwise = + let !d = jDouble f a acc + !s = jAddAffine f a d px py + in go (i - 1) (if testBit n i then s else d) + +jDouble :: Field -> Integer -> JPoint -> JPoint +jDouble _ _ JPointO = JPointO +jDouble f a (JPoint x y z) + | y == 0 = JPointO + | otherwise = JPoint x3 y3 z3 + where + red = fieldReduce f + yy = red (y * y) + delta = red (4 * x * yy) + zz = red (z * z) + m = red (3 * x * x + a * zz * zz) + x3 = red (m * m - 2 * delta) + y3 = red (m * (delta - x3) - 8 * yy * yy) + z3 = red (2 * y * z) + +-- | Add a point whose z is one, which is what a scalar multiplication always +-- adds: u1 is x1, s1 is y1, and z3 is one multiplication rather than two. +jAddAffine :: Field -> Integer -> JPoint -> Integer -> Integer -> JPoint +jAddAffine _ _ JPointO x2 y2 = JPoint x2 y2 1 +jAddAffine f a p@(JPoint x1 y1 z1) x2 y2 + | h /= 0 = JPoint x3 y3 z3 + | r /= 0 = JPointO + | otherwise = jDouble f a p + where + red = fieldReduce f + z1s = red (z1 * z1) + u2 = red (x2 * z1s) + s2 = red (y2 * z1s * z1) + h = red (u2 - x1) + r = red (s2 - y1) + h2 = red (h * h) + h3 = red (h2 * h) + x3 = red (r * r - h3 - 2 * x1 * h2) + y3 = red (r * (x1 * h2 - x3) - y1 * h3) + z3 = red (h * z1) + +fromJacobian :: Field -> JPoint -> Point curve +fromJacobian _ JPointO = PointO +fromJacobian f (JPoint x y z) = + case inverse z (fieldPrime f) of + Nothing -> PointO + Just zi -> + let red = fieldReduce f + zi2 = red (zi * zi) + in Point (red (x * zi2)) (red (y * zi2 * zi)) + +-- | Elliptic curve double-scalar multiplication. +-- -- > pointAddTwoMuls n1 p1 n2 p2 == pointAdd (pointMul n1 p1) -- > (pointMul n2 p2) -- +-- which is how it is done: the two multiplications separately, and then one +-- addition. +-- +-- This used to be Shamir's trick, one pass over the bits of both scalars at +-- once, which shares the doublings between them and is the right thing to do +-- when the two multiplications would cost the same. They no longer do. +-- 'pointMul' goes to C, and over a prime field it multiplies the base point +-- through a table of its multiples, which is a third of the price of an +-- ordinary multiplication -- and the base point is one of the two here, +-- since ECDSA verification is what asks for this. Sharing the doublings +-- with a pass in "Integer" arithmetic gives that up and more: on P-384 it +-- costs twice what two multiplications in C cost, and on the curves over a +-- binary field, whose addition needs an inversion where C has a ladder that +-- needs none, it costs two hundred times as much. +-- -- /WARNING:/ Vulnerable to timing attacks. pointAddTwoMuls - :: Curve curve + :: forall curve + . Curve curve => Scalar curve -> Point curve -> Scalar curve -> Point curve -> Point curve -pointAddTwoMuls _ PointO _ PointO = PointO -pointAddTwoMuls _ PointO n2 p2 = pointMul n2 p2 -pointAddTwoMuls n1 p1 _ PointO = pointMul n1 p1 -pointAddTwoMuls (Scalar n1) p1 (Scalar n2) p2 = go (n1, n2) - where - p0 = pointAdd p1 p2 - - go (0, 0) = PointO - go (k1, k2) = - let q = pointDouble $ go (k1 `div` 2, k2 `div` 2) - in case (odd k1, odd k2) of - (True, True) -> pointAdd p0 q - (True, False) -> pointAdd p1 q - (False, True) -> pointAdd p2 q - (False, False) -> q +pointAddTwoMuls n1 p1 n2 p2 = pointAdd (pointMul n1 p1) (pointMul n2 p2) -- | Check if a point is the point at infinity. isPointAtInfinity :: Point curve -> Bool @@ -173,9 +369,13 @@ pointFromIntegers :: forall curve. Curve curve => (Integer, Integer) -> CryptoFailable (Point curve) pointFromIntegers (x, y) - | isPointValid (Proxy :: Proxy curve) x y = CryptoPassed $ Point x y - | otherwise = - CryptoFailed $ CryptoError_PointCoordinatesInvalid + | not (isPointValid (Proxy :: Proxy curve) x y) = + CryptoFailed CryptoError_PointCoordinatesInvalid + | not (isPointInSubgroup (Proxy :: Proxy curve) p) = + CryptoFailed CryptoError_PointSubgroupInvalid + | otherwise = CryptoPassed p + where + p = Point x y -- | check if a point is on specific curve -- @@ -206,6 +406,24 @@ ] where ty = curveType proxy + cc = curveParameters proxy + +-- | Check that a point is in the subgroup the base point generates, which is +-- the further check 'isPointValid' does not make. A point that is on the +-- curve but outside that subgroup answers a multiplication modulo an order +-- smaller than the group's, so the multiplier -- a private number, where the +-- point came from a peer -- is revealed modulo that small order. +-- +-- Where the cofactor is 1 the subgroup is the whole curve group and the +-- answer is 'True' for any point on the curve, at no cost. Otherwise the +-- point is multiplied by the group order and the answer is whether that +-- reaches the point at infinity, which costs one scalar multiplication. +isPointInSubgroup + :: forall proxy curve. Curve curve => proxy curve -> Point curve -> Bool +isPointInSubgroup proxy p + | curveEccH cc == 1 = True + | otherwise = pointMul (Scalar (curveEccN cc) :: Scalar curve) p == PointO + where cc = curveParameters proxy -- | div and mod diff --git a/Crypto/ECC/Simple/Types.hs b/Crypto/ECC/Simple/Types.hs --- a/Crypto/ECC/Simple/Types.hs +++ b/Crypto/ECC/Simple/Types.hs @@ -153,6 +153,19 @@ data SEC_t571k1 = SEC_t571k1 deriving (Show, Read, Eq) data SEC_t571r1 = SEC_t571r1 deriving (Show, Read, Eq) +{-# DEPRECATED + SEC_t113r1, SEC_t113r2, SEC_t131r1, SEC_t131r2, SEC_t163k1, SEC_t163r1, + SEC_t163r2, SEC_t193r1, SEC_t193r2, SEC_t233k1, SEC_t233r1, SEC_t239k1, + SEC_t283k1, SEC_t283r1, SEC_t409k1, SEC_t409r1, SEC_t571k1, SEC_t571r1 + [ "This curve is over a binary field, and those are obsolete." + , "They are also the curves whose cofactor is not 1, so a point from" + , "a peer needs the subgroup check that costs a further scalar" + , "multiplication; pyca/cryptography deprecated them for removal in" + , "the release that fixed CVE-2026-26007. This one will go in a" + , "later major version of crypton. Prefer a prime curve, or X25519." + ] + #-} + -- | Define names for known recommended curves. instance Curve SEC_p112r1 where curveType _ = typeSEC_p112r1 diff --git a/Crypto/Error/Types.hs b/Crypto/Error/Types.hs --- a/Crypto/Error/Types.hs +++ b/Crypto/Error/Types.hs @@ -50,6 +50,13 @@ CryptoError_SaltTooSmall | CryptoError_OutputLengthTooSmall | CryptoError_OutputLengthTooBig + | -- | A parameter is outside the range the algorithm accepts. Appended to + -- keep the 'Enum' values of the constructors above unchanged. + CryptoError_ParameterInvalid + | -- | A point satisfies the curve equation but lies outside the subgroup + -- the base point generates, so multiplying it would answer modulo a + -- small order. Appended for the same reason as the constructor above. + CryptoError_PointSubgroupInvalid deriving (Show, Eq, Enum, Data) instance E.Exception CryptoError diff --git a/Crypto/Hash/SHAKE.hs b/Crypto/Hash/SHAKE.hs --- a/Crypto/Hash/SHAKE.hs +++ b/Crypto/Hash/SHAKE.hs @@ -44,9 +44,9 @@ -- | SHAKE128 (128 bits) extendable output function. Supports an arbitrary -- digest size, to be specified as a type parameter of kind 'Nat'. -- --- Note: outputs from @'SHAKE128' n@ and @'SHAKE128' m@ for the same input are +-- Note: outputs from @t'SHAKE128' n@ and @t'SHAKE128' m@ for the same input are -- correlated (one being a prefix of the other). Results are unrelated to --- 'SHAKE256' results. +-- t'SHAKE256' results. data SHAKE128 (bitlen :: Nat) = SHAKE128 deriving (Show, Data) @@ -68,9 +68,9 @@ -- | SHAKE256 (256 bits) extendable output function. Supports an arbitrary -- digest size, to be specified as a type parameter of kind 'Nat'. -- --- Note: outputs from @'SHAKE256' n@ and @'SHAKE256' m@ for the same input are +-- Note: outputs from @t'SHAKE256' n@ and @t'SHAKE256' m@ for the same input are -- correlated (one being a prefix of the other). Results are unrelated to --- 'SHAKE128' results. +-- t'SHAKE128' results. data SHAKE256 (bitlen :: Nat) = SHAKE256 deriving (Show, Data) diff --git a/Crypto/Internal/ByteArray.hs b/Crypto/Internal/ByteArray.hs --- a/Crypto/Internal/ByteArray.hs +++ b/Crypto/Internal/ByteArray.hs @@ -16,6 +16,7 @@ constAllZero, allocAndFreezePrimIO, allocAndFreezePrim, + bxor, ) where import Data.ByteArray @@ -24,7 +25,7 @@ import Data.Bits ((.|.)) import qualified Data.Primitive.ByteArray as Prim -import Data.Word (Word8) +import Data.Word (Word32, Word8) import Foreign.Ptr (Ptr, castPtr) import Foreign.Storable (peekByteOff) @@ -54,3 +55,22 @@ e <- peekByteOff p i loop p (i + 1) (acc .|. e) len = Data.ByteArray.length b + +-- | @a@ exclusive-ored with @b@, as long as the shorter of the two. +-- +-- 'Data.ByteArray.xor' does this a byte at a time through an IO applicative, +-- which allocates about fifty bytes of heap for every byte it produces. That +-- is more than a block cipher costs: it was four fifths of the time counter +-- mode spent on anything but AES, whose modes are in C and do not come this +-- way. +bxor :: (ByteArrayAccess a, ByteArrayAccess b, ByteArray c) => a -> b -> c +bxor a b = unsafeDoIO $ + alloc n $ \pd -> + withByteArray a $ \pa -> + withByteArray b $ \pb -> + c_memxor pd pa pb (fromIntegral n) + where + n = min (Data.ByteArray.length a) (Data.ByteArray.length b) + +foreign import ccall unsafe "crypton_memxor.h crypton_memxor" + c_memxor :: Ptr Word8 -> Ptr Word8 -> Ptr Word8 -> Word32 -> IO () diff --git a/Crypto/Internal/ECC.hs b/Crypto/Internal/ECC.hs new file mode 100644 --- /dev/null +++ b/Crypto/Internal/ECC.hs @@ -0,0 +1,524 @@ +{-# LANGUAGE BangPatterns #-} + +-- | +-- Module : Crypto.Internal.ECC +-- License : BSD-style +-- Maintainer : Kazu Yamamoto +-- Stability : experimental +-- Portability : Good +-- +-- The C scalar multiplication for curves over a prime field, which both of +-- the elliptic curve APIs reach for. +module Crypto.Internal.ECC ( + MulResult (..), + CurveField (..), + curveMul, + primeCurveMul, + primeCurveTableMul, + baseTable, + binaryCurveMul, + binaryCurveC, +) where + +import Crypto.Internal.Compat (unsafeDoIO) +import Crypto.Number.Basic (numBits, numBytes) +import Crypto.Number.F2m (addF2m, divF2m, mulF2m, squareF2m) +import qualified Crypto.Number.Serialize.Internal as Internal +import Crypto.PubKey.ECC.Types ( + Curve (..), + CurveCommon (..), + CurveName, + CurvePrime (..), + Point (..), + getCurveByName, + ) +import Data.Bits (testBit) +import Data.Word (Word32, Word8) +import Foreign.C.Types (CInt (..)) +import Foreign.ForeignPtr (ForeignPtr, mallocForeignPtrBytes, withForeignPtr) +import Foreign.Marshal.Alloc (allocaBytes) +import Foreign.Ptr (Ptr, plusPtr) + +-- | What the C made of it. +data MulResult + = -- | the point it arrived at + MulPoint !Integer !Integer + | -- | the point at infinity, which has no coordinates + MulInfinity + | -- | not something the C works with, so the caller has to + MulUnsupported + deriving (Show, Eq) + +-- | Multiply a point by a scalar on the curve @y^2 = x^3 + a*x + b@ over the +-- field of @p@, which has to be an odd prime. The point has to be on the +-- curve and not the point at infinity, and its coordinates, @a@ and @b@ have +-- to be under @p@; the caller has all of that to hand and the C does not +-- check it. +-- +-- The scalar is walked four bits at a time over the whole of the width asked +-- for, so its value is hidden but that width is not. Ask for the width of +-- the curve's order, which is public, and every scalar in range costs the +-- same. +primeCurveMul + :: Integer + -- ^ p + -> Integer + -- ^ a + -> Integer + -- ^ b + -> Int + -- ^ how many bytes of scalar to walk + -> Integer + -- ^ the scalar + -> Integer + -- ^ the point's x + -> Integer + -- ^ the point's y + -> MulResult +primeCurveMul p a b klen k px py + | p <= 0 || even p || klen <= 0 || k < 0 = MulUnsupported + | otherwise = unsafeDoIO $ + allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of + (outx : outy : cx : cy : ca : cb : cp : ck : _) -> do + _ <- Internal.i2ospOf px cx plen + _ <- Internal.i2ospOf py cy plen + _ <- Internal.i2ospOf a ca plen + _ <- Internal.i2ospOf b cb plen + _ <- Internal.i2ospOf p cp plen + _ <- Internal.i2ospOf k ck klen + r <- + c_ecc_mul + outx + outy + cx + cy + ck + (fromIntegral klen) + ca + cb + cp + (fromIntegral plen) + -- the scalar is the caller's secret, and this is the last place + -- it is written out in the clear + Internal.i2ospOf 0 ck klen >> return () + case r of + 0 -> do + !x <- Internal.os2ip outx plen + !y <- Internal.os2ip outy plen + return (MulPoint x y) + 1 -> return MulInfinity + _ -> return MulUnsupported + _ -> return MulUnsupported -- there are eight, but say so anyway + where + !plen = numBytes p + -- What the buffer holds, in this order: the two coordinates out, the two + -- in, a, b, the prime, and the scalar. The room to take and where each + -- one starts both come from here, so they cannot drift apart. + -- + -- They did once, and nothing caught it: the memory is a pinned array on + -- the GHC heap, so writing past it is invisible to valgrind, which sees + -- one large allocation, and to the sanity checks of the debug RTS, which + -- found nothing when the mistake was put back to try them. One runner + -- out of eighteen died of it and the rest went green. The way to be + -- right about this is not to have two numbers to keep the same. + widths = [plen, plen, plen, plen, plen, plen, plen, klen] + +foreign import ccall unsafe "crypton_ecc_table_size" + c_ecc_table_size :: Word32 -> Word32 -> Word32 + +foreign import ccall safe "crypton_ecc_table_build" + c_ecc_table_build + :: Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> IO CInt + +foreign import ccall safe "crypton_ecc_table_mul" + c_ecc_table_mul + :: Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> IO CInt + +foreign import ccall safe "crypton_ecc_mul" + c_ecc_mul + :: Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> IO CInt + +-- | What a curve is made of, as much of it as a multiplication needs. +data CurveField + = -- | over a prime field: the prime, a and b + Prime !Integer !Integer !Integer + | -- | over a binary field: the polynomial and b + Binary !Integer !Integer + deriving (Show, Eq) + +-- | Multiply a point by a scalar, through the C wherever the C takes it. +-- +-- Both elliptic curve APIs come here, so that the decision -- the table for a +-- base point, the C for anything else, what is left over -- is made once and +-- in one place. One of those APIs cannot be reached from outside the library +-- on a curve over a binary field, and this is how that copy stays the same +-- code as the copy everybody runs. +-- +-- The caller has seen to it that the point is on the curve, which is what the +-- C takes for granted, and deals with 'MulUnsupported' in whatever way it +-- has. +curveMul + :: CurveField + -> Integer + -- ^ the order of the curve + -> Integer + -- ^ the scalar + -> Integer + -- ^ the point's x + -> Integer + -- ^ the point's y + -> Bool + -- ^ whether that point is the curve's base point + -> MulResult +curveMul field order k px py isBase = case field of + Prime p a b + | isBase + , klen == numBytes order + , Just table <- baseTable p a b klen px py -> + primeCurveTableMul table p a b klen k + | otherwise -> primeCurveMul p a b klen k px py + Binary fx b + | px == 0 -> MulUnsupported -- its own negation, and easier the long way + | otherwise -> case binaryCurveC fx b klen k px py of + -- the ladder in Haskell, for a field the C will not take + MulUnsupported -> binaryCurveMul fx b (klen * 8) k px py + r -> r + where + -- Walk the width of the order, which is public, so a scalar in range -- + -- which is every secret one -- costs the same whatever it is. A scalar + -- may still be given out of range, and then the width has to follow it or + -- the high bits would be dropped. + !klen = max (numBytes k) (numBytes order) + +-- | The table for the base point of a curve the library knows, which is the +-- point signing and making a key multiply and the only point worth keeping a +-- table for. The curves are told apart by their numbers, which are public, +-- so both of the elliptic curve APIs find the same table. +-- +-- Each is built when it is first wanted and kept for as long as the program +-- runs, and a curve nobody multiplies the base point of never has one built. +-- Building costs 2.8 ms for secp256k1, 5.5 for secp384r1 and 10.6 for +-- secp521r1, and the last two take 221 KB and 456 KB. A multiplication with +-- the table takes about a third of what one without it takes, so the build +-- pays for itself after about fifteen of them: a program that signs many +-- times wins, and one that signs once and exits does not. +baseTable + :: Integer + -- ^ p + -> Integer + -- ^ a + -> Integer + -- ^ b + -> Int + -- ^ how many bytes of scalar are wanted + -> Integer + -- ^ the base point's x + -> Integer + -- ^ the base point's y + -> Maybe (ForeignPtr Word8) +baseTable p a b klen gx gy = + case lookup (p, a, b, klen, gx, gy) baseTables of + Just table -> table + Nothing -> Nothing + +type TableKey = (Integer, Integer, Integer, Int, Integer, Integer) + +baseTables :: [(TableKey, Maybe (ForeignPtr Word8))] +baseTables = + [ ((p, a, b, klen, gx, gy), primeCurveTable p a b klen gx gy) + | name <- [minBound .. maxBound] :: [CurveName] + , CurveFP (CurvePrime p cc) <- [getCurveByName name] + , Point gx gy <- [ecc_g cc] + , let a = ecc_a cc + , let b = ecc_b cc + , let klen = numBytes (ecc_n cc) + ] +{-# NOINLINE baseTables #-} + +-- | The multiples of a point that 'primeCurveTableMul' wants: for every four +-- bits of a scalar, the sixteen points those bits can call for. Building it +-- costs a few thousand point operations, and what it saves is all the +-- doublings of every multiplication that uses it, so it is worth keeping for +-- as long as the point is -- which for a curve's base point is forever. +-- +-- The arguments are as for 'primeCurveMul'. 'Nothing' means the C would not +-- take them. +primeCurveTable + :: Integer + -- ^ p + -> Integer + -- ^ a + -> Integer + -- ^ b + -> Int + -- ^ how many bytes of scalar the table is to cover + -> Integer + -- ^ the point's x + -> Integer + -- ^ the point's y + -> Maybe (ForeignPtr Word8) +primeCurveTable p a b klen px py + | p <= 0 || even p || klen <= 0 || size == 0 = Nothing + | otherwise = unsafeDoIO $ do + table <- mallocForeignPtrBytes (fromIntegral size) + allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of + (cx : cy : ca : cb : cp : _) -> do + _ <- Internal.i2ospOf px cx plen + _ <- Internal.i2ospOf py cy plen + _ <- Internal.i2ospOf a ca plen + _ <- Internal.i2ospOf b cb plen + _ <- Internal.i2ospOf p cp plen + r <- withForeignPtr table $ \t -> + c_ecc_table_build + t + cx + cy + (fromIntegral klen) + ca + cb + cp + (fromIntegral plen) + return $ if r == 0 then Just table else Nothing + _ -> return Nothing -- there are five, but say so anyway + where + !plen = numBytes p + !size = c_ecc_table_size (fromIntegral plen) (fromIntegral klen) + -- the point, a, b and the prime, all of the prime's width. The room to + -- take and where each one starts both come from here, so they cannot + -- drift apart: they did once, and nothing caught it -- see the note on + -- primeCurveMul. + widths = [plen, plen, plen, plen, plen] + +-- | Multiply the point a table was built for by a scalar of the width the +-- table was built for. One addition for every four bits and no doublings. +primeCurveTableMul + :: ForeignPtr Word8 + -- ^ the table + -> Integer + -- ^ p + -> Integer + -- ^ a + -> Integer + -- ^ b + -> Int + -- ^ the width the table was built for + -> Integer + -- ^ the scalar + -> MulResult +primeCurveTableMul table p a b klen k + | p <= 0 || even p || klen <= 0 || k < 0 = MulUnsupported + | otherwise = unsafeDoIO $ + allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of + (outx : outy : ca : cb : cp : ck : _) -> do + _ <- Internal.i2ospOf a ca plen + _ <- Internal.i2ospOf b cb plen + _ <- Internal.i2ospOf p cp plen + _ <- Internal.i2ospOf k ck klen + r <- withForeignPtr table $ \t -> + c_ecc_table_mul + outx + outy + t + ck + (fromIntegral klen) + ca + cb + cp + (fromIntegral plen) + Internal.i2ospOf 0 ck klen >> return () + case r of + 0 -> do + !x <- Internal.os2ip outx plen + !y <- Internal.os2ip outy plen + return (MulPoint x y) + 1 -> return MulInfinity + _ -> return MulUnsupported + _ -> return MulUnsupported -- there are six, but say so anyway + where + !plen = numBytes p + widths = [plen, plen, plen, plen, plen, klen] + +-- | Multiply a point by a scalar on the curve @y^2 + x*y = x^3 + a*x^2 + b@ +-- over the binary field of @fx@, by Montgomery's ladder. +-- +-- The ladder carries the multiples of two consecutive numbers, whose +-- difference is therefore the point itself, and every bit of the scalar costs +-- one addition and one doubling of them whichever way it goes. Only the x +-- coordinates are carried -- the difference being known is what lets them be +-- -- and the y is worked out at the end from the two of them, which is what +-- makes the coordinates projective: one division for the whole +-- multiplication rather than one for every step. +-- +-- The point has to be on the curve and to have an x, which is what the +-- caller has to hand: the one point with no x is its own negation and is +-- easier multiplied the long way. The scalar is walked over the whole of +-- the width asked for, so its value is hidden but that width is not. +binaryCurveMul + :: Integer + -- ^ the polynomial the field is over + -> Integer + -- ^ b + -> Int + -- ^ how many bits of scalar to walk + -> Integer + -- ^ the scalar + -> Integer + -- ^ the point's x + -> Integer + -- ^ the point's y + -> MulResult +binaryCurveMul fx b bits k x y + | bits <= 0 || k < 0 || x == 0 = MulUnsupported + | otherwise = recover (go (bits - 1) (1, 0) (x, 1)) + where + infixl 6 .+. + (.+.) = addF2m + sqr = squareF2m fx + mul = mulF2m fx + + -- The two of them added, which the difference between them being the + -- point makes possible from their x coordinates alone. It does not + -- matter which way round they come. + madd (xa, za) (xb, zb) = + let t1 = mul xa zb + t2 = mul xb za + z = sqr (t1 .+. t2) + in (mul x z .+. mul t1 t2, z) + + -- One of them doubled. + mdouble (xa, za) = + let xa2 = sqr xa + za2 = sqr za + in (sqr xa2 .+. mul b (sqr za2), mul xa2 za2) + + -- Nothing is at infinity to begin with and the point is next to it, and + -- from there each bit takes the pair to twice where it was. The bangs + -- are what make both halves happen: without them the one the bit does not + -- call for would stay a thunk, and the work would follow the scalar. + go i p1 p2 + | i < 0 = (p1, p2) + | testBit k i = + let !s = madd p1 p2 + !d = mdouble p2 + in go (i - 1) s d + | otherwise = + let !s = madd p1 p2 + !d = mdouble p1 + in go (i - 1) d s + + -- x1 is the answer and x2 is one point further on; together with the + -- point they give the y that the ladder does not carry. + recover ((x1, z1), (x2, z2)) + | z1 == 0 = MulInfinity -- the multiple is at infinity + | z2 == 0 = MulPoint x (x .+. y) -- the one after it is, so this is -P + | otherwise = case (divF2m fx x1 z1, divF2m fx x2 z2) of + (Just xa, Just xb) -> + let u = xa .+. x + v = xb .+. x + inner = mul u v .+. sqr x .+. y + in case divF2m fx (mul u inner) x of + Just w -> MulPoint xa (w .+. y) + Nothing -> MulUnsupported + _ -> MulUnsupported + +-- | Multiply a point by a scalar on a curve over a binary field, in C. +-- +-- The ladder is the same one 'binaryCurveMul' walks, but the field arithmetic +-- is carry-less multiplication -- the processor's where it has it, and four +-- interleaved groups of bits where it does not -- rather than 'Integer' +-- shifts and exclusive ors, and nothing in it branches on the scalar or +-- indexes memory with it. +-- +-- The point has to be on the curve and to have an x, and the scalar is walked +-- over the whole of the width asked for, as for 'primeCurveMul'. +binaryCurveC + :: Integer + -- ^ the polynomial the field is over + -> Integer + -- ^ b + -> Int + -- ^ how many bytes of scalar to walk + -> Integer + -- ^ the scalar + -> Integer + -- ^ the point's x + -> Integer + -- ^ the point's y + -> MulResult +binaryCurveC fx b klen k px py + | fx <= 1 || klen <= 0 || k < 0 || px <= 0 || flen <= 0 = MulUnsupported + | otherwise = unsafeDoIO $ + allocaBytes (sum widths) $ \base -> case scanl plusPtr base widths of + (outx : outy : cx : cy : cb : cf : ck : _) -> do + _ <- Internal.i2ospOf px cx flen + _ <- Internal.i2ospOf py cy flen + _ <- Internal.i2ospOf b cb flen + _ <- Internal.i2ospOf fx cf fxlen + _ <- Internal.i2ospOf k ck klen + r <- + c_f2m_mul + outx + outy + cx + cy + ck + (fromIntegral klen) + cb + (fromIntegral flen) + cf + (fromIntegral fxlen) + Internal.i2ospOf 0 ck klen >> return () + case r of + 0 -> do + !x <- Internal.os2ip outx flen + !y <- Internal.os2ip outy flen + return (MulPoint x y) + 1 -> return MulInfinity + _ -> return MulUnsupported + _ -> return MulUnsupported -- there are seven, but say so anyway + where + -- the field is the degree of the polynomial, which is one under its width + !flen = (numBits fx - 1 + 7) `div` 8 + !fxlen = numBytes fx + widths = [flen, flen, flen, flen, flen, fxlen, klen] + +foreign import ccall safe "crypton_f2m_mul" + c_f2m_mul + :: Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Word32 + -> IO CInt diff --git a/Crypto/Internal/Nat.hs b/Crypto/Internal/Nat.hs --- a/Crypto/Internal/Nat.hs +++ b/Crypto/Internal/Nat.hs @@ -33,7 +33,7 @@ IsLE bitlen n 'False = 'False #endif --- | ensure the given `bitlen` is lesser or equal to `n` +-- | ensure the given @bitlen@ is lesser or equal to @n@ -- type IsAtMost (bitlen :: Nat) (n :: Nat) = IsLE bitlen n (bitlen <=? n) ~ 'True @@ -48,7 +48,7 @@ IsGE bitlen n 'False = 'False #endif --- | ensure the given `bitlen` is greater or equal to `n` +-- | ensure the given @bitlen@ is greater or equal to @n@ -- type IsAtLeast (bitlen :: Nat) (n :: Nat) = IsGE bitlen n (n <=? bitlen) ~ 'True @@ -208,6 +208,6 @@ Mod8 63 = 7 Mod8 n = Mod8 (n - 64) --- | ensure the given `bitlen` is divisible by 8 +-- | ensure the given @bitlen@ is divisible by 8 -- type IsDivisibleBy8 bitLen = IsDiv8 bitLen bitLen ~ 'True diff --git a/Crypto/KDF/Argon2.hs b/Crypto/KDF/Argon2.hs --- a/Crypto/KDF/Argon2.hs +++ b/Crypto/KDF/Argon2.hs @@ -24,10 +24,10 @@ hash, ) where -import Control.Monad (when) import Crypto.Error import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B +import Crypto.Internal.Compat (unsafeDoIO) import Data.Word import Foreign.C import Foreign.Ptr @@ -57,17 +57,17 @@ -- | The time cost, which defines the amount of computation realized and therefore the execution time, given in number of iterations. -- --- 'FFI.ARGON2_MIN_TIME' <= 'hashIterations' <= 'FFI.ARGON2_MAX_TIME' +-- 'FFI.ARGON2_MIN_TIME' <= 'iterations' <= 'FFI.ARGON2_MAX_TIME' type TimeCost = Word32 -- | The memory cost, which defines the memory usage, given in kibibytes. -- --- max 'FFI.ARGON2_MIN_MEMORY' (8 * 'hashParallelism') <= 'hashMemory' <= 'FFI.ARGON2_MAX_MEMORY' +-- max 'FFI.ARGON2_MIN_MEMORY' (8 * 'parallelism') <= 'memory' <= 'FFI.ARGON2_MAX_MEMORY' type MemoryCost = Word32 -- | A parallelism degree, which defines the number of parallel threads. -- --- 'FFI.ARGON2_MIN_LANES' <= 'hashParallelism' <= 'FFI.ARGON2_MAX_LANES' && 'FFI.ARGON_MIN_THREADS' <= 'hashParallelism' <= 'FFI.ARGON2_MAX_THREADS' +-- 'FFI.ARGON2_MIN_LANES' <= 'parallelism' <= 'FFI.ARGON2_MAX_LANES' && 'FFI.ARGON_MIN_THREADS' <= 'parallelism' <= 'FFI.ARGON2_MAX_THREADS' type Parallelism = Word32 -- | Parameters that can be adjusted to change the runtime performance of the @@ -104,6 +104,11 @@ , version = Version13 } +-- | Hash a password with Argon2. +-- +-- Options the underlying implementation refuses -- iterations, memory or +-- parallelism outside the range it accepts -- are reported as +-- 'CryptoError_ParameterInvalid'. hash :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) => Options @@ -115,22 +120,28 @@ | saltLen < saltMinLength = CryptoFailed CryptoError_SaltTooSmall | outLen < outputMinLength = CryptoFailed CryptoError_OutputLengthTooSmall | outLen > outputMaxLength = CryptoFailed CryptoError_OutputLengthTooBig - | otherwise = CryptoPassed $ B.allocAndFreeze outLen $ \out -> do - res <- B.withByteArray password $ \pPass -> - B.withByteArray salt $ \pSalt -> - argon2_hash - (iterations options) - (memory options) - (parallelism options) - pPass - (csizeOfInt passwordLen) - pSalt - (csizeOfInt saltLen) - out - (csizeOfInt outLen) - (cOfVariant $ variant options) - (cOfVersion $ version options) - when (res /= 0) $ error "argon2: hash: internal error" + | otherwise = unsafeDoIO $ do + -- the bounds on iterations, memory and parallelism are checked by the + -- C implementation, which reports them through its return code + (res, out) <- B.allocRet outLen $ \pOut -> + B.withByteArray password $ \pPass -> + B.withByteArray salt $ \pSalt -> + argon2_hash + (iterations options) + (memory options) + (parallelism options) + pPass + (csizeOfInt passwordLen) + pSalt + (csizeOfInt saltLen) + pOut + (csizeOfInt outLen) + (cOfVariant $ variant options) + (cOfVersion $ version options) + return $ + if res == 0 + then CryptoPassed out + else CryptoFailed CryptoError_ParameterInvalid where saltLen = B.length salt passwordLen = B.length password diff --git a/Crypto/KDF/BCrypt.hs b/Crypto/KDF/BCrypt.hs --- a/Crypto/KDF/BCrypt.hs +++ b/Crypto/KDF/BCrypt.hs @@ -37,6 +37,16 @@ -- if passwords are UTF-8 encoded (which they should be) and less than 256 -- characters long. -- +-- Only the first 72 bytes of a password are used. The rest is silently +-- ignored, so two passwords sharing a 72-byte prefix produce the same hash and +-- validate against each other. That is what the original implementation does +-- and is kept for compatibility, but it means a longer +-- passphrase buys nothing past that point, and the limit is on /bytes/ rather +-- than characters -- a UTF-8 passphrase reaches it sooner than its length in +-- characters suggests. Where passwords may be longer, hash them to a fixed +-- size first, or use "Crypto.KDF.Argon2" or "Crypto.KDF.Scrypt", which have no +-- such limit. +-- -- The cost parameter can be between 4 and 31 inclusive, but anything less than -- 10 is probably not strong enough. High values may be prohibitively slow -- depending on your hardware. Choose the highest value you can without having @@ -44,22 +54,17 @@ -- depending on the account, since it is unique to an individual hash. module Crypto.KDF.BCrypt ( hashPassword, + tryHashPassword, validatePassword, validatePasswordEither, bcrypt, + tryBcrypt, ) where -import Control.Monad (forM_, unless, when) -import Crypto.Cipher.Blowfish.Primitive ( - Context, - createKeySchedule, - encrypt, - expandKey, - expandKeyWithSalt, - freezeKeySchedule, - ) -import Crypto.Internal.Compat +import Control.Monad (unless, when) +import Crypto.Cipher.Blowfish.Primitive (bcryptHash) +import Crypto.Error import Crypto.Random (MonadRandom, getRandomBytes) import Data.ByteArray ( ByteArray, @@ -78,48 +83,94 @@ -- -- Each increment of the cost approximately doubles the time taken. -- The 16 bytes of random salt will be generated internally. +-- +-- A cost outside 4 to 31 raises 'CryptoError_ParameterInvalid'; +-- 'tryHashPassword' reports it instead. hashPassword :: (MonadRandom m, ByteArray password, ByteArray hash) => Int - -- ^ The cost parameter. Should be between 4 and 31 (inclusive). - -- Values which lie outside this range will be adjusted accordingly. + -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything + -- else is refused. -> password -- ^ The password. Should be the UTF-8 encoded bytes of the password text. + -- Only the first 72 bytes are used; see the module documentation. -> m hash -- ^ The bcrypt hash in standard format. -hashPassword cost password = do +hashPassword cost password = throwCryptoError <$> tryHashPassword cost password + +-- | Create a bcrypt hash for a password with a provided cost value, +-- reporting a cost the implementation refuses rather than raising. +-- +-- The salt is generated internally and is always the right length, so the +-- cost is the only thing here that can be wrong. +tryHashPassword + :: (MonadRandom m, ByteArray password, ByteArray hash) + => Int + -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything + -- else is reported. + -> password + -- ^ The password. Should be the UTF-8 encoded bytes of the password text. + -- Only the first 72 bytes are used; see the module documentation. + -> m (CryptoFailable hash) + -- ^ The bcrypt hash in standard format. +tryHashPassword cost password = do salt <- getRandomBytes 16 - return $ bcrypt cost (salt :: Bytes) password + return $ tryBcrypt cost (salt :: Bytes) password -- | Create a bcrypt hash for a password with a provided cost value and salt. -- --- Cost value under 4 will be automatically adjusted back to 10 for safety reason. +-- A cost outside 4 to 31, or a salt that is not 16 bytes long, raises +-- 'CryptoError_ParameterInvalid'; 'tryBcrypt' reports the same conditions as +-- 'CryptoFailed'. bcrypt :: (ByteArray salt, ByteArray password, ByteArray output) => Int - -- ^ The cost parameter. Should be between 4 and 31 (inclusive). - -- Values which lie outside this range will be adjusted accordingly. + -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything + -- else is refused. -> salt -- ^ The salt. Must be 16 bytes in length or an error will be raised. -> password -- ^ The password. Should be the UTF-8 encoded bytes of the password text. + -- Only the first 72 bytes are used; see the module documentation. -> output -- ^ The bcrypt hash in standard format. -bcrypt cost salt password = B.concat [header, B.snoc costBytes dollar, b64 salt, b64 hash] +bcrypt cost salt password = throwCryptoError (tryBcrypt cost salt password) + +-- | Create a bcrypt hash for a password with a provided cost value and salt, +-- reporting a parameter the implementation refuses rather than raising. +-- +-- bcrypt is defined for a cost of 4 to 31, and a cost outside that is +-- reported rather than replaced by one inside it: a caller that asks for +-- something this does not do should hear so, not receive a hash at a cost it +-- did not choose. +tryBcrypt + :: (ByteArray salt, ByteArray password, ByteArray output) + => Int + -- ^ The cost parameter. Must be between 4 and 31 inclusive; anything + -- else is refused. + -> salt + -- ^ The salt. Must be 16 bytes in length. + -> password + -- ^ The password. Should be the UTF-8 encoded bytes of the password text. + -- Only the first 72 bytes are used; see the module documentation. + -> CryptoFailable output + -- ^ The bcrypt hash in standard format. +tryBcrypt cost salt password + | cost < 4 || cost > 31 = CryptoFailed CryptoError_ParameterInvalid + | B.length salt /= 16 = CryptoFailed CryptoError_ParameterInvalid + | otherwise = + CryptoPassed $ + B.concat [header, B.snoc costBytes dollar, b64 salt, b64 hash] where - hash = rawHash 'b' realCost salt password + hash = rawHash 'b' cost salt password header = B.pack [dollar, fromIntegral (ord '2'), fromIntegral (ord 'b'), dollar] dollar = fromIntegral (ord '$') zero = fromIntegral (ord '0') costBytes = B.pack - [ zero + fromIntegral (realCost `div` 10) - , zero + fromIntegral (realCost `mod` 10) + [ zero + fromIntegral (cost `div` 10) + , zero + fromIntegral (cost `mod` 10) ] - realCost - | cost < 4 = 10 -- 4 is virtually pointless so go for 10 - | cost > 31 = 31 - | otherwise = cost b64 :: ByteArray ba => ba -> ba b64 = convertToBase Base64OpenBSD @@ -128,6 +179,9 @@ -- -- Returns @False@ if the password doesn't match the hash, or if the hash is -- invalid or an unsupported version. +-- +-- Only the first 72 bytes of the password are compared; see the module +-- documentation. validatePassword :: (ByteArray password, ByteArray hash) => password -> hash -> Bool validatePassword password bcHash = either (const False) id (validatePasswordEither password bcHash) @@ -135,7 +189,7 @@ -- | Check a password against a bcrypt hash -- -- As for @validatePassword@ but will provide error information if the hash is invalid or --- an unsupported version. +-- an unsupported version. The same 72-byte limit applies. validatePasswordEither :: (ByteArray password, ByteArray hash) => password -> hash -> Either String Bool validatePasswordEither password bcHash = do @@ -145,47 +199,12 @@ rawHash :: (ByteArrayAccess salt, ByteArray password, ByteArray output) => Char -> Int -> salt -> password -> output -rawHash _ cost salt password = B.take 23 hash -- Another compatibility bug. Ignore last byte of hash +rawHash _ cost salt password = case bcryptHash cost salt key of + Just hash -> B.take 23 hash -- Another compatibility bug. Ignore last byte of hash + Nothing -> error "bcrypt: the cost or the salt is not one bcrypt takes" where - hash = loop (0 :: Int) orpheanBeholder - - loop i input - | i < 64 = loop (i + 1) (encrypt ctx input) - | otherwise = input - -- Truncate the password if necessary and append a null byte for C compatibility - key = B.snoc (B.take 72 password) 0 - - ctx = expensiveBlowfishContext key salt cost - - -- The BCrypt plaintext: "OrpheanBeholderScryDoubt" - orpheanBeholder = - B.pack - [ 79 - , 114 - , 112 - , 104 - , 101 - , 97 - , 110 - , 66 - , 101 - , 104 - , 111 - , 108 - , 100 - , 101 - , 114 - , 83 - , 99 - , 114 - , 121 - , 68 - , 111 - , 117 - , 98 - , 116 - ] + key = B.snoc (B.take 72 (B.convert password :: Bytes)) 0 -- "$2a$10$XajjQvNhvvRt5GSeFk1xFeyqRrsxkhBkUiQeg0dt.wU1qD4aFDcga" parseBCryptHash :: ByteArray ba => ba -> Either String BCryptHash @@ -217,20 +236,3 @@ salt <- convertFromBase Base64OpenBSD s hash <- convertFromBase Base64OpenBSD h return (salt, hash) - --- | Create a key schedule for the BCrypt "EKS" version. --- --- Salt must be a 128-bit byte array. --- Cost must be between 4 and 31 inclusive --- See -expensiveBlowfishContext - :: (ByteArrayAccess key, ByteArrayAccess salt) => key -> salt -> Int -> Context -expensiveBlowfishContext keyBytes saltBytes cost - | B.length saltBytes /= 16 = error "bcrypt salt must be 16 bytes" - | otherwise = unsafeDoIO $ do - ks <- createKeySchedule - expandKeyWithSalt ks keyBytes saltBytes - forM_ [1 .. 2 ^ cost :: Int] $ \_ -> do - expandKey ks keyBytes - expandKey ks saltBytes - freezeKeySchedule ks diff --git a/Crypto/KDF/BCryptPBKDF.hs b/Crypto/KDF/BCryptPBKDF.hs --- a/Crypto/KDF/BCryptPBKDF.hs +++ b/Crypto/KDF/BCryptPBKDF.hs @@ -9,14 +9,16 @@ module Crypto.KDF.BCryptPBKDF ( Parameters (..), generate, + tryGenerate, hashInternal, + tryHashInternal, ) where import qualified Control.Exception as E import Control.Monad (when) -import qualified Crypto.Cipher.Blowfish.Box as Blowfish -import qualified Crypto.Cipher.Blowfish.Primitive as Blowfish +import Crypto.Cipher.Blowfish.Primitive (bcryptPbkdfHash) +import Crypto.Error import Crypto.Hash.Algorithms (SHA512 (..)) import Crypto.Hash.Types ( Context, @@ -48,17 +50,30 @@ deriving (Eq, Ord, Show) -- | Derive a key of specified length using the bcrypt_pbkdf algorithm. +-- +-- Parameters outside the ranges documented for t'Parameters' raise +-- 'CryptoError_ParameterInvalid'; 'tryGenerate' reports the same condition as +-- 'CryptoFailed'. generate :: (B.ByteArray pass, B.ByteArray salt, B.ByteArray output) => Parameters -> pass -> salt -> output -generate params pass salt - | iterCounts params < 1 = error "BCryptPBKDF: iterCounts must be > 0" - | keyLen < 1 || keyLen > 1024 = - error "BCryptPBKDF: outputLength must be in 1..1024" - | otherwise = B.unsafeCreate keyLen deriveKey +generate params pass salt = throwCryptoError (tryGenerate params pass salt) + +-- | Derive a key of specified length using the bcrypt_pbkdf algorithm, +-- reporting parameters the implementation refuses rather than raising. +tryGenerate + :: (B.ByteArray pass, B.ByteArray salt, B.ByteArray output) + => Parameters + -> pass + -> salt + -> CryptoFailable output +tryGenerate params pass salt + | iterCounts params < 1 = CryptoFailed CryptoError_ParameterInvalid + | keyLen < 1 || keyLen > 1024 = CryptoFailed CryptoError_ParameterInvalid + | otherwise = CryptoPassed $ B.unsafeCreate keyLen deriveKey where outLen, tmpLen, blkLen, keyLen, passLen, saltLen, ctxLen, hashLen, blocks :: Int outLen = 32 @@ -76,8 +91,6 @@ -- Allocate all necessary memory. The algorithm shall not allocate -- any more dynamic memory after this point. ForeignPtrs allocate -- pinned memory, so raw pointers to them are stable. - ksClean <- Blowfish.createKeySchedule - ksDirty <- Blowfish.createKeySchedule ctxFP <- mallocForeignPtrBytes ctxLen :: IO (ForeignPtr Word8) outFP <- mallocForeignPtrBytes outLen :: IO (ForeignPtr Word8) tmpFP <- mallocForeignPtrBytes tmpLen :: IO (ForeignPtr Word8) @@ -116,8 +129,7 @@ hashInternalUpdate shaPtr blkPtr (fromIntegral blkLen) hashInternalFinalize shaPtr (castPtr saltHashPtr) let saltHashBS = BSI.fromForeignPtr saltHashFP 0 hashLen - Blowfish.copyKeySchedule ksDirty ksClean - hashInternalMutable ksDirty passHashBS saltHashBS tmpPtr + hashInternalMutable passHashBS saltHashBS tmpPtr memCopy outPtr tmpPtr outLen -- Remaining rounds. forM_ [2 .. iterCounts params] $ const $ do @@ -125,8 +137,7 @@ hashInternalUpdate shaPtr tmpPtr (fromIntegral tmpLen) hashInternalFinalize shaPtr (castPtr saltHashPtr) let saltHashBS2 = BSI.fromForeignPtr saltHashFP 0 hashLen - Blowfish.copyKeySchedule ksDirty ksClean - hashInternalMutable ksDirty passHashBS saltHashBS2 tmpPtr + hashInternalMutable passHashBS saltHashBS2 tmpPtr memXor outPtr outPtr tmpPtr outLen -- Spread the current out buffer evenly over the key buffer. -- After both loops have run every byte of the key buffer @@ -141,49 +152,40 @@ -- | Internal hash function used by `generate`. -- -- Normal users should not need this. +-- +-- Inputs that are not 512 bits long raise 'CryptoError_ParameterInvalid'; +-- 'tryHashInternal' reports the same condition as 'CryptoFailed'. hashInternal :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt, B.ByteArray output) => pass -> salt -> output -hashInternal passHash saltHash - | B.length passHash /= 64 = error "passHash must be 512 bits" - | B.length saltHash /= 64 = error "saltHash must be 512 bits" - | otherwise = unsafeDoIO $ do - ks0 <- Blowfish.createKeySchedule - B.alloc 32 $ \outPtr -> hashInternalMutable ks0 passHash saltHash outPtr +hashInternal passHash saltHash = + throwCryptoError (tryHashInternal passHash saltHash) +-- | Internal hash function used by 'tryGenerate', reporting inputs the +-- implementation refuses rather than raising. +-- +-- Normal users should not need this. +tryHashInternal + :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt, B.ByteArray output) + => pass + -> salt + -> CryptoFailable output +tryHashInternal passHash saltHash + | B.length passHash /= 64 = CryptoFailed CryptoError_ParameterInvalid + | B.length saltHash /= 64 = CryptoFailed CryptoError_ParameterInvalid + | otherwise = CryptoPassed $ unsafeDoIO $ do + B.alloc 32 $ \outPtr -> hashInternalMutable passHash saltHash outPtr + hashInternalMutable :: (B.ByteArrayAccess pass, B.ByteArrayAccess salt) - => Blowfish.KeySchedule - -> pass + => pass -> salt -> Ptr Word8 -> IO () -hashInternalMutable bfks passHash saltHash outPtr = do - Blowfish.expandKeyWithSalt bfks passHash saltHash - forM_ [0 .. 63 :: Int] $ const $ do - Blowfish.expandKey bfks saltHash - Blowfish.expandKey bfks passHash - -- "OxychromaticBlowfishSwatDynamite" represented as 4 Word64 in big-endian. - store 0 =<< cipher 64 0x4f78796368726f6d - store 8 =<< cipher 64 0x61746963426c6f77 - store 16 =<< cipher 64 0x6669736853776174 - store 24 =<< cipher 64 0x44796e616d697465 - where - store :: Int -> Word64 -> IO () - store o w64 = do - pokeByteOff outPtr (o + 0) (fromIntegral (w64 `shiftR` 32) :: Word8) - pokeByteOff outPtr (o + 1) (fromIntegral (w64 `shiftR` 40) :: Word8) - pokeByteOff outPtr (o + 2) (fromIntegral (w64 `shiftR` 48) :: Word8) - pokeByteOff outPtr (o + 3) (fromIntegral (w64 `shiftR` 56) :: Word8) - pokeByteOff outPtr (o + 4) (fromIntegral (w64 `shiftR` 0) :: Word8) - pokeByteOff outPtr (o + 5) (fromIntegral (w64 `shiftR` 8) :: Word8) - pokeByteOff outPtr (o + 6) (fromIntegral (w64 `shiftR` 16) :: Word8) - pokeByteOff outPtr (o + 7) (fromIntegral (w64 `shiftR` 24) :: Word8) - cipher :: Int -> Word64 -> IO Word64 - cipher 0 block = return block - cipher i block = Blowfish.cipherBlockMutable bfks block >>= cipher (i - 1) +hashInternalMutable passHash saltHash outPtr = + bcryptPbkdfHash passHash saltHash outPtr finallyErase :: ForeignPtr Word8 -> Int -> IO () -> IO () finallyErase fp len action = diff --git a/Crypto/KDF/HKDF.hs b/Crypto/KDF/HKDF.hs --- a/Crypto/KDF/HKDF.hs +++ b/Crypto/KDF/HKDF.hs @@ -1,4 +1,5 @@ {-# LANGUAGE BangPatterns #-} +{-# LANGUAGE ScopedTypeVariables #-} -- | -- Module : Crypto.KDF.HKDF @@ -15,9 +16,11 @@ extract, extractSkip, expand, + tryExpand, toPRK, ) where +import Crypto.Error import Crypto.Hash import Crypto.Internal.ByteArray ( ByteArray, @@ -60,8 +63,13 @@ extractSkip ikm = PRK_NoExpand $ B.convert ikm -- | Expand key material of specific length out of the parameters +-- +-- Requests exceeding the RFC 5869 limit of @255 * HashLen@ raise +-- 'CryptoError_OutputLengthTooBig'; 'tryExpand' reports the same condition as +-- 'CryptoFailed'. expand - :: (HashAlgorithm a, ByteArrayAccess info, ByteArray out) + :: forall a info out + . (HashAlgorithm a, ByteArrayAccess info, ByteArray out) => PRK a -- ^ Pseudo Random Key -> info @@ -71,8 +79,27 @@ -> out -- ^ Output data expand prkAt infoAt outputLength = - let hF = hFGet prkAt - in B.concat $ loop hF B.empty outputLength 1 + throwCryptoError (tryExpand prkAt infoAt outputLength) + +-- | Expand key material of specific length out of the parameters, reporting a +-- length the RFC refuses rather than raising. +tryExpand + :: forall a info out + . (HashAlgorithm a, ByteArrayAccess info, ByteArray out) + => PRK a + -- ^ Pseudo Random Key + -> info + -- ^ Optional context and application specific information + -> Int + -- ^ Output length in bytes + -> CryptoFailable out + -- ^ Output data +tryExpand prkAt infoAt outputLength + | outputLength > 255 * hashDigestSize (undefined :: a) = + CryptoFailed CryptoError_OutputLengthTooBig + | otherwise = + let hF = hFGet prkAt + in CryptoPassed $ B.concat $ loop hF B.empty outputLength 1 where hFGet :: (HashAlgorithm a, ByteArrayAccess b) => PRK a -> (b -> HMAC a) hFGet prk = case prk of diff --git a/Crypto/KDF/PBKDF2.hs b/Crypto/KDF/PBKDF2.hs --- a/Crypto/KDF/PBKDF2.hs +++ b/Crypto/KDF/PBKDF2.hs @@ -14,9 +14,13 @@ prfHMAC, Parameters (..), generate, + tryGenerate, fastPBKDF2_SHA1, + tryFastPBKDF2_SHA1, fastPBKDF2_SHA256, + tryFastPBKDF2_SHA256, fastPBKDF2_SHA512, + tryFastPBKDF2_SHA512, ) where import Data.Bits @@ -25,6 +29,7 @@ import Foreign.Marshal.Alloc import Foreign.Ptr (Ptr, plusPtr) +import Crypto.Error import Crypto.Hash (HashAlgorithm) import qualified Crypto.MAC.HMAC as HMAC @@ -55,11 +60,27 @@ data Parameters = Parameters { iterCounts :: Int -- ^ the number of user-defined iterations for the algorithms. e.g. WPA2 uses 4000. + -- (must be > 0) , outputLength :: Int -- ^ the number of bytes to generate out of PBKDF2 + -- (must not be negative) } +-- | Report parameters no PBKDF2 entry point accepts. +-- +-- An iteration count below one derives a key that is not a key at all, and a +-- negative output length asks for a buffer that cannot be allocated. +validateParameters :: Parameters -> Maybe CryptoError +validateParameters params + | iterCounts params < 1 = Just CryptoError_ParameterInvalid + | outputLength params < 0 = Just CryptoError_ParameterInvalid + | otherwise = Nothing + -- | generate the pbkdf2 key derivation function from the output +-- +-- Parameters outside the ranges documented for t'Parameters' raise +-- 'CryptoError_ParameterInvalid'; 'tryGenerate' reports the same condition as +-- 'CryptoFailed'. generate :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray ba) => PRF password @@ -68,7 +89,20 @@ -> salt -> ba generate prf params password salt = - B.allocAndFreeze (outputLength params) $ \p -> do + throwCryptoError (tryGenerate prf params password salt) + +-- | generate the pbkdf2 key derivation function from the output, reporting +-- parameters the implementation refuses rather than raising. +tryGenerate + :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray ba) + => PRF password + -> Parameters + -> password + -> salt + -> CryptoFailable ba +tryGenerate prf params password salt + | Just err <- validateParameters params = CryptoFailed err + | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \p -> do memSet p 0 (outputLength params) loop 1 (outputLength params) p where @@ -113,8 +147,13 @@ b = fromIntegral ((w `shiftR` 16) .&. 0xff) c = fromIntegral ((w `shiftR` 8) .&. 0xff) d = fromIntegral (w .&. 0xff) -{-# NOINLINE generate #-} +{-# NOINLINE tryGenerate #-} +-- | PBKDF2 with HMAC-SHA1, using the bundled C implementation. +-- +-- Parameters outside the ranges documented for t'Parameters' raise +-- 'CryptoError_ParameterInvalid'; 'tryFastPBKDF2_SHA1' reports the same condition +-- as 'CryptoFailed'. fastPBKDF2_SHA1 :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) => Parameters @@ -122,7 +161,19 @@ -> salt -> out fastPBKDF2_SHA1 params password salt = - B.allocAndFreeze (outputLength params) $ \outPtr -> + throwCryptoError (tryFastPBKDF2_SHA1 params password salt) + +-- | PBKDF2 with HMAC-SHA1, reporting parameters the implementation refuses +-- rather than raising. +tryFastPBKDF2_SHA1 + :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) + => Parameters + -> password + -> salt + -> CryptoFailable out +tryFastPBKDF2_SHA1 params password salt + | Just err <- validateParameters params = CryptoFailed err + | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \outPtr -> B.withByteArray password $ \passPtr -> B.withByteArray salt $ \saltPtr -> c_crypton_fastpbkdf2_hmac_sha1 @@ -134,6 +185,11 @@ outPtr (fromIntegral $ outputLength params) +-- | PBKDF2 with HMAC-SHA256, using the bundled C implementation. +-- +-- Parameters outside the ranges documented for t'Parameters' raise +-- 'CryptoError_ParameterInvalid'; 'tryFastPBKDF2_SHA256' reports the same condition +-- as 'CryptoFailed'. fastPBKDF2_SHA256 :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) => Parameters @@ -141,7 +197,19 @@ -> salt -> out fastPBKDF2_SHA256 params password salt = - B.allocAndFreeze (outputLength params) $ \outPtr -> + throwCryptoError (tryFastPBKDF2_SHA256 params password salt) + +-- | PBKDF2 with HMAC-SHA256, reporting parameters the implementation refuses +-- rather than raising. +tryFastPBKDF2_SHA256 + :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) + => Parameters + -> password + -> salt + -> CryptoFailable out +tryFastPBKDF2_SHA256 params password salt + | Just err <- validateParameters params = CryptoFailed err + | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \outPtr -> B.withByteArray password $ \passPtr -> B.withByteArray salt $ \saltPtr -> c_crypton_fastpbkdf2_hmac_sha256 @@ -153,6 +221,11 @@ outPtr (fromIntegral $ outputLength params) +-- | PBKDF2 with HMAC-SHA512, using the bundled C implementation. +-- +-- Parameters outside the ranges documented for t'Parameters' raise +-- 'CryptoError_ParameterInvalid'; 'tryFastPBKDF2_SHA512' reports the same condition +-- as 'CryptoFailed'. fastPBKDF2_SHA512 :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) => Parameters @@ -160,7 +233,19 @@ -> salt -> out fastPBKDF2_SHA512 params password salt = - B.allocAndFreeze (outputLength params) $ \outPtr -> + throwCryptoError (tryFastPBKDF2_SHA512 params password salt) + +-- | PBKDF2 with HMAC-SHA512, reporting parameters the implementation refuses +-- rather than raising. +tryFastPBKDF2_SHA512 + :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray out) + => Parameters + -> password + -> salt + -> CryptoFailable out +tryFastPBKDF2_SHA512 params password salt + | Just err <- validateParameters params = CryptoFailed err + | otherwise = CryptoPassed $ B.allocAndFreeze (outputLength params) $ \outPtr -> B.withByteArray password $ \passPtr -> B.withByteArray salt $ \saltPtr -> c_crypton_fastpbkdf2_hmac_sha512 diff --git a/Crypto/KDF/Scrypt.hs b/Crypto/KDF/Scrypt.hs --- a/Crypto/KDF/Scrypt.hs +++ b/Crypto/KDF/Scrypt.hs @@ -14,6 +14,7 @@ module Crypto.KDF.Scrypt ( Parameters (..), generate, + tryGenerate, ) where import Control.Monad (forM_) @@ -21,6 +22,7 @@ import Foreign.Marshal.Alloc import Foreign.Ptr (Ptr, plusPtr) +import Crypto.Error import Crypto.Hash (SHA256 (..)) import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) import qualified Crypto.Internal.ByteArray as B @@ -44,18 +46,31 @@ :: Ptr Word8 -> Word32 -> Word64 -> Ptr Word8 -> Ptr Word8 -> IO () -- | Generate the scrypt key derivation data +-- +-- Parameters the implementation refuses raise a 'CryptoError'; 'tryGenerate' +-- reports the same condition as 'CryptoFailed'. generate :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray output) => Parameters -> password -> salt -> output -generate params password salt - | r params * p params >= 0x40000000 = - error "Scrypt: invalid parameters: r and p constraint" - | popCount (n params) /= 1 = - error "Scrypt: invalid parameters: n not a power of 2" - | otherwise = unsafeDoIO $ do +generate params password salt = throwCryptoError (tryGenerate params password salt) + +-- | Generate the scrypt key derivation data, reporting parameters the +-- implementation refuses rather than raising. +-- +-- @n@ has to be a power of two, and @r@ times @p@ has to stay below 2^30. +tryGenerate + :: (ByteArrayAccess password, ByteArrayAccess salt, ByteArray output) + => Parameters + -> password + -> salt + -> CryptoFailable output +tryGenerate params password salt + | r params * p params >= 0x40000000 = CryptoFailed CryptoError_ParameterInvalid + | popCount (n params) /= 1 = CryptoFailed CryptoError_ParameterInvalid + | otherwise = CryptoPassed $ unsafeDoIO $ do let b = PBKDF2.generate prf (PBKDF2.Parameters 1 intLen) password salt :: B.Bytes newSalt <- B.copy b $ \bPtr -> allocaBytesAligned (128 * (fromIntegral $ n params) * (r params)) 8 $ \v -> @@ -77,4 +92,4 @@ where prf = PBKDF2.prfHMAC SHA256 intLen = p params * 128 * r params -{-# NOINLINE generate #-} +{-# NOINLINE tryGenerate #-} diff --git a/Crypto/MAC/CMAC.hs b/Crypto/MAC/CMAC.hs --- a/Crypto/MAC/CMAC.hs +++ b/Crypto/MAC/CMAC.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE BangPatterns #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-} -- | @@ -17,11 +18,12 @@ ) where import Data.Bits (setBit, shiftL, testBit) -import Data.List (foldl') +import Data.ByteString (ByteString) +import qualified Data.ByteString as S import Data.Word -import Prelude hiding (foldl') import Crypto.Cipher.Types +import Crypto.Cipher.Types.Block (IV (..)) import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B @@ -41,28 +43,46 @@ -- ^ input message -> CMAC cipher -- ^ output tag -cmac k msg = - CMAC $ foldl' (\c m -> ecbEncrypt k $ bxor c m) zeroV ms +cmac k msg = CMAC $ B.convert $ step (chain zeroV whole) final where bytes = blockSize k - zeroV = B.replicate bytes 0 :: Bytes + zeroV = S.replicate bytes 0 (k1, k2) = subKeys k - ms = cmacChunks k k1 k2 $ B.convert msg -cmacChunks :: (BlockCipher k, ByteArray ba) => k -> ba -> ba -> ba -> [ba] -cmacChunks k k1 k2 = rec' - where - rec' msg - | B.null tl = - if lack == 0 - then [bxor k1 hd] - else [bxor k2 $ hd `B.append` B.pack (0x80 : replicate (lack - 1) 0)] - | otherwise = hd : rec' tl - where - bytes = blockSize k - (hd, tl) = B.splitAt bytes msg - lack = bytes - B.length hd + -- The message is held as a ByteString and sliced, never consumed. 'Bytes' + -- has no shared representation, so splitting one repeatedly -- which is + -- what this used to do, once per block -- copied whatever was left of the + -- message each time, and so the message about n/2 times in all. + msgBytes = B.convert msg :: ByteString + msgLen = S.length msgBytes + -- the last block is the one the subkeys are for, and it is a whole block + -- only when there is one to be had + lastLen + | msgLen > 0 && msgLen `mod` bytes == 0 = bytes + | otherwise = msgLen `mod` bytes + (whole, rest) = S.splitAt (msgLen - lastLen) msgBytes + final + | lastLen == bytes = bxor k1 rest + | otherwise = + bxor k2 $ + S.concat [rest, S.singleton 0x80, S.replicate (bytes - lastLen - 1) 0] + + -- CMAC chains its blocks the way CBC does, so the running state is the + -- last ciphertext block of a CBC encryption. Handing the cipher a chunk + -- at a time rather than a block at a time is what makes that worth saying: + -- for AES it reaches the C implementation of CBC, where a block at a time + -- reached a foreign call per sixteen bytes. + chunkBytes = bytes * 2048 + chain !c bs + | S.null bs = c + | otherwise = + let (hd, tl) = S.splitAt chunkBytes bs + out = cbcEncrypt k (IV c) hd + in chain (S.drop (S.length hd - bytes) out) tl + + step c m = ecbEncrypt k (bxor c m) :: ByteString + -- | make sub-keys used in CMAC subKeys :: (BlockCipher k, ByteArray ba) @@ -102,7 +122,7 @@ sl1 = shiftL x 1 bxor :: ByteArray ba => ba -> ba -> ba -bxor = B.xor +bxor = B.bxor ----- diff --git a/Crypto/MAC/HMAC.hs b/Crypto/MAC/HMAC.hs --- a/Crypto/MAC/HMAC.hs +++ b/Crypto/MAC/HMAC.hs @@ -63,7 +63,7 @@ hmacLazy secret msg = finalize $ updates (initialize secret) (L.toChunks msg) -- | Represent an ongoing HMAC state, that can be appended with 'update' --- and finalize to an HMAC with 'hmacFinalize' +-- and finalize to an HMAC with 'finalize' data Context hashalg = Context !(Hash.Context hashalg) !(Hash.Context hashalg) -- | Initialize a new incremental HMAC context diff --git a/Crypto/MAC/KMAC.hs b/Crypto/MAC/KMAC.hs --- a/Crypto/MAC/KMAC.hs +++ b/Crypto/MAC/KMAC.hs @@ -99,7 +99,7 @@ kmac str key msg = finalize $ updates (initialize str key) [msg] -- | Represent an ongoing KMAC state, that can be appended with 'update' and --- finalized to a 'KMAC' with 'finalize'. +-- finalized to a t'KMAC' with 'finalize'. newtype Context a = Context (H.Context a) -- | Initialize a new incremental KMAC context with the supplied customization diff --git a/Crypto/MAC/KeyedBlake2.hs b/Crypto/MAC/KeyedBlake2.hs --- a/Crypto/MAC/KeyedBlake2.hs +++ b/Crypto/MAC/KeyedBlake2.hs @@ -48,7 +48,7 @@ KeyedBlake2 x == KeyedBlake2 y = B.constEq x y -- | Represent an ongoing Blake2 state, that can be appended with 'update' and --- finalized to a 'KeyedBlake2' with 'finalize'. +-- finalized to a t'KeyedBlake2' with 'finalize'. newtype Context a = Context (H.Context a) -- | Initialize a new incremental keyed Blake2 context with the supplied key. diff --git a/Crypto/MAC/Poly1305.hs b/Crypto/MAC/Poly1305.hs --- a/Crypto/MAC/Poly1305.hs +++ b/Crypto/MAC/Poly1305.hs @@ -12,6 +12,8 @@ module Crypto.MAC.Poly1305 ( Ctx, State, + Key, + key, Auth (..), authTag, @@ -46,6 +48,20 @@ newtype State = State ScrubbedBytes deriving (ByteArrayAccess) +-- | A Poly1305 key: thirty-two bytes, and the length is checked here rather +-- than at every use. 'initialize' and 'auth' take one of these and cannot +-- fail, so a caller that holds a key does not carry an error case for a +-- length it already knows is right. +newtype Key = Key ScrubbedBytes + deriving (ByteArrayAccess, Eq, NFData) + +-- | Take thirty-two bytes for a key. A different length is reported as +-- 'CryptoError_MacKeyInvalid'; nothing else about a key can be wrong. +key :: ByteArrayAccess ba => ba -> CryptoFailable Key +key k + | B.length k /= 32 = CryptoFailed CryptoError_MacKeyInvalid + | otherwise = CryptoPassed $ Key $ B.convert k + -- | Poly1305 State. use State instead of Ctx type Ctx = State @@ -63,6 +79,12 @@ instance Eq Auth where (Auth a1) == (Auth a2) = B.constEq a1 a2 +-- | @sizeof(poly1305_ctx)@: the accumulator and the key, either as the +-- limbs the C implementation works in or as the state the assembly keeps, +-- and the buffer for a partial block. See @cbits/crypton_poly1305.h@. +sizeCtx :: Int +sizeCtx = 232 + foreign import ccall unsafe "crypton_poly1305.h crypton_poly1305_init" c_poly1305_init :: Ptr State -> Ptr Word8 -> IO () @@ -73,15 +95,10 @@ c_poly1305_finalize :: Ptr Word8 -> Ptr State -> IO () -- | initialize a Poly1305 context -initialize - :: ByteArrayAccess key - => key - -> CryptoFailable State -initialize key - | B.length key /= 32 = CryptoFailed $ CryptoError_MacKeyInvalid - | otherwise = CryptoPassed $ State $ B.allocAndFreeze 84 $ \ctxPtr -> - B.withByteArray key $ \keyPtr -> - c_poly1305_init (castPtr ctxPtr) keyPtr +initialize :: Key -> State +initialize k = State $ B.allocAndFreeze sizeCtx $ \ctxPtr -> + B.withByteArray k $ \keyPtr -> + c_poly1305_init (castPtr ctxPtr) keyPtr {-# NOINLINE initialize #-} -- | update a context with a bytestring @@ -111,15 +128,13 @@ {-# NOINLINE finalize #-} -- | One-pass authorization creation -auth :: (ByteArrayAccess key, ByteArrayAccess ba) => key -> ba -> Auth -auth key d - | B.length key /= 32 = error "Poly1305: key length expected 32 bytes" - | otherwise = Auth $ B.allocAndFreeze 16 $ \dst -> do - _ <- B.alloc 84 (onCtx dst) :: IO ScrubbedBytes - return () +auth :: ByteArrayAccess ba => Key -> ba -> Auth +auth k d = Auth $ B.allocAndFreeze 16 $ \dst -> do + _ <- B.alloc sizeCtx (onCtx dst) :: IO ScrubbedBytes + return () where onCtx dst ctxPtr = - B.withByteArray key $ \keyPtr -> do + B.withByteArray k $ \keyPtr -> do c_poly1305_init (castPtr ctxPtr) keyPtr B.withByteArray d $ \dataPtr -> c_poly1305_update (castPtr ctxPtr) dataPtr (fromIntegral $ B.length d) diff --git a/Crypto/Number/Basic.hs b/Crypto/Number/Basic.hs --- a/Crypto/Number/Basic.hs +++ b/Crypto/Number/Basic.hs @@ -56,7 +56,7 @@ -- | Get the extended GCD of two integer using integer divMod -- --- gcde 'a' 'b' find (x,y,gcd(a,b)) where ax + by = d +-- gcde @a@ @b@ find (x,y,gcd(a,b)) where ax + by = d gcde :: Integer -> Integer -> (Integer, Integer, Integer) gcde a b = onGmpUnsupported (gmpGcde a b) $ @@ -118,8 +118,15 @@ (q, r) = i `divMod` 256 -- | Compute the number of bytes for an integer +-- +-- Out of 'numBits' rather than out of GMP's own count in base 256. GHC's +-- bignum sizes a number in base two by looking at its highest limb, and in +-- any other base -- 256 included -- by dividing the number down to nothing: +-- on a 2048-bit modulus that is 1.65 us against 0.01, and every serialization +-- here asks for the size before it allocates. Eight bits to the byte does +-- the rest. numBytes :: Integer -> Int -numBytes n = gmpSizeInBytes n `onGmpUnsupported` ((numBits n + 7) `div` 8) +numBytes n = (numBits n + 7) `div` 8 -- | Express an integer as an odd number and a power of 2 asPowerOf2AndOdd :: Integer -> (Int, Integer) diff --git a/Crypto/Number/F2m.hs b/Crypto/Number/F2m.hs --- a/Crypto/Number/F2m.hs +++ b/Crypto/Number/F2m.hs @@ -7,7 +7,7 @@ -- -- This module provides basic arithmetic operations over F₂m. Performance is -- not optimal and it doesn't provide protection against timing --- attacks. The 'm' parameter is implicitly derived from the irreducible +-- attacks. The @m@ parameter is implicitly derived from the irreducible -- polynomial where applicable. module Crypto.Number.F2m ( BinaryPolynomial, @@ -23,9 +23,21 @@ quadraticF2m, ) where +import Crypto.Internal.WordArray import Crypto.Number.Basic -import Data.Bits (setBit, shift, testBit, unsafeShiftR, xor) +import Data.Bits ( + setBit, + shift, + shiftL, + shiftR, + testBit, + unsafeShiftR, + xor, + (.&.), + (.|.), + ) import Data.List (foldl') +import Data.Word (Word32) import Prelude hiding (foldl') -- | Binary Polynomial represented by an integer @@ -53,17 +65,50 @@ error "modF2m: negative number represent no binary polynomial" | fx == 0 = error "modF2m: cannot divide by zero polynomial" | fx == 1 = 0 - | otherwise = go i + | otherwise = case tailExponents fx of + Just es -> fold es i + Nothing -> go i where lfx = log2 fx + -- one bit at a time, for a modulus with too many terms to be worth the + -- other way go n | s == 0 = n `addF2m` fx | s < 0 = n | otherwise = go $ n `addF2m` shift fx s where s = log2 n - lfx + + -- x^m is the rest of the modulus, so everything above bit m folds back in + -- as a copy of the number's top shifted by each of the modulus's lower + -- exponents: a handful of shifts, where the loop above takes one step per + -- bit of excess + mask = (1 `shiftL` lfx) - 1 + fold es n + | n <= mask = n + | otherwise = + fold + es + (foldl' (\acc e -> acc `xor` (hi `shiftL` e)) (n .&. mask) es) + where + hi = n `shiftR` lfx {-# INLINE modF2m #-} +-- | The exponents of a modulus below its leading term, when there are few +-- enough of them to reduce with. +-- +-- Every binary curve in use has a trinomial or a pentanomial here, which is +-- three or five exponents; sixteen is the point past which folding stops being +-- the cheaper way. +tailExponents :: BinaryPolynomial -> Maybe [Int] +tailExponents fx = go (log2 fx - 1) 0 [] + where + go i n acc + | i < 0 = Just acc + | n > 16 = Nothing + | testBit fx i = go (i - 1) (n + 1 :: Int) (i : acc) + | otherwise = go (i - 1) n acc + -- | Multiplication over F₂m. -- -- This function is undefined for negative arguments, because their bit @@ -80,14 +125,46 @@ || n2 < 0 = error "mulF2m: negative number represent no binary polynomial" | fx == 0 = error "mulF2m: cannot multiply modulo zero polynomial" - | otherwise = modF2m fx $ go (if n2 `mod` 2 == 1 then n1 else 0) (log2 n2) + | otherwise = modF2m fx (go n2 0 0) where - go n s - | s == 0 = n - | otherwise = - if testBit n2 s - then go (n `addF2m` shift n1 s) (s - 1) - else go n (s - 1) + -- Four bits of the multiplier at a time, against the sixteen multiples of + -- n1 that four bits can ask for. A bit at a time is four times the + -- shifting and exclusive-oring, and each of those allocates. + go 0 _ acc = acc + go v sh acc = + go (v `shiftR` 4) (sh + 4) (acc `xor` (multiple (v .&. 0xf) `shiftL` sh)) + + m2 = n1 `shiftL` 1 + m4 = n1 `shiftL` 2 + m8 = n1 `shiftL` 3 + m3 = m2 `xor` n1 + m5 = m4 `xor` n1 + m6 = m4 `xor` m2 + m7 = m6 `xor` n1 + m9 = m8 `xor` n1 + m10 = m8 `xor` m2 + m11 = m10 `xor` n1 + m12 = m8 `xor` m4 + m13 = m12 `xor` n1 + m14 = m12 `xor` m2 + m15 = m14 `xor` n1 + + multiple 1 = n1 + multiple 2 = m2 + multiple 3 = m3 + multiple 4 = m4 + multiple 5 = m5 + multiple 6 = m6 + multiple 7 = m7 + multiple 8 = m8 + multiple 9 = m9 + multiple 10 = m10 + multiple 11 = m11 + multiple 12 = m12 + multiple 13 = m13 + multiple 14 = m14 + multiple 15 = m15 + multiple _ = 0 {-# INLINEABLE mulF2m #-} -- | Squaring over F₂m. @@ -114,11 +191,29 @@ -> Integer squareF2m' n | n < 0 = error "mulF2m: negative number represent no binary polynomial" - | otherwise = - foldl' - (\acc s -> if testBit n s then setBit acc (2 * s) else acc) - 0 - [0 .. log2 n] + | otherwise = go n 0 0 + where + -- A byte at a time, through a table of the sixteen-bit patterns a byte + -- spreads into. A bit at a time is eight times the work, and setting a + -- bit of an Integer allocates another one. + go 0 _ acc = acc + go v sh acc = + go + (v `shiftR` 8) + (sh + 16) + ( acc + .|. (fromIntegral (arrayRead32 spreadTable (fromIntegral (v .&. 0xff))) `shiftL` sh) + ) + +-- | Each byte, with a zero inserted between every pair of its bits. +spreadTable :: Array32 +spreadTable = array32 256 [spread b | b <- [0 .. 255]] + where + spread :: Int -> Word32 + spread b = + foldl' (\acc i -> if testBit b i then setBit acc (2 * i) else acc) 0 [0 .. 7] +{-# NOINLINE spreadTable #-} + {-# INLINE squareF2m' #-} -- | Exponentiation in F₂m by computing @a^b mod fx@. diff --git a/Crypto/Number/ModArithmetic.hs b/Crypto/Number/ModArithmetic.hs --- a/Crypto/Number/ModArithmetic.hs +++ b/Crypto/Number/ModArithmetic.hs @@ -6,13 +6,31 @@ -- Maintainer : Vincent Hanquez -- Stability : experimental -- Portability : Good +-- +-- Modular arithmetic on 'Integer'. +-- +-- == What an 'Integer' shows +-- +-- An 'Integer' is as long as its value needs, and every operation on one +-- costs what that length says. A secret that happens to be short is +-- multiplied, reduced and compared in fewer words than a full-length one, and +-- the difference is there to be measured. 'expSafe' and 'inverseSafe' keep +-- the /value/ of an exponent or of a number being inverted out of the work +-- they do, and that is as far as an 'Integer' can be taken: hiding the length +-- as well means a fixed-width representation, which is what the curve modules +-- and 'expSafe' itself use underneath. module Crypto.Number.ModArithmetic ( + -- * Exceptions + CoprimesAssertionError (..), + ModulusAssertionError (..), + -- * Exponentiation expSafe, expFast, -- * Inverse computing inverse, + inverseSafe, inverseCoprimes, inverseFermat, @@ -22,8 +40,15 @@ ) where import qualified Control.Exception as E +import Crypto.Internal.Compat (unsafeDoIO) import Crypto.Number.Basic import Crypto.Number.Compat +import qualified Crypto.Number.Serialize.Internal as Internal +import Data.Memory.PtrMethods (memSet) +import Data.Word (Word32, Word8) +import Foreign.C.Types (CInt (..)) +import Foreign.Marshal.Alloc (allocaBytes) +import Foreign.Ptr (Ptr, plusPtr) -- | Raised when two numbers are supposed to be coprimes but are not. data CoprimesAssertionError = CoprimesAssertionError @@ -37,12 +62,25 @@ -- Modulo need to be odd otherwise the normal fast modular exponentiation -- is used. -- --- When used with integer-simple, this function is not different --- from expFast, and thus provide the same unstudied and dubious --- timing and side channels claims. +-- With an odd modulo the work is done in C, four bits of exponent at a time: +-- four squarings and one multiplication by a small power of the base, taken +-- from a table of sixteen which is read by touching every entry and keeping +-- one of them with a mask. So each group of four bits costs the same five +-- multiplications and the same sixteen reads whatever those bits are, and +-- nothing branches on the exponent or indexes memory with it. -- --- Before GHC 8.4.2, powModSecInteger is missing from integer-gmp, --- so expSafe has the same security as expFast. +-- What the exponent still shows is its length: it is rounded up to a whole +-- 64-bit word and every bit of that is walked over, so its value is hidden +-- but its size is not. The @mpz_powm_sec@ of GMP, which GHC stopped +-- offering in integer-gmp 1.1 and which this replaces, hides exactly as much. +-- +-- The base is taken to be public -- in this library it is a ciphertext, a +-- public value from a peer, or a generator -- and is reduced modulo the +-- modulus in the ordinary way first. +-- +-- Hiding the exponent has a price: against the windowed exponentiation of +-- GMP, which is what this function used to end up calling, a 2048-bit +-- modulus costs somewhat over twice as much. expSafe :: Integer -- ^ base @@ -53,15 +91,67 @@ -> Integer -- ^ result expSafe b e m - | odd m = - gmpPowModSecInteger b e m - `onGmpUnsupported` ( gmpPowModInteger b e m - `onGmpUnsupported` exponentiation b e m - ) + | odd m && m > 1 && e >= 0 = + gmpPowModSecInteger b e m `onGmpUnsupported` expSec (b `mod` m) e m + -- a modulus of one, and a negative exponent asking for an inverse, are + -- left to the path they have always taken | otherwise = gmpPowModInteger b e m `onGmpUnsupported` exponentiation b e m +-- | The windowed exponentiation itself, in C. The base has to be reduced +-- already, the exponent to be zero or more, and the modulus odd and above +-- one. +expSec :: Integer -> Integer -> Integer -> Integer +expSec b e m = unsafeDoIO $ + allocaBytes (sum widths) $ \start -> case scanl plusPtr start widths of + (out : base : expo : modu : _) -> do + _ <- Internal.i2ospOf b base mLen + _ <- Internal.i2ospOf e expo eLen + _ <- Internal.i2ospOf m modu mLen + r <- + c_powm_sec + out + base + (fromIntegral mLen) + expo + (fromIntegral eLen) + modu + (fromIntegral mLen) + -- the exponent is the caller's secret, and this is the last place it + -- is written out in the clear + memSet expo 0 eLen + if r == 0 + then do + !v <- Internal.os2ip out mLen + return v + else + return + ( gmpPowModInteger b e m + `onGmpUnsupported` exponentiation b e m + ) + _ -> return 0 -- there are four, but say so anyway + where + !mLen = numBytes m + -- the answer, the base, the exponent and the modulus. The room to take + -- and where each one starts both come from here, so they cannot drift + -- apart. + widths = [mLen, mLen, eLen, mLen] + -- whole words of exponent, so that the count of them says as little as + -- what GMP's own secure exponentiation lets slip + !eLen = 8 * ((numBytes e + 7) `div` 8) + +foreign import ccall safe "crypton_powm_sec" + c_powm_sec + :: Ptr Word8 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Word32 + -> Ptr Word8 + -> Word32 + -> IO CInt + -- | Compute the modular exponentiation of base^exponent using -- the fastest algorithm without any consideration for -- hiding parameters. @@ -105,7 +195,7 @@ -- is known to exists. -- -- If the numbers are not defined as coprime, this function --- will raise a 'CoprimesAssertionError'. +-- will raise a t'CoprimesAssertionError'. inverseCoprimes :: Integer -> Integer -> Integer inverseCoprimes g m = case inverse g m of @@ -144,6 +234,27 @@ inverseFermat :: Integer -> Integer -> Integer inverseFermat g p = expSafe g (p - 2) p +-- | @inverseSafe@ computes the modular inverse without letting the number +-- being inverted steer how long the work takes, which is what 'inverse' does: +-- the extended Euclidean algorithm takes a number of steps that follows the +-- bits it is given, and a nonce inverted that way has been taken apart before +-- by watching the steps go by. +-- +-- The moduli this is for -- the order of a group -- are prime, so the inverse +-- comes from 'inverseFermat' instead. When the modulus is not prime, that +-- answer is not an inverse, and the result is checked and 'inverse' asked +-- instead, so this agrees with 'inverse' on every input. That fallback is +-- reached only by parameters that are already broken. +-- +-- It costs what an exponentiation costs: around thirty times an 'inverse' +-- for a 256-bit modulus. +inverseSafe :: Integer -> Integer -> Maybe Integer +inverseSafe g m + | m > 1 && (g * r) `mod` m == 1 = Just r + | otherwise = inverse g m + where + r = inverseFermat g m + -- | Raised when the assumption about the modulus is invalid. data ModulusAssertionError = ModulusAssertionError deriving (Show) @@ -153,7 +264,7 @@ -- | Modular square root of @g@ modulo a prime @p@. -- -- If the modulus is found not to be prime, the function will raise a --- 'ModulusAssertionError'. +-- t'ModulusAssertionError'. -- -- This implementation is variable time and should be used with public -- parameters only. diff --git a/Crypto/Number/Prime.hs b/Crypto/Number/Prime.hs --- a/Crypto/Number/Prime.hs +++ b/Crypto/Number/Prime.hs @@ -23,23 +23,69 @@ import Crypto.Number.Compat import Crypto.Number.Generate import Crypto.Number.ModArithmetic (expSafe) +import Crypto.Number.Serialize (i2osp) import Crypto.Random.Probabilistic import Crypto.Random.Types +import Crypto.Internal.ByteArray (Bytes) + import Data.Bits -- | Returns if the number is probably prime. --- First a list of small primes are implicitely tested for divisibility, --- then a fermat primality test is used with arbitrary numbers and --- then the Miller Rabin algorithm is used with an accuracy of 30 recursions. +-- +-- The small primes are tested for divisibility first, and then the +-- Miller-Rabin algorithm with an accuracy of 30 rounds. +-- +-- A Fermat test of fifty consecutive bases used to run between the two. It +-- ruled out nothing Miller-Rabin does not: a strong probable prime to a base +-- is a Fermat probable prime to that base, and the converse is what Carmichael +-- numbers are. What it cost was fifty modular exponentiations on every number +-- that turned out to be prime -- 2167 of the 3480 microseconds spent on a +-- 512-bit prime, and about two thirds of the time to generate one. isProbablyPrime :: Integer -> Bool -isProbablyPrime !n +isProbablyPrime = probablyPrime 30 + +-- | The same, with the number of rounds said outright. +-- +-- Thirty rounds is what a number from anywhere gets: whoever handed it over +-- may have built it to pass, and against that the only thing to go on is that +-- each round with a base drawn at random catches three quarters of the +-- composites there are, whatever the number is. Thirty of them leave one +-- chance in 2^60. +probablyPrime :: Int -> Integer -> Bool +probablyPrime rounds !n + | n < 2 = False | any (\p -> p `divides` n) (filter (< n) firstPrimes) = False - | n >= 2 && n <= 2903 = True - | primalityTestFermat 50 (n `div` 2) n = - primalityTestMillerRabin 30 n - | otherwise = False + | n <= 2903 = True + | otherwise = primalityTestMillerRabin rounds n +-- | How many rounds a candidate drawn here needs. +-- +-- A number nobody chose is a different matter from one somebody did. The +-- composites that survive a round are rare, and the ones that survive several +-- are rarer than the bound above says: Damgard, Landrock and Pomerance +-- worked out how much rarer for a candidate drawn at random, and Table 4.4 of +-- the Handbook of Applied Cryptography puts their numbers in a table -- two +-- rounds at 1300 bits, three at 850, five at 550, and so on, for one chance +-- in 2^80. +-- +-- This is twice that, and never more than the thirty a number from anywhere +-- gets, which leaves the chance far under one in 2^100 at every size. It is +-- what makes generating a prime worth doing: the thirty rounds were half the +-- time it took. +roundsForDrawn :: Int -> Int +roundsForDrawn bits + | bits >= 1300 = 6 + | bits >= 850 = 8 + | bits >= 650 = 10 + | bits >= 550 = 12 + | bits >= 450 = 14 + | bits >= 400 = 16 + | bits >= 350 = 18 + | bits >= 300 = 20 + | bits >= 250 = 24 + | otherwise = 30 + -- | Generate a prime number of the required bitsize (i.e. in the range -- [2^(b-1)+2^(b-2), 2^b)). -- @@ -55,7 +101,7 @@ throwCryptoError $ CryptoFailed $ CryptoError_PrimeSizeInvalid else do sp <- generateParams bits (Just SetTwoHighest) True - let prime = findPrimeFrom sp + let prime = findPrimeFromDrawn (roundsForDrawn bits) sp if prime < 1 `shiftL` bits then return $ prime @@ -76,7 +122,12 @@ throwCryptoError $ CryptoFailed $ CryptoError_PrimeSizeInvalid else do sp <- generateParams bits (Just SetTwoHighest) True - let p = findPrimeFromWith (\i -> isProbablyPrime (2 * i + 1)) (sp `div` 2) + let rounds = roundsForDrawn bits + p = + findPrimeFromWithRounds + rounds + (\i -> probablyPrime rounds (2 * i + 1)) + (sp `div` 2) let val = 2 * p + 1 if val < 1 `shiftL` bits then @@ -85,16 +136,26 @@ -- | Find a prime from a starting point where the property hold. findPrimeFromWith :: (Integer -> Bool) -> Integer -> Integer -findPrimeFromWith prop !n - | even n = findPrimeFromWith prop (n + 1) +findPrimeFromWith = findPrimeFromWithRounds 30 + +-- | The same, with the number of rounds said outright: the walk starts where +-- the caller says, and only a caller that drew that starting point itself is +-- entitled to the smaller number. +findPrimeFromWithRounds :: Int -> (Integer -> Bool) -> Integer -> Integer +findPrimeFromWithRounds rounds prop !n + | even n = findPrimeFromWithRounds rounds prop (n + 1) | otherwise = - if not (isProbablyPrime n) - then findPrimeFromWith prop (n + 2) + if not (probablyPrime rounds n) + then findPrimeFromWithRounds rounds prop (n + 2) else if prop n then n - else findPrimeFromWith prop (n + 2) + else findPrimeFromWithRounds rounds prop (n + 2) +-- | Find a prime from a starting point that the caller drew itself. +findPrimeFromDrawn :: Int -> Integer -> Integer +findPrimeFromDrawn rounds = findPrimeFromWithRounds rounds (\_ -> True) + -- | Find a prime from a starting point with no specific property. findPrimeFrom :: Integer -> Integer findPrimeFrom n = @@ -104,11 +165,18 @@ -- | Miller Rabin algorithm return if the number is probably prime or composite. -- the tries parameter is the number of recursion, that determines the accuracy of the test. +-- +-- The witnesses are drawn from a generator derived from @n@ itself and from a +-- secret drawn once per process: testing the same number twice gives the same +-- answer, testing two numbers draws independent witnesses for each, and an +-- attacker choosing the number cannot tell which witnesses it will face. primalityTestMillerRabin :: Int -> Integer -> Bool primalityTestMillerRabin tries !n = case gmpTestPrimeMillerRabin tries n of GmpSupported b -> b - GmpUnsupported -> probabilistic run + -- the material is forced only once a witness is drawn, which the + -- guards in run reach only for an odd n above 3 + GmpUnsupported -> probabilisticFrom (i2osp n :: Bytes) run where run | n <= 3 = error "Miller-Rabin requires tested value to be > 3" diff --git a/Crypto/OTP.hs b/Crypto/OTP.hs --- a/Crypto/OTP.hs +++ b/Crypto/OTP.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE BangPatterns #-} {-# LANGUAGE ScopedTypeVariables #-} -- | One-time password implementation as defined by the @@ -29,6 +30,7 @@ OTP, OTPDigits (..), OTPTime, + minimumDigestSize, hotp, resynchronize, totp, @@ -41,14 +43,15 @@ where import Control.Monad (unless) -import Crypto.Hash (HashAlgorithm, SHA1 (..)) +import Crypto.Hash (HashAlgorithm, SHA1 (..), hashDigestSize) import Crypto.Internal.ByteArray (ByteArrayAccess, Bytes) import qualified Crypto.Internal.ByteArray as B import Crypto.MAC.HMAC -import Data.Bits (shiftL, (.&.), (.|.)) +import Data.Bits (complement, shiftL, shiftR, xor, (.&.), (.|.)) import Data.ByteArray.Mapping (fromW64BE) -import Data.List (elemIndex) +import Data.List (foldl') import Data.Word +import Prelude hiding (foldl') -- | A one-time password which is a sequence of 4 to 9 digits. type OTP = Word32 @@ -60,6 +63,21 @@ -- | An integral time value in seconds. type OTPTime = Word64 +-- | The smallest hash digest 'hotp' can be used with, in bytes. +-- +-- RFC 4226 section 5.3 defines dynamic truncation over the 20-byte HMAC-SHA-1 +-- output: the offset is the low four bits of the last byte, so it selects any +-- of the first 16 bytes, and four bytes are then read starting there. The +-- highest byte that can be reached is therefore byte 18, and a shorter digest +-- would make that read run off the end of the MAC. +minimumDigestSize :: Int +minimumDigestSize = 20 + +-- | Calculate an HOTP value as defined by RFC 4226. +-- +-- The hash must produce a digest of at least 'minimumDigestSize' bytes, which +-- is what the dynamic truncation step is defined over; 'error' is raised +-- otherwise. hotp :: forall hash key . (HashAlgorithm hash, ByteArrayAccess key) @@ -72,10 +90,19 @@ -- ^ Counter value synchronized between the client and server -> OTP -- ^ The HOTP value -hotp _ d k c = dt `mod` digitsPower d +hotp _ d k c + | macLen < minimumDigestSize = + error $ + "Crypto.OTP.hotp: hash digest is " + ++ show macLen + ++ " bytes, but at least " + ++ show minimumDigestSize + ++ " are required" + | otherwise = dt `mod` digitsPower d where mac = hmac k (fromW64BE c :: Bytes) :: HMAC hash - offset = fromIntegral (B.index mac (B.length mac - 1) .&. 0xf) + macLen = B.length mac + offset = fromIntegral (B.index mac (macLen - 1) .&. 0xf) dt = (fromIntegral (B.index mac offset .&. 0x7f) `shiftL` 24) .|. (fromIntegral (B.index mac (offset + 1) .&. 0xff) `shiftL` 16) @@ -84,6 +111,12 @@ -- | Attempt to resynchronize the server's counter value -- with the client, given a sequence of HOTP values. +-- +-- Every counter in the window is tried and every submitted value is compared, +-- whatever matches, so the time taken does not depend on where in the window +-- the client's counter was found, nor on how many of the submitted values were +-- right. The cost of a call is therefore one HMAC per counter in the window +-- plus one per extra value, every time. resynchronize :: (HashAlgorithm hash, ByteArrayAccess key) => hash @@ -101,17 +134,46 @@ -> Maybe Word64 -- ^ The new counter value, synchronized with the client's current counter -- or Nothing if the submitted OTP values didn't match anywhere within the window -resynchronize h d s k c (p1, extras) = do - offBy <- fmap fromIntegral (elemIndex p1 range) - checkExtraOtps (c + offBy + 1) extras +resynchronize h d s k c (p1, extras) + | accepted == 0 = Nothing + | otherwise = Just (afterFirst + fromIntegral (length extras)) where - checkExtraOtps ctr [] = Just ctr - checkExtraOtps ctr (p : ps) - | hotp h d k ctr /= p = Nothing - | otherwise = checkExtraOtps (ctr + 1) ps + -- Every counter in the window is tried and every extra value is compared, + -- whatever matches: the search does not stop at the first hit and the + -- check of the extra values does not stop at the first miss. Each skipped + -- counter used to save an HMAC, so the time taken revealed where in the + -- window the client's counter sat and how many of its extra values were + -- right -- the second of which a client that submits guesses cannot learn + -- from the answer itself, since that is 'Nothing' either way. + accepted = matched .&. extrasMatched range = map (hotp h d k) [c .. c + fromIntegral s] + -- the offset of the first match, accumulated without stopping there + (matched, offset) = foldl' pick (0, 0) (zip [0 ..] range) + pick (!m, !off) (i, candidate) = (m .|. hit, off .|. (hit .&. i)) + where + -- zero once something has matched, so only the first match counts + hit = eqMask candidate p1 .&. complement m + + -- the counter the first submitted value matched, plus one + afterFirst = c + offset + 1 + + -- the counters continue past the window, and wrap where the old + -- 'checkExtraOtps' wrapped + extrasMatched = + foldl' step (complement 0) (zip (iterate (+ 1) afterFirst) extras) + step acc (ctr, p) = acc .&. eqMask (hotp h d k ctr) p + +-- | All ones when the two values are equal, zero otherwise, without branching +-- on either of them. +eqMask :: OTP -> OTP -> Word64 +eqMask a b = negate (fromIntegral (1 - nonZero)) + where + v = a `xor` b + -- 0 when v is zero, 1 otherwise + nonZero = (v .|. negate v) `shiftR` 31 + digitsPower :: OTPDigits -> Word32 digitsPower OTP4 = 10000 digitsPower OTP5 = 100000 @@ -148,6 +210,13 @@ mkTOTPParams h t0 x d skew = do unless (x > 0) (Left "Time step must be greater than zero") unless (x <= 300) (Left "Time step cannot be greater than 300 seconds") + unless + (hashDigestSize h >= minimumDigestSize) + ( Left $ + "Hash digest must be at least " + ++ show minimumDigestSize + ++ " bytes" + ) return (TP h t0 x d skew) -- | Calculate a totp value for the given time. @@ -171,12 +240,18 @@ -> OTPTime -> OTP -> Bool -totpVerify (TP h t0 x d skew) k now otp = otp `elem` map (hotp h d k) (range window []) +totpVerify (TP h t0 x d skew) k now otp = matched /= 0 where t = timeToCounter now t0 x window = fromIntegral (fromEnum skew) range 0 acc = t : acc range n acc = range (n - 1) ((t - n) : (t + n) : acc) + + -- every candidate is compared, and none of the comparisons stops early, so + -- neither which step matched nor how far a mismatch got is visible in how + -- long this takes + matched = foldl' step 0 (map (hotp h d k) (range window [])) + step acc candidate = acc .|. eqMask candidate otp timeToCounter :: Word64 -> Word64 -> Word16 -> Word64 timeToCounter now t0 x = (now - t0) `div` fromIntegral x diff --git a/Crypto/PubKey/Curve25519.hs b/Crypto/PubKey/Curve25519.hs --- a/Crypto/PubKey/Curve25519.hs +++ b/Crypto/PubKey/Curve25519.hs @@ -26,6 +26,7 @@ generateSecretKey, ) where +import Crypto.Debug (DebugShow (..), debugShowBytes) import Data.Bits import Data.Word import Foreign.Ptr @@ -47,6 +48,9 @@ -- | A Curve25519 Secret key newtype SecretKey = SecretKey ScrubbedBytes deriving (Show, Eq, ByteArrayAccess, NFData) + +instance DebugShow SecretKey where + debugShow = debugShowBytes "SecretKey" -- | A Curve25519 public key newtype PublicKey = PublicKey Bytes diff --git a/Crypto/PubKey/Curve448.hs b/Crypto/PubKey/Curve448.hs --- a/Crypto/PubKey/Curve448.hs +++ b/Crypto/PubKey/Curve448.hs @@ -28,6 +28,7 @@ generateSecretKey, ) where +import Crypto.Debug (DebugShow (..), debugShowBytes) import Data.Word import Foreign.Ptr @@ -46,6 +47,9 @@ -- | A Curve448 Secret key newtype SecretKey = SecretKey ScrubbedBytes deriving (Show, Eq, ByteArrayAccess, NFData) + +instance DebugShow SecretKey where + debugShow = debugShowBytes "SecretKey" -- | A Curve448 public key newtype PublicKey = PublicKey Bytes diff --git a/Crypto/PubKey/DH.hs b/Crypto/PubKey/DH.hs --- a/Crypto/PubKey/DH.hs +++ b/Crypto/PubKey/DH.hs @@ -17,9 +17,17 @@ calculatePublic, generatePublic, getShared, + tryGetShared, ) where +import Crypto.Debug (DebugShow (..)) +import Crypto.Error ( + CryptoError (..), + CryptoFailable (..), + throwCryptoError, + ) import Crypto.Internal.Imports +import Crypto.Number.Basic (numBytes) import Crypto.Number.Generate (generateMax) import Crypto.Number.ModArithmetic (expSafe) import Crypto.Number.Prime (generateSafePrime) @@ -45,8 +53,16 @@ -- | Represent Diffie Hellman private number X. newtype PrivateNumber = PrivateNumber Integer - deriving (Show, Read, Eq, Enum, Real, Num, Ord, NFData) + deriving (Read, Eq, Enum, Real, Num, Ord, NFData) +-- | The number is not shown. Use 'Crypto.Debug.debugShow' to see it. +instance Show PrivateNumber where + show _ = "PrivateNumber " + +instance DebugShow PrivateNumber where + debugShow (PrivateNumber n) = + showString "PrivateNumber " . showsPrec 11 n $ "" + -- | Represent Diffie Hellman shared secret. newtype SharedKey = SharedKey ScrubbedBytes deriving (Show, Eq, ByteArrayAccess, NFData) @@ -83,5 +99,29 @@ -- commented until 0.3 {-# DEPRECATED generatePublic "use calculatePublic" #-} -- | generate a shared key using our private number and the other party public number +-- +-- This raises the 'CryptoError' that 'tryGetShared' reports. Use 'tryGetShared' +-- where the failure has to be handled. getShared :: Params -> PrivateNumber -> PublicNumber -> SharedKey -getShared (Params p _ bits) (PrivateNumber x) (PublicNumber y) = SharedKey $ i2ospOf_ ((bits + 7) `div` 8) $ expSafe y x p +getShared params x y = throwCryptoError $ tryGetShared params x y + +-- | generate a shared key using our private number and the other party public +-- number, reporting a rejected public number instead of raising. +-- +-- The public number comes from the other party, so it is checked to satisfy +-- @1 < y < p-1@ as RFC 7919 section 5.1 requires. The excluded values +-- generate the subgroup @{1}@ or @{1, p-1}@, so the shared secret they produce +-- is one of a handful of constants and carries none of our private number's +-- secrecy. A value outside that range is reported as +-- 'CryptoError_ParameterInvalid'. +-- +-- Note this is the only check made here: it does not establish that @y@ lies +-- in the subgroup generated by @g@, which needs the subgroup order that +-- t'Params' does not carry. +tryGetShared + :: Params -> PrivateNumber -> PublicNumber -> CryptoFailable SharedKey +tryGetShared (Params p _ _) (PrivateNumber x) (PublicNumber y) + | y <= 1 || y >= p - 1 = CryptoFailed CryptoError_ParameterInvalid + -- the size of p, not params_bits: only p and g travel on the wire, so a + -- caller-supplied bit size can disagree with p + | otherwise = CryptoPassed $ SharedKey $ i2ospOf_ (numBytes p) $ expSafe y x p diff --git a/Crypto/PubKey/DSA.hs b/Crypto/PubKey/DSA.hs --- a/Crypto/PubKey/DSA.hs +++ b/Crypto/PubKey/DSA.hs @@ -8,6 +8,24 @@ -- Portability : Good -- -- An implementation of the Digital Signature Algorithm (DSA) +-- +-- == What is kept from the clock, and what is not +-- +-- Signing keeps the private number and the ephemeral @k@ out of the two +-- places whose duration would otherwise follow them: the exponentiation is +-- 'Crypto.Number.ModArithmetic.expSafe', which walks the exponent a fixed +-- four bits at a time, and @k@ is inverted by Fermat's little theorem rather +-- than by the extended Euclidean algorithm, whose number of steps follows the +-- bits it is given. +-- +-- What is left is the arithmetic around them. @x * r@, the addition and the +-- reduction modulo @q@ are 'Integer' operations, and an 'Integer' costs what +-- its size says: a private number that happens to be short is multiplied in +-- fewer words than a full-length one. The same holds in +-- "Crypto.PubKey.ElGamal" and "Crypto.PubKey.Rabin.Basic". Removing it means +-- leaving 'Integer' for a fixed-width representation, which is what +-- "Crypto.PubKey.RSA" does for its exponentiation and the curve modules do +-- throughout; there is nothing a caller can do about it from here. module Crypto.PubKey.DSA ( Params (..), Signature (..), @@ -33,14 +51,14 @@ toPrivateKey, ) where +import Crypto.Debug (DebugShow (..)) import Data.Data -import Data.Maybe import Crypto.Hash import Crypto.Internal.ByteArray (ByteArrayAccess) import Crypto.Internal.Imports import Crypto.Number.Generate -import Crypto.Number.ModArithmetic (expFast, expSafe, inverse) +import Crypto.Number.ModArithmetic (expFast, expSafe, inverse, inverseSafe) import Crypto.PubKey.Internal (dsaTruncHash) import Crypto.Random.Types @@ -98,15 +116,52 @@ , private_x :: PrivateNumber -- ^ DSA private X } - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +-- | The parameters are shown; @private_x@ is not. Use +-- 'Crypto.Debug.debugShow' to see it. +instance Show PrivateKey where + showsPrec d k = + showParen (d > 10) $ + showString "PrivateKey {private_params = " + . shows (private_params k) + . showString ", private_x = }" + +instance DebugShow PrivateKey where + debugShow k = + showString "PrivateKey {private_params = " + . shows (private_params k) + . showString ", private_x = " + . shows (private_x k) + . showChar '}' + $ "" + instance NFData PrivateKey where rnf (PrivateKey params x) = x `seq` params `seq` () -- | Represent a DSA key pair data KeyPair = KeyPair Params PublicNumber PrivateNumber - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +instance Show KeyPair where + showsPrec d (KeyPair params y _) = + showParen (d > 10) $ + showString "KeyPair " + . showsPrec 11 params + . showChar ' ' + . showsPrec 11 y + . showString " " + +instance DebugShow KeyPair where + debugShow (KeyPair params y x) = + showString "KeyPair " + . showsPrec 11 params + . showChar ' ' + . showsPrec 11 y + . showChar ' ' + . showsPrec 11 x + $ "" + instance NFData KeyPair where rnf (KeyPair params y x) = x `seq` y `seq` params `seq` () @@ -139,18 +194,21 @@ -> msg -- ^ message to sign -> Maybe Signature -signWith k pk hashAlg msg - | r == 0 || s == 0 = Nothing - | otherwise = Just $ Signature r s +signWith k pk hashAlg msg = do + -- k comes from the caller and is only invertible when it is coprime with + -- q, which the caller cannot check without knowing q is prime. It is also + -- a secret worth as much as the private key, so it is inverted without + -- the extended Euclidean algorithm, whose steps follow the bits of what + -- it is given + kInv <- inverseSafe k q + let hm = dsaTruncHash hashAlg msg q + r = expSafe g k p `mod` q + s = (kInv * (hm + x * r)) `mod` q + if r == 0 || s == 0 then Nothing else Just $ Signature r s where -- parameters (Params p g q) = private_params pk x = private_x pk - -- compute r,s - kInv = fromJust $ inverse k q - hm = dsaTruncHash hashAlg msg q - r = expSafe g k p `mod` q - s = (kInv * (hm + x * r)) `mod` q -- | sign message using the private key. sign @@ -171,12 +229,15 @@ verify hashAlg pk (Signature r s) m -- Reject the signature if either 0 < r < q or 0 < s < q is not satisfied. | r <= 0 || r >= q || s <= 0 || s >= q = False - | otherwise = v == r + -- s is invertible for every 0 < s < q when q is prime, but the parameters + -- arrive with the public key and a composite q admits an s that is not + | otherwise = maybe False (r ==) v where (Params p g q) = public_params pk y = public_y pk hm = dsaTruncHash hashAlg m q - w = fromJust $ inverse s q - u1 = (hm * w) `mod` q - u2 = (r * w) `mod` q - v = ((expFast g u1 p) * (expFast y u2 p)) `mod` p `mod` q + v = do + w <- inverse s q + let u1 = (hm * w) `mod` q + u2 = (r * w) `mod` q + return $ ((expFast g u1 p) * (expFast y u2 p)) `mod` p `mod` q diff --git a/Crypto/PubKey/ECC/DH.hs b/Crypto/PubKey/ECC/DH.hs --- a/Crypto/PubKey/ECC/DH.hs +++ b/Crypto/PubKey/ECC/DH.hs @@ -14,12 +14,18 @@ generatePrivate, calculatePublic, getShared, + tryGetShared, ) where +import Crypto.Error ( + CryptoError (..), + CryptoFailable (..), + throwCryptoError, + ) import Crypto.Number.Generate (generateMax) import Crypto.Number.Serialize (i2ospOf_) import Crypto.PubKey.DH (SharedKey (..)) -import Crypto.PubKey.ECC.Prim (pointMul) +import Crypto.PubKey.ECC.Prim (isPointInSubgroup, isPointValid, pointMul) import Crypto.PubKey.ECC.Types ( Curve, Point (..), @@ -47,10 +53,37 @@ -- | Generating a shared key using our private number and -- the other party public point. +-- +-- This raises the 'Crypto.Error.CryptoError' that 'tryGetShared' reports. Use +-- 'tryGetShared' where the failure has to be handled. getShared :: Curve -> PrivateNumber -> PublicPoint -> SharedKey -getShared curve db qa = SharedKey $ i2ospOf_ ((nbBits + 7) `div` 8) x +getShared curve db qa = throwCryptoError $ tryGetShared curve db qa + +-- | Generating a shared key using our private number and the other party +-- public point, reporting a rejected point instead of raising. +-- +-- The public point comes from the other party, so it is checked before it is +-- multiplied. A point that does not satisfy the curve equation is reported as +-- 'CryptoError_PointCoordinatesInvalid'. +-- +-- Satisfying the equation is not by itself membership of the subgroup the base +-- point generates; the two coincide only when the cofactor is 1. On a curve +-- whose cofactor is above 1 the other party can offer a point of small order, +-- and the value that comes back then depends on our private number only +-- through its residue modulo that order, which hands them those bits. So the +-- point is also required to be in the subgroup, by 'isPointInSubgroup', and is +-- reported as 'CryptoError_PointSubgroupInvalid' when it is not. That check +-- costs one further scalar multiplication, and is skipped where the cofactor +-- is 1 and it cannot fail. +-- +-- An exchange that yields the point at infinity, and so has no x coordinate to +-- derive the key from, is reported as 'CryptoError_ScalarMultiplicationInvalid'. +tryGetShared :: Curve -> PrivateNumber -> PublicPoint -> CryptoFailable SharedKey +tryGetShared curve db qa + | not (isPointValid curve qa) = CryptoFailed CryptoError_PointCoordinatesInvalid + | not (isPointInSubgroup curve qa) = CryptoFailed CryptoError_PointSubgroupInvalid + | otherwise = case pointMul curve db qa of + Point x _ -> CryptoPassed $ SharedKey $ i2ospOf_ ((nbBits + 7) `div` 8) x + PointO -> CryptoFailed CryptoError_ScalarMultiplicationInvalid where - x = case pointMul curve db qa of - Point x' _ -> x' - _ -> error "getShared" nbBits = curveSizeBits curve diff --git a/Crypto/PubKey/ECC/ECDSA.hs b/Crypto/PubKey/ECC/ECDSA.hs --- a/Crypto/PubKey/ECC/ECDSA.hs +++ b/Crypto/PubKey/ECC/ECDSA.hs @@ -1,7 +1,11 @@ {-# LANGUAGE DeriveDataTypeable #-} --- | /WARNING:/ Signature operations may leak the private key. Signature verification --- should be safe. +-- | /WARNING:/ Signature operations may leak the private key. The nonce is +-- inverted without a side channel on every curve, and on P-256 the scalar +-- multiplication is the constant-time C implementation, but what surrounds +-- them is 'Integer' arithmetic, whose cost follows the values it is given, and +-- on every other curve the multiplication follows the nonce as well. +-- Signature verification takes only public values and should be safe. module Crypto.PubKey.ECC.ECDSA ( Signature (..), ExtendedSignature (..), @@ -25,6 +29,7 @@ deterministicNonce, ) where +import Crypto.Debug (DebugShow (..)) import Control.Monad import Data.Bits import Data.ByteArray (ByteArrayAccess, ScrubbedBytes) @@ -66,8 +71,26 @@ { private_curve :: Curve , private_d :: PrivateNumber } - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +-- | The curve is shown; @private_d@ is not. Use +-- 'Crypto.Debug.debugShow' to see it. +instance Show PrivateKey where + showsPrec d k = + showParen (d > 10) $ + showString "PrivateKey {private_curve = " + . shows (private_curve k) + . showString ", private_d = }" + +instance DebugShow PrivateKey where + debugShow k = + showString "PrivateKey {private_curve = " + . shows (private_curve k) + . showString ", private_d = " + . shows (private_d k) + . showChar '}' + $ "" + -- | ECDSA Public Key. data PublicKey = PublicKey { public_curve :: Curve @@ -77,8 +100,27 @@ -- | ECDSA Key Pair. data KeyPair = KeyPair Curve PublicPoint PrivateNumber - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +instance Show KeyPair where + showsPrec d (KeyPair c q _) = + showParen (d > 10) $ + showString "KeyPair " + . showsPrec 11 c + . showChar ' ' + . showsPrec 11 q + . showString " " + +instance DebugShow KeyPair where + debugShow (KeyPair c q x) = + showString "KeyPair " + . showsPrec 11 c + . showChar ' ' + . showsPrec 11 q + . showChar ' ' + . showsPrec 11 x + $ "" + -- | Public key of a ECDSA Key pair. toPublicKey :: KeyPair -> PublicKey toPublicKey (KeyPair curve pub _) = PublicKey curve pub @@ -103,8 +145,10 @@ let z = dsaTruncHashDigest digest n CurveCommon _ _ g n _ = common_curve curve (i, r, p) <- pointDecompose curve $ pointMul curve k g - kInv <- inverse k n - let s = kInv * (z + r * d) `mod` n + kInv <- scalarInverse curve k + -- kInv and d are secret, so the arithmetic that mixes them goes through + -- the curve's own, which on P-256 is the C implementation's + let s = scalarMul curve kInv (scalarAdd curve z (scalarMul curve r d)) when (r == 0 || s == 0) Nothing return $ if s <= n `unsafeShiftR` 1 diff --git a/Crypto/PubKey/ECC/P256.hs b/Crypto/PubKey/ECC/P256.hs --- a/Crypto/PubKey/ECC/P256.hs +++ b/Crypto/PubKey/ECC/P256.hs @@ -36,6 +36,7 @@ scalarZero, scalarN, scalarIsZero, + scalarReduce, scalarAdd, scalarSub, scalarMul, @@ -150,17 +151,17 @@ withScalar n1 $ \pn1 -> withScalar n2 $ \pn2 -> withPoint p $ \px py -> ccrypton_p256_points_mul_vartime pn1 pn2 px py dx dy --- | Check if a 'Point' is valid +-- | Check if a t'Point' is valid pointIsValid :: Point -> Bool pointIsValid p = unsafeDoIO $ withPoint p $ \px py -> do r <- ccrypton_p256_is_valid_point px py return (r /= 0) --- | Check if a 'Point' is the point at infinity +-- | Check if a t'Point' is the point at infinity pointIsAtInfinity :: Point -> Bool pointIsAtInfinity (Point b) = constAllZero b --- | Return the x coordinate as a 'Scalar' if the point is not at infinity +-- | Return the x coordinate as a t'Scalar' if the point is not at infinity pointX :: Point -> Maybe Scalar pointX p | pointIsAtInfinity p = Nothing @@ -250,6 +251,17 @@ scalarIsZero s = unsafeDoIO $ withScalar s $ \d -> do result <- ccrypton_p256_is_zero d return $ result /= 0 + +-- | Bring a scalar below the order of the curve +-- +-- 'scalarFromInteger' and 'scalarFromBinary' take any 256 bits, so a scalar +-- can arrive above the order; the arithmetic below wants it brought down +-- first. Twice the order is more than 256 bits hold, so this is a single +-- subtraction, taken or not through a mask rather than a branch. +scalarReduce :: Scalar -> Scalar +scalarReduce a = + withNewScalarFreeze $ \d -> withScalar a $ \pa -> + ccrypton_p256_mod ccrypton_SECP256r1_n pa d -- | Perform addition between two scalars -- diff --git a/Crypto/PubKey/ECC/Prim.hs b/Crypto/PubKey/ECC/Prim.hs --- a/Crypto/PubKey/ECC/Prim.hs +++ b/Crypto/PubKey/ECC/Prim.hs @@ -1,8 +1,15 @@ +{-# LANGUAGE BangPatterns #-} + -- | Elliptic Curve Arithmetic. -- --- /WARNING:/ These functions are vulnerable to timing attacks. +-- /WARNING:/ These functions are vulnerable to timing attacks, except on +-- P-256, whose multiplications go to the C implementation in +-- "Crypto.PubKey.ECC.P256". module Crypto.PubKey.ECC.Prim ( scalarGenerate, + scalarInverse, + scalarAdd, + scalarMul, pointAdd, pointNegate, pointDouble, @@ -13,21 +20,137 @@ pointCompose, isPointAtInfinity, isPointValid, + isPointInSubgroup, ) where +import Crypto.Error (maybeCryptoError) +import Crypto.Internal.ECC (CurveField (..), MulResult (..), curveMul) +import Crypto.Number.Basic (numBits) import Crypto.Number.F2m import Crypto.Number.Generate (generateBetween) import Crypto.Number.ModArithmetic +import qualified Crypto.PubKey.ECC.P256 as P256 import Crypto.PubKey.ECC.Types import Crypto.Random +import Data.Bits (shiftL, shiftR, testBit, (.&.)) import Data.Maybe +-- | P-256, the one curve here that has a C implementation: 'SEC_p256r1', also +-- known as NIST P-256 and prime256v1. +-- +-- A 'Curve' carries its parameters rather than a name, so this compares the +-- parameters. They are public, so the comparison tells an attacker nothing. +p256Curve :: Curve +p256Curve = getCurveByName SEC_p256r1 +{-# NOINLINE p256Curve #-} + +p256Order :: Integer +p256Order = ecc_n (common_curve p256Curve) + +p256Base :: Point +p256Base = ecc_g (common_curve p256Curve) + +-- | A point the C implementation will take: in range, on the curve, and not +-- the point at infinity, which it does not represent. Anything else is left +-- to the generic code, which answers for points off the curve too. +toP256 :: Point -> Maybe P256.Point +toP256 PointO = Nothing +toP256 (Point x y) + | x < 0 || y < 0 || x >= limit || y >= limit = Nothing + | P256.pointIsValid p = Just p + | otherwise = Nothing + where + limit = 1 `shiftL` 256 + p = P256.pointFromIntegers (x, y) + +fromP256 :: P256.Point -> Point +fromP256 p + | P256.pointIsAtInfinity p = PointO + | otherwise = uncurry Point (P256.pointToIntegers p) + +-- | Any 256-bit number as a scalar. +-- +-- The arithmetic below takes them as they come: a 256-bit value is barely +-- over the order, and both the multiplication and the addition bring their +-- answer back under it. 'Nothing' is for what does not fit in 256 bits, +-- which no scalar anybody signs with does. +p256Scalar :: Integer -> Maybe P256.Scalar +p256Scalar n + | n < 0 || n >= 1 `shiftL` 256 = Nothing + | otherwise = maybeCryptoError (P256.scalarFromInteger n) + +-- | The scalar reduced into the range the C implementation takes. +-- +-- Every point it accepts has the curve's order, so reducing changes no +-- answer; 'Nothing' means the multiple is the point at infinity, which is the +-- generic code's business. +-- The reduction is a single masked subtraction, so a secret scalar does not +-- steer it, which taking the remainder would: dividing takes a number of +-- steps that follows the number being divided. Anything wider than 256 bits +-- has to go through a division first, but a scalar that wide is not one +-- anybody signs with. +toP256Scalar :: Integer -> Maybe P256.Scalar +toP256Scalar n = case P256.scalarReduce <$> p256Scalar n of + Nothing -> toP256Scalar (n `mod` p256Order) -- wider than 256 bits, or below zero + Just s + | P256.scalarIsZero s -> Nothing + | otherwise -> Just s + +-- | @n1 * p1 + n2 * p2@ through the C implementation, when one of the points +-- is the base point. That is the shape signature verification uses. +p256AddTwoMuls :: Integer -> Point -> Integer -> Point -> Maybe Point +p256AddTwoMuls n1 p1 n2 p2 + | p1 == p256Base = withBase n1 n2 p2 + | p2 == p256Base = withBase n2 n1 p1 + | otherwise = Nothing + where + withBase a b q = + fromP256 + <$> (P256.pointsMulVarTime <$> toP256Scalar a <*> toP256Scalar b <*> toP256 q) + -- | Generate a valid scalar for a specific Curve scalarGenerate :: MonadRandom randomly => Curve -> randomly PrivateNumber scalarGenerate curve = generateBetween 1 (n - 1) where n = ecc_n $ common_curve curve +-- | The inverse of a scalar modulo the order of the curve, without letting +-- the scalar steer how long the work takes. This is what signing needs for +-- its nonce, which is as worth hiding as the private key itself: a handful of +-- signatures whose nonces are partly known give the key away. +-- +-- On P-256 the C implementation does it; elsewhere it is 'inverseSafe'. +-- 'Nothing' means the scalar has no inverse, which for the curves in use here +-- means it was a multiple of the order. +scalarInverse :: Curve -> Integer -> Maybe Integer +scalarInverse c k + | c == p256Curve + , Just s <- toP256Scalar k = + Just (P256.scalarToInteger (P256.scalarInvSafe s)) + | otherwise = inverseSafe k (ecc_n $ common_curve c) + +-- | Addition modulo the order of the curve. +-- +-- On P-256 this is the C implementation's arithmetic, which works in a fixed +-- width and so does not let the values steer it; elsewhere it is 'Integer' +-- arithmetic, whose cost follows the values. +scalarAdd :: Curve -> Integer -> Integer -> Integer +scalarAdd c a b + | c == p256Curve + , Just x <- p256Scalar a + , Just y <- p256Scalar b = + P256.scalarToInteger (P256.scalarAdd x y) + | otherwise = (a + b) `mod` ecc_n (common_curve c) + +-- | Multiplication modulo the order of the curve, as 'scalarAdd'. +scalarMul :: Curve -> Integer -> Integer -> Integer +scalarMul c a b + | c == p256Curve + , Just x <- p256Scalar a + , Just y <- p256Scalar b = + P256.scalarToInteger (P256.scalarMul x y) + | otherwise = (a * b) `mod` ecc_n (common_curve c) + -- TODO: Extract helper function for `fromMaybe PointO...` -- | Elliptic Curve point negation: @@ -98,47 +221,258 @@ -- | Elliptic curve point multiplication using the base -- --- /WARNING:/ Vulnerable to timing attacks. +-- On P-256 this reaches the C implementation, which multiplies the base point +-- through a table of its own. +-- +-- /WARNING:/ On every other curve, vulnerable to timing attacks. pointBaseMul :: Curve -> Integer -> Point pointBaseMul c n = pointMul c n (ecc_g $ common_curve c) --- | Elliptic curve point multiplication (double and add algorithm). +-- | Elliptic curve point multiplication. -- --- /WARNING:/ Vulnerable to timing attacks. +-- Over a prime field this goes to C, four bits of scalar at a time, with the +-- multiple to add taken from a table read by touching every entry of it. +-- Over a binary field it also goes to C, as Montgomery's ladder: it carries +-- the x coordinates of two consecutive multiples -- their difference being +-- the point is what lets it carry no more than that -- and spends one +-- addition and one doubling on every bit whichever way the bit goes, with the +-- two exchanged by a mask rather than chosen by a branch. Either way the work +-- follows the width of the curve's order and not the scalar. +-- +-- What falls back on the 'Integer' arithmetic below is a point that is not on +-- the curve, the one point of a binary curve that has no x, and a prime the C +-- will not take. +-- +-- Multiplying the base point of a curve over a prime field -- which is what +-- signing and making a key do, and nothing else does -- goes through a table +-- of its multiples, built when that curve is first asked for one and kept +-- afterwards. The build is a few milliseconds and the table a few hundred +-- kilobytes, and a multiplication that uses it takes about a third of what +-- one without it takes. +-- +-- On P-256 the multiplication goes to the C implementation in +-- "Crypto.PubKey.ECC.P256", which has a table for the base point. +-- +-- /WARNING:/ What is left of the 'Integer' arithmetic below -- a point off +-- the curve, the one point of a binary curve with no x, a prime or a +-- polynomial the C will not take -- has uniform operation counts at best, and +-- uniform operation counts are not constant time: those operations cost what +-- the values they are given cost. pointMul :: Curve -> Integer -> Point -> Point pointMul _ _ PointO = PointO pointMul c n p + -- the base point has a table of its own in the C, which is what makes key + -- generation and signing quicker than multiplying any other point + | c == p256Curve + , p == p256Base = + maybe PointO (fromP256 . P256.toPoint) (toP256Scalar n) + | c == p256Curve + , Just q <- toP256 p = + maybe PointO (\s -> fromP256 (P256.pointMul s q)) (toP256Scalar n) | n < 0 = pointMul c (-n) (pointNegate c p) | n == 0 = PointO - | n == 1 = p - | odd n = pointAdd c p (pointMul c (n - 1) p) - | otherwise = pointMul c (n `div` 2) (pointDouble c p) + | otherwise = + case c of + CurveFP (CurvePrime pr cc) -> primeMul pr cc + CurveF2m (CurveBinary fx cc) -> binaryMul fx cc + where + -- The C answers for a point on the curve; anything else keeps the + -- answers it has always had from the code below. Multiplying the base + -- point, which is what signing and making a key do, goes through the + -- table kept for it. + primeMul pr cc = case p of + Point px py + | isPointValid c p -> + answer slow $ + curveMul + (Prime pr (ecc_a cc) (ecc_b cc)) + (ecc_n cc) + n + px + py + (p == ecc_g cc) + _ -> slow + where + slow = + jacobianMul + pr + (ecc_a cc) + (max (integerBits n) (integerBits (ecc_n cc))) + n + p --- | Elliptic curve double-scalar multiplication (uses Shamir's trick). + -- The ladder answers for a point on the curve that has an x; the one + -- point with no x, and anything off the curve, keep what they had. + binaryMul fx cc = case p of + Point px py + | isPointValid c p -> + answer (affineMul n p) $ + curveMul (Binary fx (ecc_b cc)) (ecc_n cc) n px py False + _ -> affineMul n p + + -- what the C could not take goes back to the code that was here before + answer fallback r = case r of + MulPoint x y -> Point x y + MulInfinity -> PointO + MulUnsupported -> fallback + + affineMul k q + | k == 0 = PointO + | k == 1 = q + | odd k = pointAdd c q (affineMul (k - 1) q) + | otherwise = affineMul (k `div` 2) (pointDouble c q) + +-- | Number of bits needed to write n, for n > 0. +integerBits :: Integer -> Int +integerBits = go 0 + where + go acc 0 = acc + go acc k = go (acc + 1) (k `div` 2) + +-- | A point in Jacobian coordinates: @(X, Y, Z)@ stands for the affine +-- @(X\/Z^2, Y\/Z^3)@, and @JPointO@ for the point at infinity. +data JPoint = JPointO | JPoint !Integer !Integer !Integer + +-- | The field a prime curve works in, and how to reduce into it. -- +-- Most curve primes are @2^k - c@ with @c@ far smaller than the prime. +-- Reducing is then a shift, a multiplication by @c@ and an addition, where +-- dividing a number twice the width costs about four times as much: 227ns +-- against 183 for a P-384 multiplication, and 226 against 89 for P-521, whose +-- @c@ is one. +-- | The prime, the width to fold at, and what to fold back in. A @c@ of zero +-- says to divide instead, either because the prime has no such shape or +-- because it is too small for folding to pay: @c@ has to be under half the +-- width, or folding would not shrink the number, and below 256 bits the +-- handful of 'Integer' operations folding takes costs more than the division +-- it saves -- measured on P-192, where folding is 14% slower. +data Field = Field !Integer !Int !Integer + +mkField :: Integer -> Field +mkField p + | p > 0 && c > 0 && 2 * numBits c <= k && k >= 256 = Field p k c + | otherwise = Field p 0 0 + where + k = numBits p + c = (1 `shiftL` k) - p + +fieldPrime :: Field -> Integer +fieldPrime (Field p _ _) = p + +fieldReduce :: Field -> Integer -> Integer +fieldReduce (Field p k c) x + | c == 0 || x < 0 = x `mod` p + | otherwise = trim (fold x) + where + mask = (1 `shiftL` k) - 1 + fold v + | v > mask = fold ((v `shiftR` k) * c + (v .&. mask)) + | otherwise = v + trim v + | v >= p = trim (v - p) + | otherwise = v +{-# INLINE fieldReduce #-} + +-- | A point in affine coordinates: the second operand of every addition a +-- scalar multiplication makes, where knowing that z is one saves four +-- multiplications of the sixteen. +data Affine = AffineO | Affine !Integer !Integer + +jacobianMul :: Integer -> Integer -> Int -> Integer -> Point -> Point +jacobianMul _ _ _ _ PointO = PointO +jacobianMul pr a bits n (Point px py) = fromJacobian f (go (bits - 1) JPointO) + where + f = mkField pr + base = Affine px py + + -- The bangs are what make the addition happen at every bit. Without + -- them the one that is not taken stays a thunk and is never worked out, + -- so the multiplication costs a step for every bit that is set rather + -- than for every bit there is, and a single measurement tells an attacker + -- how many bits of the scalar are set. + go i acc + | i < 0 = acc + | otherwise = + let !d = jDouble f a acc + !s = jAddAffine f a d base + in go (i - 1) (if testBit n i then s else d) + +jDouble :: Field -> Integer -> JPoint -> JPoint +jDouble _ _ JPointO = JPointO +jDouble f a (JPoint x y z) + | y == 0 = JPointO + | otherwise = JPoint x3 y3 z3 + where + red = fieldReduce f + yy = red (y * y) + delta = red (4 * x * yy) + zz = red (z * z) + m = red (3 * x * x + a * zz * zz) + x3 = red (m * m - 2 * delta) + y3 = red (m * (delta - x3) - 8 * yy * yy) + z3 = red (2 * y * z) + +-- | Add a point whose z is one, which is what a scalar multiplication always +-- adds: u1 is x1, s1 is y1, and z3 is one multiplication rather than two. +jAddAffine :: Field -> Integer -> JPoint -> Affine -> JPoint +jAddAffine _ _ p AffineO = p +jAddAffine _ _ JPointO (Affine x2 y2) = JPoint x2 y2 1 +jAddAffine f a p@(JPoint x1 y1 z1) (Affine x2 y2) + | h /= 0 = JPoint x3 y3 z3 + | r /= 0 = JPointO + | otherwise = jDouble f a p + where + red = fieldReduce f + z1s = red (z1 * z1) + u2 = red (x2 * z1s) + s2 = red (y2 * z1s * z1) + h = red (u2 - x1) + r = red (s2 - y1) + h2 = red (h * h) + h3 = red (h2 * h) + x3 = red (r * r - h3 - 2 * x1 * h2) + y3 = red (r * (x1 * h2 - x3) - y1 * h3) + z3 = red (h * z1) + +fromJacobian :: Field -> JPoint -> Point +fromJacobian _ JPointO = PointO +fromJacobian f (JPoint x y z) = + case inverse z (fieldPrime f) of + Nothing -> PointO + Just zi -> + let red = fieldReduce f + zi2 = red (zi * zi) + in Point (red (x * zi2)) (red (y * zi2 * zi)) + +-- | Elliptic curve double-scalar multiplication. +-- -- > pointAddTwoMuls c n1 p1 n2 p2 == pointAdd c (pointMul c n1 p1) -- > (pointMul c n2 p2) -- +-- which, apart from P-256, is how it is done: the two multiplications +-- separately, and then one addition. P-256 has a double multiplication of +-- its own in C and takes it. +-- +-- This used to be Shamir's trick, one pass over the bits of both scalars at +-- once, which shares the doublings between them and is the right thing to do +-- when the two multiplications would cost the same. They no longer do. +-- 'pointMul' goes to C, and over a prime field it multiplies the base point +-- through a table of its multiples, which is a third of the price of an +-- ordinary multiplication -- and the base point is one of the two here, since +-- signature verification is what asks for this. Sharing the doublings with a +-- pass in 'Integer' arithmetic gives that up and more: on P-384 it costs +-- twice what two multiplications in C cost, and on the curves over a binary +-- field, whose addition needs an inversion where the C has a ladder that +-- needs none, it costs two hundred times as much. +-- +-- Both scalars are public wherever this is called from, so nothing here is +-- meant to hide them. +-- -- /WARNING:/ Vulnerable to timing attacks. pointAddTwoMuls :: Curve -> Integer -> Point -> Integer -> Point -> Point -pointAddTwoMuls _ _ PointO _ PointO = PointO -pointAddTwoMuls c _ PointO n2 p2 = pointMul c n2 p2 -pointAddTwoMuls c n1 p1 _ PointO = pointMul c n1 p1 pointAddTwoMuls c n1 p1 n2 p2 - | n1 < 0 = pointAddTwoMuls c (-n1) (pointNegate c p1) n2 p2 - | n2 < 0 = pointAddTwoMuls c n1 p1 (-n2) (pointNegate c p2) - | otherwise = go (n1, n2) - where - p0 = pointAdd c p1 p2 - - go (0, 0) = PointO - go (k1, k2) = - let q = pointDouble c $ go (k1 `div` 2, k2 `div` 2) - in case (odd k1, odd k2) of - (True, True) -> pointAdd c p0 q - (True, False) -> pointAdd c p1 q - (False, True) -> pointAdd c p2 q - (False, False) -> q + | c == p256Curve, Just r <- p256AddTwoMuls n1 p1 n2 p2 = r + | otherwise = pointAdd c (pointMul c n1 p1) (pointMul c n2 p2) -- | Decompose a point into index, residue, and parity. -- @@ -184,6 +518,33 @@ -- * x is not out of range -- * y is not out of range -- * the equation @y^2 = x^3 + a*x + b (mod p)@ holds +-- +-- over a prime curve, and the corresponding checks over a binary curve: the +-- coordinates reduce to themselves in the field, and +-- @y^2 + x*y = x^3 + a*x^2 + b@ holds. +-- +-- This is the check to make on a point that arrives from elsewhere, before +-- multiplying it by a private number. Without it the multiplication is +-- carried out in whatever group the supplied point generates rather than the +-- curve group, and if that group is small the private number can be recovered +-- from the result. +-- +-- Two things it does not establish: +-- +-- * The point at infinity is reported as valid, since it is a member of the +-- curve group. It is not a usable peer value: multiplying it by anything +-- yields the point at infinity again, which has no coordinates. Reject it +-- separately where a peer is not allowed to send it. +-- +-- * Being on the curve is not membership of the subgroup generated by the base +-- point. The two coincide only when the cofactor is 1. Of the curves in +-- 'Crypto.PubKey.ECC.Types.CurveName' that holds for every prime curve +-- except @SEC_p112r2@ and @SEC_p128r2@, whose cofactor is 4, and for no +-- binary curve, whose cofactor is 2 or 4. Where the cofactor is above 1 a +-- point on the curve may still generate a small subgroup, and ruling that +-- out needs a further check -- multiplying by the group order and requiring +-- the point at infinity, or clearing the cofactor -- that this function does +-- not make. isPointValid :: Curve -> Point -> Bool isPointValid _ PointO = True isPointValid (CurveFP (CurvePrime p cc)) (Point x y) = @@ -205,6 +566,30 @@ add = addF2m mul = mulF2m fx isValid e = modF2m fx e == e + +-- | Check that a point is in the subgroup the base point generates, which is +-- the further check 'isPointValid' does not make. A point that is on the +-- curve but outside that subgroup answers a multiplication modulo an order +-- smaller than the group's, so the multiplier -- a private number, where the +-- point came from a peer -- is revealed modulo that small order. +-- +-- Where the cofactor is 1 the subgroup is the whole curve group and the +-- answer is 'True' for any point on the curve, at no cost. Otherwise the +-- point is multiplied by the group order and the answer is whether that +-- reaches the point at infinity, which costs one scalar multiplication. This +-- is the check OpenSSL's @EC_KEY_check_key@ makes. +-- +-- The point at infinity is reported as in the subgroup, as 'isPointValid' +-- reports it valid; it is a member, and unusable for other reasons. +-- +-- A point that is not on the curve at all has no meaningful answer here, so +-- check 'isPointValid' first. +isPointInSubgroup :: Curve -> Point -> Bool +isPointInSubgroup curve p + | ecc_h cc == 1 = True + | otherwise = pointMul curve (ecc_n cc) p == PointO + where + cc = common_curve curve -- | div and mod divmod :: Integer -> Integer -> Integer -> Maybe Integer diff --git a/Crypto/PubKey/ECC/Types.hs b/Crypto/PubKey/ECC/Types.hs --- a/Crypto/PubKey/ECC/Types.hs +++ b/Crypto/PubKey/ECC/Types.hs @@ -135,6 +135,19 @@ | SEC_t571r1 deriving (Show, Read, Eq, Ord, Enum, Bounded, Data) +{-# DEPRECATED + SEC_t113r1, SEC_t113r2, SEC_t131r1, SEC_t131r2, SEC_t163k1, SEC_t163r1, + SEC_t163r2, SEC_t193r1, SEC_t193r2, SEC_t233k1, SEC_t233r1, SEC_t239k1, + SEC_t283k1, SEC_t283r1, SEC_t409k1, SEC_t409r1, SEC_t571k1, SEC_t571r1 + [ "This curve is over a binary field, and those are obsolete." + , "They are also the curves whose cofactor is not 1, so a point from" + , "a peer needs the subgroup check that costs a further scalar" + , "multiplication; pyca/cryptography deprecated them for removal in" + , "the release that fixed CVE-2026-26007. This one will go in a" + , "later major version of crypton. Prefer a prime curve, or X25519." + ] + #-} + {- curvesOIDs :: [ (CurveName, [Integer]) ] curvesOIDs = diff --git a/Crypto/PubKey/Ed25519.hs b/Crypto/PubKey/Ed25519.hs --- a/Crypto/PubKey/Ed25519.hs +++ b/Crypto/PubKey/Ed25519.hs @@ -32,6 +32,7 @@ generateSecretKey, ) where +import Crypto.Debug (DebugShow (..), debugShowBytes) import Data.Word import Foreign.C.Types import Foreign.Ptr @@ -51,6 +52,9 @@ -- | An Ed25519 Secret key newtype SecretKey = SecretKey ScrubbedBytes deriving (Show, Eq, ByteArrayAccess, NFData) + +instance DebugShow SecretKey where + debugShow = debugShowBytes "SecretKey" -- | An Ed25519 public key newtype PublicKey = PublicKey Bytes diff --git a/Crypto/PubKey/Ed448.hs b/Crypto/PubKey/Ed448.hs --- a/Crypto/PubKey/Ed448.hs +++ b/Crypto/PubKey/Ed448.hs @@ -36,6 +36,7 @@ generateSecretKey, ) where +import Crypto.Debug (DebugShow (..), debugShowBytes) import Data.Word import Foreign.C.Types import Foreign.Ptr @@ -55,6 +56,9 @@ -- | An Ed448 Secret key newtype SecretKey = SecretKey ScrubbedBytes deriving (Show, Eq, ByteArrayAccess, NFData) + +instance DebugShow SecretKey where + debugShow = debugShowBytes "SecretKey" -- | An Ed448 public key newtype PublicKey = PublicKey Bytes diff --git a/Crypto/PubKey/EdDSA.hs b/Crypto/PubKey/EdDSA.hs --- a/Crypto/PubKey/EdDSA.hs +++ b/Crypto/PubKey/EdDSA.hs @@ -51,6 +51,7 @@ generateSecretKey, ) where +import Crypto.Debug (DebugShow (..), debugShowBytes) import Data.Bits import Data.ByteArray ( ByteArray, @@ -85,6 +86,9 @@ newtype SecretKey curve = SecretKey ScrubbedBytes deriving (Show, Eq, ByteArrayAccess, NFData) +instance DebugShow (SecretKey curve) where + debugShow = debugShowBytes "SecretKey" + -- | An EdDSA public key newtype PublicKey curve hash = PublicKey Bytes deriving (Show, Eq, ByteArrayAccess, NFData) @@ -390,11 +394,13 @@ => proxy curve -> Signature curve hash -> CryptoFailable (Bytes, Point curve, Scalar curve) -decodeSignature prx (Signature bs) = do +decodeSignature prx sig@(Signature bs) = do let (bsR, bsS) = B.splitAt (publicKeySize prx) bs pR <- decodePoint prx bsR sS <- decodeScalarLE prx bsS - return (bsR, pR, sS) + if encodeSignature prx (encodePoint prx pR, pR, sS) == sig + then return (bsR, pR, sS) + else CryptoFailed CryptoError_PointFormatInvalid -- implementations are supposed to decode any scalar up to the size of the digest decodeScalarNoErr diff --git a/Crypto/PubKey/ElGamal.hs b/Crypto/PubKey/ElGamal.hs --- a/Crypto/PubKey/ElGamal.hs +++ b/Crypto/PubKey/ElGamal.hs @@ -1,3 +1,4 @@ +{-# LANGUAGE DeriveDataTypeable #-} {-# LANGUAGE GeneralizedNewtypeDeriving #-} -- | @@ -7,18 +8,43 @@ -- Stability : experimental -- Portability : Good -- --- This module is a work in progress. do not use: --- it might eat your dog, your data or even both. +-- ElGamal encryption and signature over the multiplicative group of integers +-- modulo a prime, reusing the parameters of "Crypto.PubKey.DH". -- --- TODO: provide a mapping between integer and ciphertext --- generate numbers correctly +-- /These are raw primitives, not a scheme./ The encryption here is textbook +-- ElGamal: it applies no padding, so it is malleable by construction -- +-- multiplying a ciphertext's second component by @t@ multiplies the plaintext +-- by @t@ -- and it is not IND-CCA secure. A message is an 'Integer' below the +-- modulus rather than a byte string, and nothing here maps one to the other. +-- Use it to build a scheme that adds those, or prefer +-- "Crypto.PubKey.RSA.OAEP" or "Crypto.PubKey.ECIES" where a scheme is what is +-- wanted. +-- +-- The signature primitive is likewise raw, and an ephemeral value must never +-- be reused between signatures: two signatures under the same @k@ reveal the +-- private key. +-- +-- == What is kept from the clock, and what is not +-- +-- Every exponentiation with a secret exponent is +-- 'Crypto.Number.ModArithmetic.expSafe'. Decryption inverts the shared +-- secret by Fermat's little theorem rather than by the extended Euclidean +-- algorithm, whose steps follow the bits it is given. 'sign' cannot do that +-- -- @k@ is inverted modulo @p-1@, which is even -- so it blinds instead: the +-- algorithm is handed @k@ times a fresh random unit, and the blinder is +-- divided out afterwards. 'signWith', having no randomness of its own, hands +-- it @k@. +-- +-- What is left is the 'Integer' arithmetic around all of that, whose cost +-- follows the size of the numbers. See "Crypto.PubKey.DSA" for the same note +-- at more length. module Crypto.PubKey.ElGamal ( Params, PublicNumber, PrivateNumber, EphemeralKey (..), SharedKey, - Signature, + Signature (..), -- * Generation generatePrivate, @@ -37,12 +63,13 @@ verify, ) where +import Crypto.Error import Crypto.Hash import Crypto.Internal.ByteArray (ByteArrayAccess) import Crypto.Internal.Imports import Crypto.Number.Basic (gcde) -import Crypto.Number.Generate (generateMax) -import Crypto.Number.ModArithmetic (expFast, expSafe, inverse) +import Crypto.Number.Generate (generateBetween, generateMax) +import Crypto.Number.ModArithmetic (expFast, expSafe, inverseSafe) import Crypto.Number.Serialize (os2ip) import Crypto.PubKey.DH ( Params (..), @@ -51,38 +78,59 @@ SharedKey (..), ) import Crypto.Random.Types -import Data.Maybe (fromJust) +import Data.Data -- | ElGamal Signature -data Signature = Signature (Integer, Integer) +data Signature = Signature + { sign_r :: Integer + -- ^ ElGamal r + , sign_s :: Integer + -- ^ ElGamal s + } + deriving (Show, Read, Eq, Data) +instance NFData Signature where + rnf (Signature r s) = r `seq` s `seq` () + -- | ElGamal Ephemeral key. also called Temporary key. newtype EphemeralKey = EphemeralKey Integer deriving (NFData) --- | generate a private number with no specific property --- this number is usually called a and need to be between --- 0 and q (order of the group G). +-- | generate a private number, in @[1, q-1]@ where @q@ is the order of the +-- group. Zero is excluded: it would make the public number 1 and the shared +-- value constant. generatePrivate :: MonadRandom m => Integer -> m PrivateNumber -generatePrivate q = PrivateNumber <$> generateMax q - --- | generate an ephemeral key which is a number with no specific property, --- and need to be between 0 and q (order of the group G). -generateEphemeral :: MonadRandom m => Integer -> m EphemeralKey -generateEphemeral q = toEphemeral <$> generatePrivate q - where - toEphemeral (PrivateNumber n) = EphemeralKey n +generatePrivate q = PrivateNumber <$> generateBetween 1 (q - 1) -- | generate a public number that is for the other party benefits. -- this number is usually called h=g^a generatePublic :: Params -> PrivateNumber -> PublicNumber generatePublic (Params p g _) (PrivateNumber a) = PublicNumber $ expSafe g a p +-- | Is the other party's public number usable? +-- +-- @1@ and @p-1@ generate a group of one or two elements, so the value they +-- mask the message with is one of a handful of constants. +validPublic :: Integer -> Integer -> Bool +validPublic p h = h > 1 && h < p - 1 + -- | encrypt with a specified ephemeral key --- do not reuse ephemeral key. +-- +-- The ephemeral key must lie in @[1, p-2]@ and must never be reused: zero +-- would leave the message unmasked, and a repeat lets anyone who learns one +-- plaintext recover the other. A message must be below the modulus, or +-- decryption would return it reduced. encryptWith - :: EphemeralKey -> Params -> PublicNumber -> Integer -> (Integer, Integer) -encryptWith (EphemeralKey b) (Params p g _) (PublicNumber h) m = (c1, c2) + :: EphemeralKey + -> Params + -> PublicNumber + -> Integer + -> CryptoFailable (Integer, Integer) +encryptWith (EphemeralKey b) (Params p g _) (PublicNumber h) m + | b < 1 || b > p - 2 = CryptoFailed CryptoError_ParameterInvalid + | not (validPublic p h) = CryptoFailed CryptoError_ParameterInvalid + | m < 0 || m >= p = CryptoFailed CryptoError_ParameterInvalid + | otherwise = CryptoPassed (c1, c2) where s = expSafe h b p c1 = expSafe g b p @@ -91,29 +139,52 @@ -- | encrypt a message using params and public keys -- will generate b (called the ephemeral key) encrypt - :: MonadRandom m => Params -> PublicNumber -> Integer -> m (Integer, Integer) -encrypt params@(Params p _ _) public m = (\b -> encryptWith b params public m) <$> generateEphemeral q - where - q = p - 1 -- p is prime, hence order of the group is p-1 + :: MonadRandom m + => Params + -> PublicNumber + -> Integer + -> m (CryptoFailable (Integer, Integer)) +encrypt params@(Params p _ _) public m + | p < 5 = return (CryptoFailed CryptoError_ParameterInvalid) + | otherwise = do + b <- generateBetween 1 (p - 2) + return $ encryptWith (EphemeralKey b) params public m -- | decrypt message -decrypt :: Params -> PrivateNumber -> (Integer, Integer) -> Integer -decrypt (Params p _ _) (PrivateNumber a) (c1, c2) = (c2 * sm1) `mod` p +-- +-- @c1@ must be a unit modulo @p@; a ciphertext whose first component is zero +-- or out of range is rejected rather than raising. +decrypt + :: Params -> PrivateNumber -> (Integer, Integer) -> CryptoFailable Integer +decrypt (Params p _ _) (PrivateNumber a) (c1, c2) + | c1 <= 0 || c1 >= p = CryptoFailed CryptoError_ParameterInvalid + | c2 < 0 || c2 >= p = CryptoFailed CryptoError_ParameterInvalid + | otherwise = case inverseSafe s p of + Nothing -> CryptoFailed CryptoError_ParameterInvalid + Just sm1 -> CryptoPassed ((c2 * sm1) `mod` p) where + -- the shared secret, which the extended Euclidean algorithm would take + -- apart: its steps follow the bits of what it is given, and this one is + -- worth the private number. p is prime, so Fermat gives the inverse + -- without reading it s = expSafe c1 a p - sm1 = fromJust $ inverse s p -- always inversible in Zp --- | sign a message with an explicit k number +-- | sign a message with an explicit ephemeral value -- --- if k is not appropriate, then no signature is returned. +-- @k@ has to lie in @[1, p-2]@ and be coprime with @p-1@. 'Nothing' says the +-- value handed in cannot be used: either it fails one of those two conditions, +-- or it is one of the few that produce a second component of zero. Either way +-- the answer is to draw another @k@, which is what 'sign' does. -- --- with some appropriate value of k, the signature generation can fail, --- and no signature is returned. User of this function need to retry --- with a different k value. +-- @k@ is an ephemeral private key. It has to be drawn uniformly at random, +-- kept secret, and used for one signature only: the private number follows +-- from a signature and its @k@, and equally from two signatures made with the +-- same @k@. None of that is visible to this function, which is why it takes +-- @k@ from the caller and checks only what it can. signWith :: (ByteArrayAccess msg, HashAlgorithm hash) => Integer - -- ^ random number k, between 0 and p-1 and gcd(k,p-1)=1 + -- ^ ephemeral value k, in [1, p-2] and coprime with p-1 -> Params -- ^ DH params (p,g) -> PrivateNumber @@ -123,21 +194,40 @@ -> msg -- ^ message to sign -> Maybe Signature -signWith k (Params p g _) (PrivateNumber x) hashAlg msg - | k >= p - 1 || d > 1 = Nothing -- gcd(k,p-1) is not 1 +signWith = signWithBlinder 1 + +-- | The same with a blinder for the inversion of @k@. +-- +-- @k@ is inverted modulo @p-1@, which is even, so Fermat's little theorem +-- does not reach it the way it reaches DSA's @k@ modulo a prime order: the +-- extended Euclidean algorithm is the only way there, and its steps follow +-- the bits of what it is given. What can be done instead is to hand it +-- something else: for a unit @b@, the inverse of @k*b@ times @b@ is the +-- inverse of @k@, and the steps then follow @k*b@, which is a fresh random +-- number. A blinder of 1 is no blinding, which is what the exported +-- 'signWith' has to do, having no randomness of its own. +-- +-- When @b@ shares a factor with @p-1@ the algorithm reports it the same way +-- it reports one in @k@, and the answer is the same: draw again. +signWithBlinder + :: (ByteArrayAccess msg, HashAlgorithm hash) + => Integer -> Integer -> Params -> PrivateNumber -> hash -> msg -> Maybe Signature +signWithBlinder b k (Params p g _) (PrivateNumber x) hashAlg msg + | k <= 0 || k >= p - 1 || b <= 0 || d > 1 = Nothing | s == 0 = Nothing - | otherwise = Just $ Signature (r, s) + | otherwise = Just $ Signature r s where r = expSafe g k p h = os2ip $ hashWith hashAlg msg s = ((h - x * r) * kInv) `mod` (p - 1) - (kInv, _, d) = gcde k (p - 1) + kInv = (kbInv * b) `mod` (p - 1) + (kbInv, _, d) = gcde ((k * b) `mod` (p - 1)) (p - 1) -- | sign message -- --- This function will generate a random number, however --- as the signature might fail, the function will automatically retry --- until a proper signature has been created. +-- This function draws the ephemeral value itself, and draws a fresh one on +-- each attempt until 'signWith' accepts it, so a caller who has no particular +-- @k@ in mind should use this rather than 'signWith'. sign :: (ByteArrayAccess msg, HashAlgorithm hash, MonadRandom m) => Params @@ -151,7 +241,10 @@ -> m Signature sign params@(Params p _ _) priv hashAlg msg = do k <- generateMax (p - 1) - case signWith k params priv hashAlg msg of + -- and a blinder for the inversion of k, which is the one step here that + -- the extended Euclidean algorithm has to do + b <- generateMax (p - 1) + case signWithBlinder b k params priv hashAlg msg of Nothing -> sign params priv hashAlg msg Just sig -> return sig @@ -164,7 +257,7 @@ -> msg -> Signature -> Bool -verify (Params p g _) (PublicNumber y) hashAlg msg (Signature (r, s)) +verify (Params p g _) (PublicNumber y) hashAlg msg (Signature r s) | or [r <= 0, r >= p, s <= 0, s >= (p - 1)] = False | otherwise = lhs == rhs where diff --git a/Crypto/PubKey/RSA.hs b/Crypto/PubKey/RSA.hs --- a/Crypto/PubKey/RSA.hs +++ b/Crypto/PubKey/RSA.hs @@ -1,3 +1,5 @@ +{-# LANGUAGE ScopedTypeVariables #-} + -- | -- Module : Crypto.PubKey.RSA -- License : BSD-style @@ -16,9 +18,16 @@ generateBlinder, ) where +import Crypto.Internal.ByteArray (ScrubbedBytes) import Crypto.Number.Generate (generateMax) -import Crypto.Number.ModArithmetic (inverse, inverseCoprimes) +import Crypto.Number.ModArithmetic ( + expSafe, + inverse, + inverseCoprimes, + inverseSafe, + ) import Crypto.Number.Prime (generatePrime) +import Crypto.Number.Serialize (os2ip) import Crypto.PubKey.RSA.Types import Crypto.Random.Types @@ -51,23 +60,64 @@ -- * e=0x10001 is a popular choice -- -- * e=3 is popular as well, but proven to not be as secure for some cases. +-- +-- /WARNING:/ Making a key is not constant time, and cannot be: the search for +-- the two primes takes as long as it takes, and 'Crypto.Number.Prime' is not +-- constant time either. What that leaks is about the search rather than +-- about the primes it settles on. Of the arithmetic that does touch them, +-- the inverse of one prime modulo the other is worked out without a side +-- channel, and so is the private exponent, which is the inverse of @e@ modulo +-- @(p-1)*(q-1)@: @e@ being public lets that be worked out as a remainder, an +-- inverse modulo @e@ itself, and an exact division, none of which follows the +-- number being inverted. An @e@ that is not prime keeps the extended +-- Euclidean algorithm, which for a public @e@ is one division by a small +-- number and then a few steps on numbers under it. generateWith :: (Integer, Integer) -- ^ chosen distinct primes p and q -> Int -- ^ size in bytes -> Integer - -- ^ RSA public exponent 'e' + -- ^ RSA public exponent @e@ -> Maybe (PublicKey, PrivateKey) generateWith (p, q) size e = - case inverse e phi of + case privateExponent of Nothing -> Nothing Just d -> Just (pub, priv d) where n = p * q phi = (p - 1) * (q - 1) - -- q and p should be *distinct* *prime* numbers, hence always coprime - qinv = inverseCoprimes q p + -- The private exponent is the inverse of e modulo phi, and phi is the + -- key. The extended Euclidean algorithm would take a number of steps + -- that follows it; e being public lets the work be about e instead. + -- + -- Whatever d is, e * d = 1 + k * phi for some k under e, and reading that + -- modulo e gives k = -phi^-1 mod e -- an inverse modulo a number of a + -- handful of bits, which for a prime e is Fermat. Then d is an exact + -- division by e. Nothing in that follows phi: the remainder and the + -- division are one pass each over its limbs, and the rest is arithmetic + -- the size of e. + -- + -- Fermat wants a prime e, and rather than ask whether e is one -- which + -- costs more than everything else here -- the k it gives is checked, + -- which is arithmetic the size of e. A composite e that fails the check + -- keeps the algorithm it had. + privateExponent + | e <= 1 = Nothing + | t == 0 = Nothing -- e divides phi, so there is no inverse + | (k * t) `mod` e == e - 1 = Just ((1 + k * phi) `div` e) + | otherwise = inverse e phi + where + t = phi `mod` e + k = (e - expSafe t (e - 2) e) `mod` e + -- q and p should be *distinct* *prime* numbers, hence always coprime. + -- Both of them are the key itself, so the inverse is worked out through + -- Fermat's little theorem rather than the extended Euclidean algorithm, + -- whose steps follow the numbers it is given. It falls back on the one + -- that raises, which is what a p that is not prime deserves. + qinv = case inverseSafe q p of + Just i -> i + Nothing -> inverseCoprimes q p pub = PublicKey { public_size = size @@ -91,7 +141,7 @@ => Int -- ^ size in bytes -> Integer - -- ^ RSA public exponent 'e' + -- ^ RSA public exponent @e@ -> m (PublicKey, PrivateKey) generate size e = loop where @@ -113,10 +163,30 @@ -- -- the unique parameter apart from the random number generator is the -- public key value N. +-- +-- The blinder holds a random number and its inverse. N is composite, so +-- Fermat has no answer for the inverse and it goes through the extended +-- Euclidean algorithm, whose steps follow the number handed to it -- which +-- would be the number the blinding rests on. So the algorithm is handed that +-- number multiplied by another random one instead, and its answer multiplied +-- by that number again, which leaves the inverse wanted and shows the +-- algorithm nothing that has anything to do with it. generateBlinder - :: MonadRandom m + :: forall m + . MonadRandom m => Integer -- ^ RSA public N parameter. -> m Blinder -generateBlinder n = - (\r -> Blinder r (inverseCoprimes r n)) <$> generateMax n +generateBlinder n = do + r <- generateMax n + -- The inverse goes through the extended Euclidean algorithm, whose steps + -- follow the number handed to it, and r is what the blinding rests on. + -- So another random number goes with it: the product is uniform and says + -- nothing about r on its own, and multiplying its inverse by that number + -- again leaves the inverse of r. Sixteen bytes are enough to hide it and + -- are under either prime, so the product is coprime with n whenever r is, + -- as it was before. + u <- os2ip <$> (getRandomBytes 16 :: m ScrubbedBytes) + let v = (r * u) `mod` n + rm1 = (inverseCoprimes v n * u) `mod` n + return $ Blinder r rm1 diff --git a/Crypto/PubKey/RSA/OAEP.hs b/Crypto/PubKey/RSA/OAEP.hs --- a/Crypto/PubKey/RSA/OAEP.hs +++ b/Crypto/PubKey/RSA/OAEP.hs @@ -21,18 +21,22 @@ ) where import Crypto.Hash +import Crypto.Number.Serialize (os2ip) import Crypto.PubKey.Internal (and') import Crypto.PubKey.MaskGenFunction import Crypto.PubKey.RSA (generateBlinder) import Crypto.PubKey.RSA.Prim import Crypto.PubKey.RSA.Types import Crypto.Random.Types -import Data.Bits (xor) +import Data.Bits (complement, shiftR, xor, (.&.), (.|.)) import Data.ByteString (ByteString) import qualified Data.ByteString as B +import Data.List (foldl') +import Data.Word (Word32) +import Prelude hiding (foldl') import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) -import qualified Crypto.Internal.ByteArray as B (convert) +import qualified Crypto.Internal.ByteArray as B (constEq, convert) -- | Parameters for OAEP encryption/decryption data OAEPParams hash seed output = OAEPParams @@ -109,6 +113,16 @@ -- | un-pad a OAEP encoded message. -- -- It doesn't apply the RSA decryption primitive +-- +-- The data block is scanned in full rather than up to the 01 octet separating +-- the padding from the message, and the label hash and the leading octet are +-- compared without an early exit, so neither the length of the padding nor +-- where a comparison first differs shows up in how long this takes. +-- +-- What remains visible is the result itself: whether the block was well formed, +-- and the length of the message when it was. That is the signal Manger's +-- attack needs, so a caller that decrypts attacker-supplied ciphertext must not +-- pass the distinction on. unpad :: HashAlgorithm hash => OAEPParams hash ByteString ByteString @@ -124,7 +138,9 @@ where -- parameters mgf = oaepMaskGenAlg oaep - labelHash = B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep) + labelHash = + B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep) + :: ByteString hashLen = hashDigestSize (oaepHash oaep) -- getting em's fields (pb, em0) = B.splitAt 1 em @@ -135,14 +151,32 @@ db = B.pack $ B.zipWith xor maskedDB dbmask -- getting db's fields (labelHash', db1) = B.splitAt hashLen db - (_, db2) = B.break (/= 0) db1 - (ps1, msg) = B.splitAt 1 db2 + -- index of the first nonzero octet in db1, or its length when every octet + -- is zero; all of them are looked at either way + oneIndex = + fst $ + foldl' + step + (fromIntegral (B.length db1) :: Word32, 1 :: Word32) + (zip [0 ..] (B.unpack db1)) + step (idx, unseen) (i, b) = (select found i idx, unseen .&. complement found) + where + w = fromIntegral b :: Word32 + -- 0 when b is zero, 1 otherwise + nonZero = (w .|. negate w) `shiftR` 31 + -- all ones at the first nonzero octet only + found = negate (unseen .&. nonZero) + select mask a b = (a .&. mask) .|. (b .&. complement mask) + + ps1 = B.take 1 $ B.drop (fromIntegral oneIndex) db1 + msg = B.drop (fromIntegral oneIndex + 1) db1 + paddingSuccess = and' - [ labelHash' == labelHash -- no need for constant eq - , ps1 == B.replicate 1 0x1 - , pb == B.replicate 1 0x0 + [ labelHash' `B.constEq` labelHash + , ps1 `B.constEq` B.replicate 1 0x1 + , pb `B.constEq` B.replicate 1 0x0 ] -- | Decrypt a ciphertext using OAEP @@ -151,6 +185,14 @@ -- information from the timing of the operation, the blinder can be set to None. -- -- If unsure always set a blinder or use decryptSafer +-- +-- Following RFC 8017, the ciphertext is rejected unless it is exactly as long +-- as the modulus (section 7.1.2, step 1) and its integer representative is +-- below the modulus (RSADP, section 5.1.2, step 1). The decryption primitive +-- normalises any multiple of the modulus away, so without the second check +-- @c + n@ would decrypt to the same message as @c@, and a ciphertext would not +-- be unique to its plaintext. Both checks are made on the ciphertext alone, +-- which is public, and report 'MessageSizeIncorrect'. decrypt :: HashAlgorithm hash => Maybe Blinder @@ -164,6 +206,7 @@ -> Either Error ByteString decrypt blinder oaep pk cipher | B.length cipher /= k = Left MessageSizeIncorrect + | os2ip cipher >= private_n pk = Left MessageSizeIncorrect | k < 2 * hashLen + 2 = Left InvalidParameters | otherwise = unpad oaep (private_size pk) $ dp blinder pk cipher where diff --git a/Crypto/PubKey/RSA/PKCS15.hs b/Crypto/PubKey/RSA/PKCS15.hs --- a/Crypto/PubKey/RSA/PKCS15.hs +++ b/Crypto/PubKey/RSA/PKCS15.hs @@ -25,17 +25,21 @@ ) where import Crypto.Hash +import Crypto.Number.Serialize (os2ip) import Crypto.PubKey.Internal (and') import Crypto.PubKey.RSA (generateBlinder) import Crypto.PubKey.RSA.Prim import Crypto.PubKey.RSA.Types import Crypto.Random.Types +import Data.Bits (complement, shiftR, (.&.), (.|.)) import Data.ByteString (ByteString) import Data.Word import Crypto.Internal.ByteArray (ByteArray, Bytes) import qualified Crypto.Internal.ByteArray as B +import Data.List (foldl') +import Prelude hiding (foldl') -- | A specialized class for hash algorithm that can product -- a ASN1 wrapped description the algorithm plus the content @@ -407,19 +411,44 @@ padding = 0 : 1 : (replicate (klen - siglen - 3) 0xff ++ [0]) -- | Try to remove a standard PKCS1.5 encryption padding. +-- +-- The block is scanned in full rather than up to the octet ending the padding +-- string, so how long that string is does not show up in how long this takes. +-- +-- What remains visible is the result itself: whether the padding was well +-- formed, and the length of the message when it was. That is inherent to the +-- scheme, and it is the signal Bleichenbacher's attack needs, so a caller that +-- decrypts attacker-supplied ciphertext must not pass the distinction on -- +-- TLS, for instance, continues with a random premaster secret and reports +-- nothing. unpad :: ByteArray bytearray => bytearray -> Either Error bytearray unpad packed | paddingSuccess = Right m | otherwise = Left MessageNotRecognized where + len = B.length packed (zt, ps0m) = B.splitAt 2 packed - (ps, zm) = B.span (/= 0) ps0m - (z, m) = B.splitAt 1 zm + + -- index of the first zero octet in ps0m, counted from the start of packed, + -- or len when there is none; every octet is looked at either way + zeroIndex = fst $ foldl' step (fromIntegral len :: Word32, 1 :: Word32) indexed + indexed = zip [2 ..] (B.unpack ps0m) + step (idx, unseen) (i, b) = (select found i idx, unseen .&. complement found) + where + w = fromIntegral b :: Word32 + -- 0 when b is zero, 1 otherwise + nonZero = (w .|. negate w) `shiftR` 31 + -- all ones at the first zero octet only + found = negate (unseen .&. complement nonZero) + select mask a b = (a .&. mask) .|. (b .&. complement mask) + + psLength = fromIntegral zeroIndex - 2 :: Int + m = B.drop (fromIntegral zeroIndex + 1) packed paddingSuccess = and' [ zt `B.constEq` (B.pack [0, 2] :: Bytes) - , z == B.zero 1 - , B.length ps >= 8 + , fromIntegral zeroIndex < len + , psLength >= 8 ] -- | decrypt message using the private key. @@ -430,6 +459,14 @@ -- If unsure always set a blinder or use decryptSafer -- -- The message is returned un-padded. +-- +-- Following RFC 8017, the ciphertext is rejected unless it is exactly as long +-- as the modulus (section 7.2.2, step 1) and its integer representative is +-- below the modulus (RSADP, section 5.1.2, step 1). The decryption primitive +-- normalises any multiple of the modulus away, so without the second check +-- @c + n@ would decrypt to the same message as @c@, and a ciphertext would not +-- be unique to its plaintext. Both checks are made on the ciphertext alone, +-- which is public, and report 'MessageSizeIncorrect'. decrypt :: ByteArray ba => Maybe Blinder @@ -441,6 +478,7 @@ -> Either Error ba decrypt blinder pk c | B.length c /= (private_size pk) = Left MessageSizeIncorrect + | os2ip c >= private_n pk = Left MessageSizeIncorrect -- "convert" must be apply to "c". | otherwise = unpad $ dp blinder pk $ B.convert c @@ -502,6 +540,14 @@ return (sign (Just blinder) hashAlg pk m) -- | verify message with the signed message +-- +-- Following RFC 8017, the signature is rejected unless it is exactly as long +-- as the modulus (section 8.2.2, step 1) and its integer representative is +-- below the modulus (section 5.2.2, step 1). Verification works by +-- re-encoding the expected signature and comparing it with the result of the +-- public-key operation, and that operation normalises away both the length of +-- the encoding and any multiple of the modulus; without these checks a +-- zero-padded signature, or @s + n@, would verify just as well as @s@. verify :: HashAlgorithmASN1 hashAlg => Maybe hashAlg @@ -511,10 +557,13 @@ -> ByteString -- ^ Signature -> Bool -verify hashAlg pk m sm = - case makeSignature hashAlg (public_size pk) m of - Left _ -> False - Right s -> s == (ep pk sm) +verify hashAlg pk m sm + | B.length sm /= public_size pk = False + | os2ip sm >= public_n pk = False + | otherwise = + case makeSignature hashAlg (public_size pk) m of + Left _ -> False + Right s -> s == (ep pk sm) -- | make signature digest, used in 'sign' and 'verify' makeSignature diff --git a/Crypto/PubKey/RSA/PSS.hs b/Crypto/PubKey/RSA/PSS.hs --- a/Crypto/PubKey/RSA/PSS.hs +++ b/Crypto/PubKey/RSA/PSS.hs @@ -22,6 +22,7 @@ import Crypto.Hash import Crypto.Number.Basic (numBits) +import Crypto.Number.Serialize (os2ip) import Crypto.PubKey.MaskGenFunction import Crypto.PubKey.RSA (generateBlinder) import Crypto.PubKey.RSA.Prim @@ -197,6 +198,13 @@ mHash = hashWith (pssHash params) m -- | Verify a signature using the PSS Parameters +-- +-- Following RFC 8017, the signature is rejected unless it is exactly as long +-- as the modulus (section 8.1.2, step 1) and its integer representative is +-- below the modulus (RSAVP1, section 5.2.2, step 1). The public-key operation +-- normalises any multiple of the modulus away, so without the second check +-- @s + n@ would verify as readily as @s@, and a third party could turn one +-- valid signature into another without the private key. verifyDigest :: HashAlgorithm hash => PSSParams hash ByteString ByteString @@ -211,6 +219,7 @@ -> Bool verifyDigest params pk digest s | B.length s /= k = False + | os2ip s >= public_n pk = False | B.any (/= 0) pre = False | B.last em /= pssTrailerField params = False | B.any (/= 0) ps0 = False diff --git a/Crypto/PubKey/RSA/Types.hs b/Crypto/PubKey/RSA/Types.hs --- a/Crypto/PubKey/RSA/Types.hs +++ b/Crypto/PubKey/RSA/Types.hs @@ -21,6 +21,7 @@ private_e, ) where +import Crypto.Debug (DebugShow (..)) import Crypto.Internal.Imports import Data.Data @@ -84,8 +85,39 @@ , private_qinv :: Integer -- ^ q^(-1) mod p } - deriving (Show, Read, Eq, Data, Generic) + deriving (Read, Eq, Data, Generic) +-- | The public part is shown; the secret fields are not. Use +-- 'Crypto.Debug.debugShow' to see them. +instance Show PrivateKey where + showsPrec d k = + showParen (d > 10) $ + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString + ", private_d = , private_p = \ + \, private_q = , private_dP = \ + \, private_dQ = , private_qinv = }" + +instance DebugShow PrivateKey where + debugShow k = + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_d = " + . shows (private_d k) + . showString ", private_p = " + . shows (private_p k) + . showString ", private_q = " + . shows (private_q k) + . showString ", private_dP = " + . shows (private_dP k) + . showString ", private_dQ = " + . shows (private_dQ k) + . showString ", private_qinv = " + . shows (private_qinv k) + . showChar '}' + $ "" + instance NFData PrivateKey where rnf (PrivateKey pub d p q dp dq qinv) = rnf pub `seq` @@ -113,7 +145,14 @@ -- -- note the RSA private key contains already an instance of public key for efficiency newtype KeyPair = KeyPair PrivateKey - deriving (Show, Read, Eq, Data, NFData) + deriving (Read, Eq, Data, NFData) + +instance Show KeyPair where + showsPrec d (KeyPair k) = + showParen (d > 10) $ showString "KeyPair " . showsPrec 11 k + +instance DebugShow KeyPair where + debugShow (KeyPair k) = "KeyPair (" ++ debugShow k ++ ")" -- | Public key of a RSA KeyPair toPublicKey :: KeyPair -> PublicKey diff --git a/Crypto/PubKey/Rabin/Basic.hs b/Crypto/PubKey/Rabin/Basic.hs --- a/Crypto/PubKey/Rabin/Basic.hs +++ b/Crypto/PubKey/Rabin/Basic.hs @@ -8,6 +8,23 @@ -- Portability : unknown -- -- Rabin cryptosystem for public-key cryptography and digital signature. +-- +-- == What is kept from the clock, and what is not +-- +-- The square roots modulo the secret primes are taken with +-- 'Crypto.Number.ModArithmetic.expSafe', which does not read the exponent it +-- is given. Two things here do read what they are given. +-- +-- Signing asks for the Jacobi symbol of the hash modulo each of the two +-- private primes, and the Jacobi symbol is computed by a sequence of +-- reductions whose number follows both of its arguments -- so the work done +-- per signature follows the primes. Key generation runs the extended +-- Euclidean algorithm on the two primes for the same reason. Neither has a +-- drop-in replacement here: a Jacobi symbol that does not read its arguments +-- is a different algorithm, not a different call. +-- +-- Around all of that is 'Integer' arithmetic, whose cost follows the size of +-- the numbers; see "Crypto.PubKey.DSA" for that note at more length. module Crypto.PubKey.Rabin.Basic ( PublicKey (..), PrivateKey (..), @@ -21,6 +38,7 @@ verify, ) where +import Crypto.Debug (DebugShow (..)) import Data.ByteString (ByteString) import qualified Data.ByteString as B import Data.Data @@ -53,8 +71,32 @@ , private_a :: Integer , private_b :: Integer } - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +-- | The public part is shown; the secret fields are not. Use +-- 'Crypto.Debug.debugShow' to see them. +instance Show PrivateKey where + showsPrec d k = + showParen (d > 10) $ + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_p = , private_q = , private_a = , private_b = }" + +instance DebugShow PrivateKey where + debugShow k = + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_p = " + . shows (private_p k) + . showString ", private_q = " + . shows (private_q k) + . showString ", private_a = " + . shows (private_a k) + . showString ", private_b = " + . shows (private_b k) + . showChar '}' + $ "" + -- | Rabin Signature. data Signature = Signature (Integer, Integer) deriving (Show, Read, Eq, Data) @@ -128,6 +170,12 @@ -- | Decrypt ciphertext using private key. -- +-- The ciphertext has to be what 'encrypt' produces: the big-endian encoding, +-- with no leading zero octet, of a value below the modulus. Squaring and the +-- square roots that undo it work modulo n, so without that condition @c@ and +-- @c + n@ -- and @c@ with a zero octet in front of it -- would all decrypt to +-- the same message, and a ciphertext would not be unique to its plaintext. +-- -- See algorithm 8.12 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al. decrypt :: HashAlgorithm hash @@ -138,18 +186,21 @@ -> ByteString -- ^ ciphertext -> Maybe ByteString -decrypt oaep pk c = - let p = private_p pk - q = private_q pk - a = private_a pk - b = private_b pk - n = public_n $ private_pub pk - k = numBytes n - c' = os2ip c - solutions = rights $ toList $ mapTuple (unpad oaep k . i2ospOf_ k) $ sqroot' c' p q a b n - in case solutions of - [x] -> Just x - _ -> Nothing +decrypt oaep pk c + | os2ip c >= public_n (private_pub pk) = Nothing + | c /= (i2osp (os2ip c) :: ByteString) = Nothing + | otherwise = + let p = private_p pk + q = private_q pk + a = private_a pk + b = private_b pk + n = public_n $ private_pub pk + k = numBytes n + c' = os2ip c + solutions = rights $ toList $ mapTuple (unpad oaep k . i2ospOf_ k) $ sqroot' c' p q a b n + in case solutions of + [x] -> Just x + _ -> Nothing where toList (w, x, y, z) = w : x : y : z : [] mapTuple f (w, x, y, z) = (f w, f x, f y, f z) @@ -168,10 +219,14 @@ -> ByteString -- ^ message to sign -> Either Error Signature -signWith padding pk hashAlg m = do - h <- calculateHash padding pk hashAlg m - signature <- calculateSignature h - return signature +signWith padding pk hashAlg m + -- the signature carries the padding as an integer, so a leading zero octet + -- would not survive it: verify would hash one octet less than was signed + | B.null padding || B.index padding 0 == 0 = Left InvalidParameters + | otherwise = do + h <- calculateHash padding pk hashAlg m + signature <- calculateSignature h + return signature where calculateSignature h = let p = private_p pk @@ -203,8 +258,10 @@ where findPadding = do padding <- getRandomBytes 8 - case calculateHash padding pk hashAlg m of - Right _ -> return padding + case (B.index padding 0, calculateHash padding pk hashAlg m) of + -- a padding that starts with a zero octet is one signWith refuses + (0, _) -> findPadding + (_, Right _) -> return padding _ -> findPadding -- | Calculate hash of message and padding. @@ -242,12 +299,17 @@ -> Signature -- ^ signature -> Bool -verify pk hashAlg m (Signature (padding, s)) = - let n = public_n pk - p = i2osp padding - h = os2ip $ hashWith hashAlg $ B.append p m - h' = expSafe s 2 n - in h' == h +verify pk hashAlg m (Signature (padding, s)) + -- squaring works modulo n, so s + n and -s would verify wherever s does + | s < 0 || s >= n = False + | padding < 0 = False + | otherwise = + let p = i2osp padding + h = os2ip $ hashWith hashAlg $ B.append p m + h' = expSafe s 2 n + in h' == h + where + n = public_n pk -- | Square roots modulo prime p where p is congruent 3 mod 4 -- Value a must be a quadratic residue modulo p (i.e. jacobi symbol (a/n) = 1). diff --git a/Crypto/PubKey/Rabin/Modified.hs b/Crypto/PubKey/Rabin/Modified.hs --- a/Crypto/PubKey/Rabin/Modified.hs +++ b/Crypto/PubKey/Rabin/Modified.hs @@ -9,6 +9,9 @@ -- -- Modified-Rabin public-key digital signature algorithm. -- See algorithm 11.30 in "Handbook of Applied Cryptography" by Alfred J. Menezes et al. +-- The Jacobi symbols here are taken modulo the public modulus, not the +-- private primes, so what "Crypto.PubKey.Rabin.Basic" says about that does +-- not apply; the note there about 'Integer' arithmetic does. module Crypto.PubKey.Rabin.Modified ( PublicKey (..), PrivateKey (..), @@ -17,6 +20,7 @@ verify, ) where +import Crypto.Debug (DebugShow (..)) import Data.ByteString import Data.Data @@ -44,8 +48,30 @@ -- ^ q prime number , private_d :: Integer } - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +-- | The public part is shown; the secret fields are not. Use +-- 'Crypto.Debug.debugShow' to see them. +instance Show PrivateKey where + showsPrec d k = + showParen (d > 10) $ + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_p = , private_q = , private_d = }" + +instance DebugShow PrivateKey where + debugShow k = + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_p = " + . shows (private_p k) + . showString ", private_q = " + . shows (private_q k) + . showString ", private_d = " + . shows (private_d k) + . showChar '}' + $ "" + -- | Generate a pair of (private, public) key of size in bytes. -- Prime p is congruent 3 mod 8 and prime q is congruent 7 mod 8. generate @@ -109,18 +135,23 @@ -> Integer -- ^ signature -> Bool -verify pk hashAlg m s = - let n = public_n pk - h = os2ip $ hashWith hashAlg m - s' = expSafe s 2 n - s'' = case s' `mod` 8 of - 6 -> s' - 3 -> 2 * s' - 7 -> n - s' - 2 -> 2 * (n - s') - _ -> 0 - in case s'' `mod` 16 of - 6 -> - let h' = (s'' - 6) `div` 16 - in h' == h - _ -> False +verify pk hashAlg m s + -- squaring works modulo n, so s + n and -s would verify wherever s does + | s < 0 || s >= n = False + | otherwise = go + where + n = public_n pk + go = + let h = os2ip $ hashWith hashAlg m + s' = expSafe s 2 n + s'' = case s' `mod` 8 of + 6 -> s' + 3 -> 2 * s' + 7 -> n - s' + 2 -> 2 * (n - s') + _ -> 0 + in case s'' `mod` 16 of + 6 -> + let h' = (s'' - 6) `div` 16 + in h' == h + _ -> False diff --git a/Crypto/PubKey/Rabin/OAEP.hs b/Crypto/PubKey/Rabin/OAEP.hs --- a/Crypto/PubKey/Rabin/OAEP.hs +++ b/Crypto/PubKey/Rabin/OAEP.hs @@ -14,13 +14,16 @@ unpad, ) where -import Data.Bits (xor) +import Data.Bits (complement, shiftR, xor, (.&.), (.|.)) import Data.ByteString (ByteString) import qualified Data.ByteString as B +import Data.List (foldl') +import Data.Word (Word32) +import Prelude hiding (foldl') import Crypto.Hash import Crypto.Internal.ByteArray (ByteArray, ByteArrayAccess) -import qualified Crypto.Internal.ByteArray as B (convert) +import qualified Crypto.Internal.ByteArray as B (constEq, convert) import Crypto.PubKey.Internal (and') import Crypto.PubKey.MaskGenFunction import Crypto.PubKey.Rabin.Types @@ -80,6 +83,17 @@ em = B.concat [B.singleton 0x0, maskedSeed, maskedDB] -- | Un-pad a OAEP encoded message. +-- +-- The data block is scanned in full rather than up to the 01 octet separating +-- the padding from the message, and the label hash and the leading octet are +-- compared without an early exit, so neither the length of the padding nor +-- where a comparison first differs shows up in how long this takes. This is +-- what "Crypto.PubKey.RSA.OAEP" does with the same block. +-- +-- What remains visible is the result itself: whether the block was well formed, +-- and the length of the message when it was. That is the signal Manger's +-- attack needs, so a caller that decrypts attacker-supplied ciphertext must not +-- pass the distinction on. unpad :: HashAlgorithm hash => OAEPParams hash ByteString ByteString @@ -95,7 +109,9 @@ where -- parameters mgf = oaepMaskGenAlg oaep - labelHash = B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep) + labelHash = + B.convert $ hashWith (oaepHash oaep) (maybe B.empty id $ oaepLabel oaep) + :: ByteString hashLen = hashDigestSize (oaepHash oaep) -- getting em's fields (pb, em0) = B.splitAt 1 em @@ -106,12 +122,30 @@ db = B.pack $ B.zipWith xor maskedDB dbmask -- getting db's fields (labelHash', db1) = B.splitAt hashLen db - (_, db2) = B.break (/= 0) db1 - (ps1, msg) = B.splitAt 1 db2 + -- index of the first nonzero octet in db1, or its length when every octet + -- is zero; all of them are looked at either way + oneIndex = + fst $ + foldl' + step + (fromIntegral (B.length db1) :: Word32, 1 :: Word32) + (zip [0 ..] (B.unpack db1)) + step (idx, unseen) (i, b) = (select found i idx, unseen .&. complement found) + where + w = fromIntegral b :: Word32 + -- 0 when b is zero, 1 otherwise + nonZero = (w .|. negate w) `shiftR` 31 + -- all ones at the first nonzero octet only + found = negate (unseen .&. nonZero) + select mask a b = (a .&. mask) .|. (b .&. complement mask) + + ps1 = B.take 1 $ B.drop (fromIntegral oneIndex) db1 + msg = B.drop (fromIntegral oneIndex + 1) db1 + paddingSuccess = and' - [ labelHash' == labelHash -- no need for constant eq - , ps1 == B.replicate 1 0x1 - , pb == B.replicate 1 0x0 + [ labelHash' `B.constEq` labelHash + , ps1 `B.constEq` B.replicate 1 0x1 + , pb `B.constEq` B.replicate 1 0x0 ] diff --git a/Crypto/PubKey/Rabin/RW.hs b/Crypto/PubKey/Rabin/RW.hs --- a/Crypto/PubKey/Rabin/RW.hs +++ b/Crypto/PubKey/Rabin/RW.hs @@ -10,6 +10,9 @@ -- Rabin-Williams cryptosystem for public-key encryption and digital signature. -- See pages 323 - 324 in "Computational Number Theory and Modern Cryptography" by Song Y. Yan. -- Also inspired by https://github.com/vanilala/vncrypt/blob/master/vncrypt/vnrw_gmp.c. +-- The Jacobi symbols here are taken modulo the public modulus, not the +-- private primes, so what "Crypto.PubKey.Rabin.Basic" says about that does +-- not apply; the note there about 'Integer' arithmetic does. module Crypto.PubKey.Rabin.RW ( PublicKey (..), PrivateKey (..), @@ -21,6 +24,7 @@ verify, ) where +import Crypto.Debug (DebugShow (..)) import Data.ByteString import Data.Data @@ -50,8 +54,30 @@ -- ^ q prime number , private_d :: Integer } - deriving (Show, Read, Eq, Data) + deriving (Read, Eq, Data) +-- | The public part is shown; the secret fields are not. Use +-- 'Crypto.Debug.debugShow' to see them. +instance Show PrivateKey where + showsPrec d k = + showParen (d > 10) $ + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_p = , private_q = , private_d = }" + +instance DebugShow PrivateKey where + debugShow k = + showString "PrivateKey {private_pub = " + . shows (private_pub k) + . showString ", private_p = " + . shows (private_p k) + . showString ", private_q = " + . shows (private_q k) + . showString ", private_d = " + . shows (private_d k) + . showChar '}' + $ "" + -- | Generate a pair of (private, public) key of size in bytes. -- Prime p is congruent 3 mod 8 and prime q is congruent 7 mod 8. generate @@ -118,6 +144,12 @@ hashLen = hashDigestSize (oaepHash oaep) -- | Decrypt ciphertext using private key. +-- +-- The ciphertext has to be what 'encrypt' produces: the big-endian encoding, +-- with no leading zero octet, of a value below the modulus. The primitives +-- work modulo n, so without that condition @c@ and @c + n@ -- and @c@ with a +-- zero octet in front of it -- would all decrypt to the same message, and a +-- ciphertext would not be unique to its plaintext. decrypt :: HashAlgorithm hash => OAEPParams hash ByteString ByteString @@ -127,14 +159,17 @@ -> ByteString -- ^ ciphertext -> Maybe ByteString -decrypt oaep pk c = - let d = private_d pk - n = public_n $ private_pub pk - k = numBytes n - c' = i2ospOf_ k $ dp2 n $ dp1 d n $ os2ip c - in case unpad oaep k c' of - Left _ -> Nothing - Right p -> Just p +decrypt oaep pk c + | os2ip c >= public_n (private_pub pk) = Nothing + | c /= (i2osp (os2ip c) :: ByteString) = Nothing + | otherwise = + let d = private_d pk + n = public_n $ private_pub pk + k = numBytes n + c' = i2ospOf_ k $ dp2 n $ dp1 d n $ os2ip c + in case unpad oaep k c' of + Left _ -> Nothing + Right p -> Just p -- | Sign message using hash algorithm and private key. sign @@ -165,11 +200,15 @@ -> Integer -- ^ signature -> Bool -verify pk hashAlg m s = - let n = public_n pk - h = os2ip $ hashWith hashAlg m - h' = dp2 n $ ep2 n s - in h' == h +verify pk hashAlg m s + -- squaring works modulo n, so s + n and -s would verify wherever s does + | s < 0 || s >= n = False + | otherwise = + let h = os2ip $ hashWith hashAlg m + h' = dp2 n $ ep2 n s + in h' == h + where + n = public_n pk -- | Encryption primitive 1 ep1 :: Integer -> Integer -> Either Error Integer diff --git a/Crypto/PubKey/Rabin/Types.hs b/Crypto/PubKey/Rabin/Types.hs --- a/Crypto/PubKey/Rabin/Types.hs +++ b/Crypto/PubKey/Rabin/Types.hs @@ -6,6 +6,7 @@ -- Portability : unknown module Crypto.PubKey.Rabin.Types ( Error (..), + PrimeCondition, generatePrimes, ) where diff --git a/Crypto/Random/Probabilistic.hs b/Crypto/Random/Probabilistic.hs --- a/Crypto/Random/Probabilistic.hs +++ b/Crypto/Random/Probabilistic.hs @@ -5,23 +5,49 @@ -- Stability : experimental -- Portability : Good module Crypto.Random.Probabilistic ( - probabilistic, + probabilisticFrom, ) where +import Crypto.Hash (SHA512 (..), hashWith) +import Crypto.Internal.ByteArray (ByteArrayAccess, ScrubbedBytes) +import qualified Crypto.Internal.ByteArray as B import Crypto.Internal.Compat import Crypto.Random +import Crypto.Random.ChaChaDRG (initialize) --- | This create a random number generator out of thin air with --- the system entropy; don't generally use as the IO is not exposed --- this can have unexpected random for. +-- | Run a probabilistic algorithm on a generator derived from the value it is +-- about to work on, and from a secret this process drew once. -- --- This is useful for probabilistic algorithm like Miller Rabin --- probably prime algorithm, given appropriate choice of the heuristic +-- This is useful for a probabilistic algorithm like the Miller-Rabin primality +-- test, where the caller is a pure function and has to behave like one: the +-- same value has to give the same answer for as long as the process lives. +-- Deriving the generator from the value gives that much, and it keeps the +-- draws made for two different values independent of each other -- one +-- generator made once and shared by every call would make the witnesses drawn +-- for one value the witnesses for every value. -- +-- The process secret is what makes the derivation unpredictable. The values +-- worked on may come from wherever the caller's input comes from, so the +-- generator must not be something that can be worked out from them. +-- +-- The IO is not exposed and the result is not reproducible between processes. -- Generally, it's advised not to use this function. -probabilistic :: MonadPseudoRandom ChaChaDRG a -> a -probabilistic f = fst $ withDRG drg f +probabilisticFrom + :: ByteArrayAccess seed + => seed + -- ^ the value being worked on, as bytes + -> MonadPseudoRandom ChaChaDRG a + -> a +probabilisticFrom material f = fst $ withDRG drg f where - {-# NOINLINE drg #-} - drg = unsafeDoIO drgNew -{-# NOINLINE probabilistic #-} + drg = initialize (B.take seedLength (B.convert digest :: ScrubbedBytes)) + digest = hashWith SHA512 (B.append secret (B.convert material) :: ScrubbedBytes) + -- what Crypto.Random.ChaChaDRG.initialize wants, and no more than SHA-512 + -- produces + seedLength = 40 + +-- | Drawn once, for the lifetime of the process: it is the only part of the +-- derivation above that an attacker supplying values cannot see. +secret :: ScrubbedBytes +secret = unsafeDoIO (getRandomBytes 32) +{-# NOINLINE secret #-} diff --git a/Crypto/Random/Types.hs b/Crypto/Random/Types.hs --- a/Crypto/Random/Types.hs +++ b/Crypto/Random/Types.hs @@ -53,6 +53,6 @@ getRandomBytes n = MonadPseudoRandom (randomBytesGenerate n) -- | Run a pure computation with a Deterministic Random Generator --- in the 'MonadPseudoRandom' +-- in the t'MonadPseudoRandom' withDRG :: DRG gen => gen -> MonadPseudoRandom gen a -> (a, gen) withDRG gen m = runPseudoRandom m gen diff --git a/Crypto/Tutorial.hs b/Crypto/Tutorial.hs --- a/Crypto/Tutorial.hs +++ b/Crypto/Tutorial.hs @@ -160,6 +160,7 @@ -- > import Data.ByteString (ByteString) -- > import qualified Data.ByteString as B -- > +-- > import Crypto.Error (throwCryptoError) -- > import qualified Crypto.Cipher.XSalsa as XSalsa -- > import qualified Crypto.MAC.Poly1305 as Poly1305 -- > import qualified Crypto.PubKey.Curve25519 as X25519 @@ -175,7 +176,8 @@ -- > state1 = XSalsa.derive state0 iv1 -- > (rs, state2) = XSalsa.generate state1 32 -- > (c, _) = XSalsa.combine state2 content --- > tag = Poly1305.auth (rs :: ByteString) c +-- > macKey = throwCryptoError (Poly1305.key (rs :: ByteString)) +-- > tag = Poly1305.auth macKey c -- > -- > -- | Try to open a @crypto_box@ packet and recover the content using the -- > -- 192-bit nonce, sender public key and receiver private key. @@ -192,4 +194,5 @@ -- > state1 = XSalsa.derive state0 iv1 -- > (rs, state2) = XSalsa.generate state1 32 -- > (content, _) = XSalsa.combine state2 c --- > tag = Poly1305.auth (rs :: ByteString) c +-- > macKey = throwCryptoError (Poly1305.key (rs :: ByteString)) +-- > tag = Poly1305.auth macKey c diff --git a/README.md b/README.md --- a/README.md +++ b/README.md @@ -13,81 +13,140 @@ If you have no idea what you're doing, please do not use this directly. Instead, rely on higher level protocols or implementations. -Documentation: [crypton on hackage](http://hackage.haskell.org/package/crypton) +Performance +----------- -Stability ---------- +The algorithms a TLS connection uses, measured against the previous release +and against OpenSSL on the same machine. Throughput is over 16 KiB messages; +the public key operations are one operation each; every figure is the best of +several runs, and crypton and OpenSSL are run alternately so that neither gets +the quieter machine. -Crypton APIs are stable, and we only strive to add, not change or remove. -Note that because the API exposed is wide and also expose internals things (for -power users and flexibility), certains APIs can be revised in extreme cases -where we can't just add. +Bulk encryption and hashing are measured through crypton's C layer, as +`openssl speed` measures OpenSSL's. The public key operations are measured +through crypton's Haskell API, since that is where ECDSA and RSA live and it +is what a program actually calls; the Haskell layer adds well under a +microsecond, which the X25519 and ECDH P-256 rows confirm by agreeing with a +C-level measurement to within a percent. Both releases of crypton are built +the same way -- `-optc-O3`, which is what both of them ask for -- and each +column of a table comes from one run on the machine named above it. -Versioning ----------- +### x86-64 -Next version of `0.x` is `0.(x+1)`. There's no exceptions, or API related meaning -behind the numbers. +An AMD EPYC 7763, which has AES-NI, PCLMULQDQ, AVX2, ADX and the SHA +extensions, against OpenSSL 3.0.13. -Coding Style ------------- +Throughput in MB/s, **higher is better**: -The coding style of this project mostly follows: -[haskell-style](https://github.com/tibbe/haskell-style-guide/blob/master/haskell-style.md) +| | crypton 1.1.5 | crypton 2.0.0 | OpenSSL | 2.0.0 / OpenSSL | +| --- | ---: | ---: | ---: | ---: | +| AES-128-GCM | 1331 | 4118 | 4264 | 0.97 | +| AES-256-GCM | 1090 | 3810 | 3951 | 0.96 | +| ChaCha20-Poly1305 | 398 | 2195 | 2191 | 1.00 | +| SHA-1 | 738 | 1678 | 1672 | 1.00 | +| SHA-256 | 286 | 1585 | 1570 | 1.01 | +| SHA-512 | 448 | 769 | 746 | 1.03 | +| SHA3-256 | 109 | 421 | 426 | 0.99 | -Support -------- +Time per operation in microseconds, **lower is better** -- so the last column +divides OpenSSL's time by crypton's, and is again better the larger it is: -See [Haskell packages guidelines](https://github.com/vincenthz/haskell-pkg-guidelines/blob/master/README.md#support) +| | crypton 1.1.5 | crypton 2.0.0 | OpenSSL | OpenSSL / 2.0.0 | +| --- | ---: | ---: | ---: | ---: | +| X25519 | 43.57 | 43.52 | 36.58 | 0.84 | +| ECDH P-256 | 163.8 | 163.7 | 52.36 | 0.32 | +| ECDH P-384 | 2241 | 1101 | 857.1 | 0.78 | +| Ed25519 sign | 28.52 | 28.25 | 43.49 | 1.54 | +| Ed25519 verify | 46.06 | 46.16 | 119.3 | 2.59 | +| ECDSA P-256 sign | 76.04 | 75.02 | 22.91 | 0.31 | +| ECDSA P-256 verify | 229.2 | 228.5 | 67.98 | 0.30 | +| ECDSA P-384 sign | 2271 | 387.6 | 904.3 | 2.33 | +| ECDSA P-384 verify | 2667 | 1493 | 746.2 | 0.50 | +| RSA-2048 sign/decrypt | 759.8 | 1311 | 660.1 | 0.50 | +| RSA-2048 verify/encrypt | 31.62 | 28.59 | 18.63 | 0.65 | -Known Building Issues ---------------------- +### AArch64 -On OSX <= 10.7, the system compiler doesn't understand the '-maes' option, and -with the lack of autodetection feature builtin in .cabal file, it is left on -the user to disable the aesni. See the [Disabling AESNI] section +An Apple M4, which has the AES, PMULL, SHA-1, SHA-2, SHA-512 and SHA-3 +instructions, against OpenSSL 3.6.4. -On CentOS 7 the default C compiler includes intrinsic header files incompatible -with per-function target options. Solutions are to use GCC >= 4.9 or disable -flag *use_target_attributes* (see flag configuration examples below). +Throughput in MB/s, **higher is better**: -Disabling AESNI ---------------- +| | crypton 1.1.5 | crypton 2.0.0 | OpenSSL | 2.0.0 / OpenSSL | +| --- | ---: | ---: | ---: | ---: | +| AES-128-GCM | 126 | 8702 | 10719 | 0.81 | +| AES-256-GCM | 98 | 7648 | 9154 | 0.84 | +| ChaCha20-Poly1305 | 758 | 2323 | 2244 | 1.04 | +| SHA-1 | 1199 | 3380 | 3346 | 1.01 | +| SHA-256 | 467 | 3394 | 3352 | 1.01 | +| SHA-512 | 723 | 1868 | 1851 | 1.01 | +| SHA3-256 | 548 | 1091 | 1054 | 1.04 | -It may be useful to disable AESNI for building, testing or runtime purposes. -This is achieved with the *support_aesni* flag. +Time per operation in microseconds, **lower is better**; the last column again +divides OpenSSL's time by crypton's: -As part of configure of crypton: +| | crypton 1.1.5 | crypton 2.0.0 | OpenSSL | OpenSSL / 2.0.0 | +| --- | ---: | ---: | ---: | ---: | +| X25519 | 18.44 | 18.41 | 18.41 | 1.00 | +| ECDH P-256 | 69.46 | 56.24 | 24.68 | 0.44 | +| ECDH P-384 | 3252 | 511.1 | 379.7 | 0.74 | +| Ed25519 sign | 13.75 | 13.14 | 15.90 | 1.21 | +| Ed25519 verify | 18.17 | 18.04 | 39.27 | 2.18 | +| ECDSA P-256 sign | 32.56 | 27.91 | 11.05 | 0.40 | +| ECDSA P-256 verify | 96.50 | 80.19 | 32.82 | 0.41 | +| ECDSA P-384 sign | 3219 | 169.3 | 403.4 | 2.38 | +| ECDSA P-384 verify | 3807 | 688.1 | 335.1 | 0.49 | +| RSA-2048 sign/decrypt | 451.8 | 605.4 | 325.0 | 0.54 | +| RSA-2048 verify/encrypt | 18.32 | 15.28 | 8.50 | 0.56 | -``` - cabal configure --flag='-support_aesni' -``` +### What the numbers say -or as part of an installation: +1.1.5 had no AArch64 code of its own at all, which is why AES-GCM there is +sixty-nine times what it was. On x86-64 it had AES-NI and nothing else. The +curves over a prime field other than P-256 moved from Haskell `Integer` +arithmetic into C, which is the nineteenfold change in ECDSA P-384 signing on +the M4. X25519 and Ed25519 are unchanged between the two releases, and the +rows say so: where they differ by half a percent, that is the measurement and +not the code. P-256 is unchanged on x86-64 and a fifth faster on AArch64, +which is the paragraph below. -``` - cabal install --constraint="crypton -support_aesni" -``` +Where crypton is behind, it is behind for three separate reasons. -For help with cabal flags, see: [stackoverflow : is there a way to define flags for cabal](http://stackoverflow.com/questions/23523869/is-there-any-way-to-define-flags-for-cabal-dependencies) +*P-256.* crypton's field arithmetic is C where OpenSSL's is hand-written +assembly, and that is what is left of the difference: the two differ by about +the same factor on every P-256 row, and nothing above the field -- a wider +window, a different addition formula, another field representation -- recovers +a useful part of it. -Links ------ +The AArch64 rows are better than the x86-64 ones because of where a field +multiplication's latency goes. It ends in a carry chain the width of the +number, and the curve arithmetic has independent products that could cover +that chain -- but only if the compiler inlines the reduction instead of +calling it, since a call is a fence. Asking it to costs code and pays where +there are registers enough to hold two chains at once: a quarter on AArch64, +where there are thirty-one, and nothing on x86-64, where there are fifteen and +the same request makes it slower. So x86-64 is left to the compiler's own +judgement and stays at 0.3. -* [ChaCha](http://cr.yp.to/chacha.html) -* [ChaCha-test-vectors](https://github.com/secworks/chacha_testvectors.git) -* [Poly1305](http://cr.yp.to/mac.html) -* [Poly1305-test-vectors](http://tools.ietf.org/html/draft-nir-cfrg-chacha20-poly1305-06#page-12) -* [Salsa](http://cr.yp.to/snuffle.html) -* [Salsa128-test-vectors](https://github.com/alexwebr/salsa20/blob/master/test_vectors.128) -* [Salsa256-test-vectors](https://github.com/alexwebr/salsa20/blob/master/test_vectors.256) -* [XSalsa](https://cr.yp.to/snuffle/xsalsa-20081128.pdf) -* [PBKDF2](http://tools.ietf.org/html/rfc2898) -* [PBKDF2-test-vectors](http://www.ietf.org/rfc/rfc6070.txt) -* [Scrypt](http://www.tarsnap.com/scrypt.html) -* [Curve25519](http://cr.yp.to/ecdh.html) -* [Ed25519](http://ed25519.cr.yp.to/papers.html) -* [Ed448-Goldilocks](http://ed448goldilocks.sourceforge.net/) -* [EdDSA-test-vectors](http://www.ietf.org/rfc/rfc8032.txt) -* [AFIS](http://clemens.endorphin.org/cryptography) +*RSA signing.* 2.0.0 is slower than 1.1.5 here on purpose. Its modular +exponentiation no longer indexes a table with the bits of the exponent, and +hiding the exponent is what the difference buys. What is left of the gap +against OpenSSL is the Montgomery multiplication, which is assembly there and +C here. +*The AVX-512 instructions.* Neither machine above has them. On one that does +-- an EPYC 9V74, measured the same way -- OpenSSL uses them for AES-GCM and +ChaCha20 and reaches 12003 and 3789 MB/s, against 4745 and 2372 for crypton, +whose vendored assembly is generated without them. Those ratios are 0.40 and +0.63 rather than 0.97 and 1.00. Nothing else in either table moves by more +than a few percent between the two processors. + +One row wants a word of its own: crypton's `Ed25519.sign` derives the public +key from the secret key every time it signs, so that a caller who passes a +public key that does not match cannot be made to leak the private one. That +costs a second scalar multiplication, which OpenSSL's signing does not pay. + +SHA-1 is in the tables because a number of protocols and file formats still +ask for it, not because it is a good choice for anything new. The algorithms +that nothing should ask for any more -- MD5, 3DES, RC4, CBC mode -- are left +out. diff --git a/benchs/Bench.hs b/benchs/Bench.hs --- a/benchs/Bench.hs +++ b/benchs/Bench.hs @@ -5,7 +5,7 @@ module Main where -import Gauge.Main +import Test.Tasty.Bench import Crypto.Cipher.AES import qualified Crypto.Cipher.AESGCMSIV as AESGCMSIV @@ -386,19 +386,22 @@ , bgroup "Ed25519" benchEd25519 ] where + -- the environment is a key pair and a signature that the benchmarked + -- operation only reads, so building it once outside the timed region is + -- the same measurement gauge's perBatchEnv made benchGen prx alg = - [ bench "sign" $ perBatchEnv (genEnv prx alg) (run_gen_sign prx) - , bench "verify" $ perBatchEnv (genEnv prx alg) (run_gen_verify prx) + [ env (genEnv prx alg) $ bench "sign" . nfIO . run_gen_sign prx + , env (genEnv prx alg) $ bench "verify" . nfIO . run_gen_verify prx ] benchGenEd25519 = benchGen (Just Curve_Edwards25519) SHA512 benchEd25519 = - [ bench "sign" $ perBatchEnv ed25519Env run_ed25519_sign - , bench "verify" $ perBatchEnv ed25519Env run_ed25519_verify + [ env ed25519Env $ bench "sign" . nfIO . run_ed25519_sign + , env ed25519Env $ bench "verify" . nfIO . run_ed25519_verify ] msg = B.empty -- empty message = worst-case scenario showing API overhead - genEnv prx alg _ = do + genEnv prx alg = do sec <- EdDSA.generateSecretKey prx let pub = EdDSA.toPublic prx alg sec sig = EdDSA.sign prx sec pub msg @@ -408,7 +411,7 @@ run_gen_verify prx (_, pub, sig) = return (EdDSA.verify prx pub msg sig) - ed25519Env _ = do + ed25519Env = do sec <- Ed25519.generateSecretKey let pub = Ed25519.toPublic sec sig = Ed25519.sign sec pub msg diff --git a/benchs/Number/F2m.hs b/benchs/Number/F2m.hs --- a/benchs/Number/F2m.hs +++ b/benchs/Number/F2m.hs @@ -2,20 +2,23 @@ module Number.F2m (benchF2m) where -import Gauge.Main import System.Random +import Test.Tasty.Bench import Crypto.Number.Basic (log2) import Crypto.Number.F2m genInteger :: Int -> Int -> Integer -genInteger salt bits = - head - . dropWhile ((< bits) . log2) - . scanl (\a r -> a * 2 ^ (31 :: Int) + abs r) 0 - . randoms - . mkStdGen - $ salt + bits +genInteger salt bits = case candidates of + x : _ -> x + [] -> error "genInteger: the stream of candidates ran out" + where + candidates = + dropWhile ((< bits) . log2) + . scanl (\a r -> a * 2 ^ (31 :: Int) + abs r) 0 + . randoms + . mkStdGen + $ salt + bits benchMod :: Int -> Benchmark benchMod bits = bench (show bits) $ nf (modF2m m) a diff --git a/cbits/aes/armv8.c b/cbits/aes/armv8.c new file mode 100644 --- /dev/null +++ b/cbits/aes/armv8.c @@ -0,0 +1,398 @@ +/* + * AES using the ARMv8-A Cryptographic Extensions. + * + * The generic code in aes/generic.c is S-box table driven, which on AArch64 + * was the only thing available: crypton_aes.c only ever swapped in the AES-NI + * implementation, and that is gated on x86. This provides the AArch64 + * equivalent. + * + * The key schedule is laid out exactly as x86ni.c lays it out, because + * crypton_aes.c leaves some operations -- OCB and CCM -- pointing at the + * generic implementation even once the accelerated table is installed, and + * those read the forward schedule. So: the forward round keys k[0..nbr] + * first, in the order crypton_aes_generic_init writes them, then + * InvMixColumns(k[nbr-1]) down to InvMixColumns(k[1]) for decryption. The two + * ends of the decryption schedule, k[nbr] and k[0], are read back out of the + * forward half rather than stored twice, which is what makes AES-256 fit in + * the 16*14*2 bytes of aes_key.data. + */ + +#include +#include +#include +#if defined(__linux__) +#include +#include +#endif +#include "crypton_aes.h" +#include "crypton_bitfn.h" + +/* + * The AES and PMULL instructions are extensions, so a translation unit + * compiled for baseline ARMv8-A may not use them. Mark the functions that do, + * the way cbits/aes/x86ni.h marks their x86 counterparts, rather than raising + * -march for every file in the library: the flag use_target_attributes picks + * between the two, and with it set -- which is the default -- nothing else + * enables the extensions, so without these the file does not compile at all on + * a toolchain whose baseline lacks them. Apple's does not lack them, which is + * why only Linux noticed. + * + * "+crypto" rather than "crypto": GCC rejects the latter. + */ +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_ARMV8_CRYPTO __attribute__((target("+crypto"))) +#else +#define TARGET_ARMV8_CRYPTO +#endif + +/* forward round keys: nbr + 1 of them, written by the generic key expansion */ +#define FWD(key) ((const uint8_t *) (key)->data) +/* InvMixColumns(k[nbr-1]) .. InvMixColumns(k[1]): nbr - 1 of them */ +#define INV(key) (((const uint8_t *) (key)->data) + 16 * ((key)->nbr + 1)) + +/* + * The key schedule of FIPS 197 5.2, with the S-box the schedule needs coming + * from the instructions rather than a table in memory. + * + * AArch64 has no counterpart to x86's AESKEYGENASSIST, but AESE is + * AddRoundKey, SubBytes and ShiftRows together, so against a zero key it is + * SubBytes and ShiftRows. Give it a word in all four columns and ShiftRows + * only moves identical bytes between them, which leaves every column holding + * SubWord of that word. RotWord is then a byte rotation, and on a register + * whose four words are equal a rotation of the whole register by one byte + * rotates each word. + * + * The words stay in vector registers throughout: a word moved to a general + * register and back costs more than the instruction it is moved for. + * + * The exposure this removes is a small one -- sixteen lookups at addresses + * derived from the key, once per key, against the per-block indexing the + * instructions exist to remove -- but a key schedule is the one thing an + * attacker most wants and it costs little to keep it out of the cache. + */ +TARGET_ARMV8_CRYPTO +static uint32x4_t sub_word(uint32x4_t w) +{ + return vreinterpretq_u32_u8( + vaeseq_u8(vreinterpretq_u8_u32(w), vdupq_n_u8(0))); +} + +TARGET_ARMV8_CRYPTO +static uint32x4_t sub_rot_word(uint32x4_t w) +{ + const uint8x16_t s = vreinterpretq_u8_u32(sub_word(w)); + + return vreinterpretq_u32_u8(vextq_u8(s, s, 1)); +} + +TARGET_ARMV8_CRYPTO +void crypton_aes_armv8_init(aes_key *key, uint8_t *origkey, uint8_t size) +{ + /* 2^0 .. 2^9 in GF(2^8), which is as far as any key size reaches */ + static const uint32_t rcon[10] = { + 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, 0x1b, 0x36, + }; + uint32_t *w = (uint32_t *) key->data; + uint8_t *inv; + int nk, nw, i; + + switch (size) { + case 16: key->nbr = 10; break; + case 24: key->nbr = 12; break; + case 32: key->nbr = 14; break; + default: return; + } + nk = size / 4; /* words of key */ + nw = 4 * (key->nbr + 1); /* words of schedule */ + + memcpy(w, origkey, size); + for (i = nk; i < nw; i++) { + uint32x4_t t = vld1q_dup_u32(w + i - 1); + + if (i % nk == 0) + t = veorq_u32(sub_rot_word(t), + vdupq_n_u32(rcon[i / nk - 1])); + else if (nk > 6 && i % nk == 4) + t = sub_word(t); + vst1q_lane_u32(w + i, veorq_u32(t, vld1q_dup_u32(w + i - nk)), 0); + } + + /* and the inverted round keys the decryption modes read */ + inv = ((uint8_t *) key->data) + 16 * (key->nbr + 1); + for (i = 1; i < key->nbr; i++) { + uint8x16_t rk = + vld1q_u8(((const uint8_t *) key->data) + 16 * (key->nbr - i)); + vst1q_u8(inv + 16 * (i - 1), vaesimcq_u8(rk)); + } +} + +/* + * Whether the extensions are actually present. + * + * They are mandatory on Apple silicon, and on other AArch64 systems the + * kernel reports them through the auxiliary vector. A system without them + * keeps the generic implementation. + */ +int crypton_aes_armv8_available(void) +{ +#if defined(__APPLE__) + return 1; +#elif defined(__linux__) + return (getauxval(AT_HWCAP) & HWCAP_AES) != 0; +#else + return 0; +#endif +} + +/* + * GHASH using PMULL, the AArch64 counterpart to PCLMULQDQ. + * + * This is a transliteration of gfmul_pclmuldq in x86ni.c rather than a fresh + * formulation: that code is already pinned by the GCM known-answer tests, and + * every operation it uses has a direct NEON equivalent, so translating it is + * easier to check than reasoning about a new reduction from scratch. + * + * _mm_shuffle_epi8 with a reversing mask -> vrev64q_u8 then vextq_u8 + * _mm_clmulepi64_si128 -> vmull_p64 / vmull_high_p64 + * _mm_slli_si128 / _mm_srli_si128 -> vextq_u8 against zero + * _mm_slli_epi32 / _mm_srli_epi32 -> vshlq_n_u32 / vshrq_n_u32 + */ + +/* reverse all 16 bytes */ +TARGET_ARMV8_CRYPTO +static inline uint8x16_t bswap128(uint8x16_t v) +{ + return vextq_u8(vrev64q_u8(v), vrev64q_u8(v), 8); +} + +/* shift the whole register left by n bytes, as _mm_slli_si128 does */ +#define SHIFT_LEFT_BYTES(v, n) vextq_u8(vdupq_n_u8(0), (v), 16 - (n)) +/* and right, as _mm_srli_si128 does */ +#define SHIFT_RIGHT_BYTES(v, n) vextq_u8((v), vdupq_n_u8(0), (n)) + +#define SHL32(v, n) vreinterpretq_u8_u32(vshlq_n_u32(vreinterpretq_u32_u8(v), (n))) +#define SHR32(v, n) vreinterpretq_u8_u32(vshrq_n_u32(vreinterpretq_u32_u8(v), (n))) + +TARGET_ARMV8_CRYPTO +static inline uint8x16_t clmul_ll(uint8x16_t a, uint8x16_t b) +{ + return vreinterpretq_u8_p128(vmull_p64( + (poly64_t) vgetq_lane_u64(vreinterpretq_u64_u8(a), 0), + (poly64_t) vgetq_lane_u64(vreinterpretq_u64_u8(b), 0))); +} + +TARGET_ARMV8_CRYPTO +static inline uint8x16_t clmul_lh(uint8x16_t a, uint8x16_t b) +{ + return vreinterpretq_u8_p128(vmull_p64( + (poly64_t) vgetq_lane_u64(vreinterpretq_u64_u8(a), 0), + (poly64_t) vgetq_lane_u64(vreinterpretq_u64_u8(b), 1))); +} + +TARGET_ARMV8_CRYPTO +static inline uint8x16_t clmul_hl(uint8x16_t a, uint8x16_t b) +{ + return vreinterpretq_u8_p128(vmull_p64( + (poly64_t) vgetq_lane_u64(vreinterpretq_u64_u8(a), 1), + (poly64_t) vgetq_lane_u64(vreinterpretq_u64_u8(b), 0))); +} + +TARGET_ARMV8_CRYPTO +static inline uint8x16_t clmul_hh(uint8x16_t a, uint8x16_t b) +{ + return vreinterpretq_u8_p128(vmull_high_p64( + vreinterpretq_p64_u8(a), vreinterpretq_p64_u8(b))); +} + +/* + * The 256-bit carry-less product of a (normal byte order) and b (already + * reversed, as it sits in the table), before the reflection fixup and the + * reduction. Split out from the reduction because both of those are linear + * over XOR: several products can be added together and fixed up just once, + * which is what gf_mul4 below does. + */ +TARGET_ARMV8_CRYPTO +static inline void clmul_pmull(uint8x16_t a, uint8x16_t b, + uint8x16_t *lo, uint8x16_t *hi) +{ + uint8x16_t t3, t4, t5, t6; + + a = bswap128(a); + + t3 = clmul_ll(a, b); + t4 = clmul_lh(a, b); + t5 = clmul_hl(a, b); + t6 = clmul_hh(a, b); + + t4 = veorq_u8(t4, t5); + t5 = SHIFT_LEFT_BYTES(t4, 8); + t4 = SHIFT_RIGHT_BYTES(t4, 8); + + *lo = veorq_u8(t3, t5); + *hi = veorq_u8(t6, t4); +} + +/* Shift the 256-bit product left by one to undo GCM's bit reflection, then + * reduce modulo the GCM polynomial. This is the expensive half. */ +TARGET_ARMV8_CRYPTO +static inline uint8x16_t gfred_pmull(uint8x16_t t3, uint8x16_t t6) +{ + uint8x16_t t2, t4, t5, t7, t8, t9; + + t7 = SHR32(t3, 31); + t8 = SHR32(t6, 31); + t3 = SHL32(t3, 1); + t6 = SHL32(t6, 1); + + t9 = SHIFT_RIGHT_BYTES(t7, 12); + t8 = SHIFT_LEFT_BYTES(t8, 4); + t7 = SHIFT_LEFT_BYTES(t7, 4); + t3 = vorrq_u8(t3, t7); + t6 = vorrq_u8(t6, t8); + t6 = vorrq_u8(t6, t9); + + t7 = SHL32(t3, 31); + t8 = SHL32(t3, 30); + t9 = SHL32(t3, 25); + + t7 = veorq_u8(t7, t8); + t7 = veorq_u8(t7, t9); + t8 = SHIFT_RIGHT_BYTES(t7, 4); + t7 = SHIFT_LEFT_BYTES(t7, 12); + t3 = veorq_u8(t3, t7); + + t2 = SHR32(t3, 1); + t4 = SHR32(t3, 2); + t5 = SHR32(t3, 7); + t2 = veorq_u8(t2, t4); + t2 = veorq_u8(t2, t5); + t2 = veorq_u8(t2, t8); + t3 = veorq_u8(t3, t2); + t6 = veorq_u8(t6, t3); + + return bswap128(t6); +} + +TARGET_ARMV8_CRYPTO +static uint8x16_t gfmul_pmull(uint8x16_t a, const uint8_t *htable) +{ + uint8x16_t lo, hi; + + clmul_pmull(a, vld1q_u8(htable), &lo, &hi); + return gfred_pmull(lo, hi); +} + +/* + * With PMULL there is no 4-bit table to fill: H goes in at index 0, byte + * reversed, so that gfmul_pmull does not have to swap it every time. This + * mirrors crypton_aesni_hinit_pclmul. + * + * Indices 1..7 get H^2 .. H^8, which is what lets a group of blocks fold + * into one reduction: gf_mul4 uses the first four, the GCM loop all eight. + * The table has sixteen slots, so they are free. + */ +TARGET_ARMV8_CRYPTO +void crypton_aes_armv8_hinit_pmull(block128 *htable, const block128 *h) +{ + uint8x16_t p; + int i; + + htable[0].q[0] = bitfn_swap64(h->q[1]); + htable[0].q[1] = bitfn_swap64(h->q[0]); + + p = vld1q_u8((const uint8_t *) h); + for (i = 1; i < 8; i++) { + p = gfmul_pmull(p, (const uint8_t *) &htable[0]); + vst1q_u8((uint8_t *) &htable[i], bswap128(p)); + } +} + +TARGET_ARMV8_CRYPTO +void crypton_aes_armv8_gf_mul_pmull(block128 *a, const block128 *htable) +{ + vst1q_u8((uint8_t *) a, + gfmul_pmull(vld1q_u8((const uint8_t *) a), (const uint8_t *) htable)); +} + +/* + * Four GHASH steps -- ((((a^b0)H ^ b1)H ^ b2)H ^ b3)H -- with a single + * reduction. Expanded that is (a^b0)H^4 ^ b1*H^3 ^ b2*H^2 ^ b3*H, so the + * four products can be summed first and reduced once, which is where the + * time goes. Aggregated reduction, from the Intel GCM paper. + */ +TARGET_ARMV8_CRYPTO +void crypton_aes_armv8_gf_mul4_pmull(block128 *a, const block128 *blocks, + const block128 *htable) +{ + uint8x16_t lo, hi, l, h; + int i; + + clmul_pmull(veorq_u8(vld1q_u8((const uint8_t *) a), + vld1q_u8((const uint8_t *) &blocks[0])), + vld1q_u8((const uint8_t *) &htable[3]), &lo, &hi); + + for (i = 1; i < 4; i++) { + clmul_pmull(vld1q_u8((const uint8_t *) &blocks[i]), + vld1q_u8((const uint8_t *) &htable[3 - i]), &l, &h); + lo = veorq_u8(lo, l); + hi = veorq_u8(hi, h); + } + + vst1q_u8((uint8_t *) a, gfred_pmull(lo, hi)); +} + +int crypton_aes_armv8_pmull_available(void) +{ +#if defined(__APPLE__) + return 1; +#elif defined(__linux__) + return (getauxval(AT_HWCAP) & HWCAP_PMULL) != 0; +#else + return 0; +#endif +} + +/* + * The XTS tweak advances by doubling in GF(2^128), which + * crypton_aes_generic_gf_mulx does through memory. Here it stays in a + * register: shift both halves left by one, carry the low half's top bit into + * the high half, and fold the bit that leaves the top back in as 0x87. The + * block is little-endian, so lane 0 is the low half. + */ +TARGET_ARMV8_CRYPTO +static inline uint8x16_t gfmulx_neon(uint8x16_t v) +{ + const uint64x2_t x = vreinterpretq_u64_u8(v); + const uint64x2_t zero = vdupq_n_u64(0); + const uint64x2_t carry = vshrq_n_u64(x, 63); + /* the low half's carry becomes the high half's bit 0 */ + const uint64x2_t into_hi = vextq_u64(zero, carry, 1); + /* and the high half's becomes all ones, or nothing, in the low half */ + const uint64x2_t out = vsubq_u64(zero, vextq_u64(carry, zero, 1)); + const uint64x2_t poly = vsetq_lane_u64(0x87, zero, 0); + + return vreinterpretq_u8_u64(veorq_u64( + vorrq_u64(vshlq_n_u64(x, 1), into_hi), vandq_u64(out, poly))); +} + +/* + * The modes, generated once per key size. See armv8_impl.c for why the + * round count has to be a compile-time constant. + */ +#define SIZED(m) m##128 +#define NBR 10 +#include +#undef SIZED +#undef NBR + +#define SIZED(m) m##192 +#define NBR 12 +#include +#undef SIZED +#undef NBR + +#define SIZED(m) m##256 +#define NBR 14 +#include +#undef SIZED +#undef NBR diff --git a/cbits/aes/armv8_impl.c b/cbits/aes/armv8_impl.c new file mode 100644 --- /dev/null +++ b/cbits/aes/armv8_impl.c @@ -0,0 +1,552 @@ +/* + * Included from armv8.c once per key size, with NBR set to the number of + * rounds and SIZED() naming the functions. This mirrors x86ni_impl.c. + * + * Two things here want compile-time constants, and both are worth having. + * With the round count fixed the compiler keeps the round keys scheduled + * instead of reloading them against a count read out of the key. With the + * blocks in flight fixed it interleaves that many independent chains, which + * is what covers the latency of AESE and AESMC -- one block at a time leaves + * the pipeline waiting on itself. On Apple silicon the two together are + * worth about four times a loop that does one block with a round count from + * memory. + * + * The blocks are named by constant index throughout, and every step is + * written out one per block rather than left to a loop over s[i]. Such a + * loop is only as good as the compiler's willingness to unroll it, and GCC + * at -O2 declines: s[] then lives on the stack and each round turns into a + * load and a store, which measured slower than the one-block code this + * replaces. Spelling the steps out costs nothing and leaves nothing to + * decide. + */ + +/* Eight chains is where the return flattens out on the cores measured. */ +#define WAY 8 + +#define EACH1(m) m(0) +#define EACH8(m) m(0) m(1) m(2) m(3) m(4) m(5) m(6) m(7) +/* the blocks after the first; GHASH folds block 0 in with the tag */ +#define EACH7(m) m(1) m(2) m(3) m(4) m(5) m(6) m(7) + +#define LOAD_IN(i) s[i] = vld1q_u8((const uint8_t *) (input + (i))); +#define STORE_OUT(i) vst1q_u8((uint8_t *) (output + (i)), s[i]); + +#define ENC_STEP(i) s[i] = vaesmcq_u8(vaeseq_u8(s[i], k_)); +#define ENC_LAST(i) s[i] = veorq_u8(vaeseq_u8(s[i], k_), l_); +#define DEC_STEP(i) s[i] = vaesimcq_u8(vaesdq_u8(s[i], k_)); +#define DEC_LAST(i) s[i] = veorq_u8(vaesdq_u8(s[i], k_), l_); + +/* Encrypt the blocks EACH names, in place in s[]. rk must be in scope. */ +#define ENC_ROUNDS(EACH) \ + do { \ + int r_; \ + for (r_ = 0; r_ < NBR - 1; r_++) { \ + const uint8x16_t k_ = vld1q_u8(rk + 16 * r_); \ + EACH(ENC_STEP) \ + } \ + { \ + const uint8x16_t k_ = vld1q_u8(rk + 16 * (NBR - 1)); \ + const uint8x16_t l_ = vld1q_u8(rk + 16 * NBR); \ + EACH(ENC_LAST) \ + } \ + } while (0) + +/* + * Decrypt them. fwd and inv must be in scope: the schedule is k[nbr], + * imc(k[nbr-1]) .. imc(k[1]), k[0], so the two ends come from the forward + * keys and the middle from the inverted ones. + */ +#define DEC_ROUNDS(EACH) \ + do { \ + int r_; \ + { \ + const uint8x16_t k_ = vld1q_u8(fwd + 16 * NBR); \ + EACH(DEC_STEP) \ + } \ + for (r_ = 0; r_ < NBR - 2; r_++) { \ + const uint8x16_t k_ = vld1q_u8(inv + 16 * r_); \ + EACH(DEC_STEP) \ + } \ + { \ + const uint8x16_t k_ = vld1q_u8(inv + 16 * (NBR - 2));\ + const uint8x16_t l_ = vld1q_u8(fwd); \ + EACH(DEC_LAST) \ + } \ + } while (0) + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_encrypt_block)(aes_block *output, aes_key *key, aes_block *input) +{ + const uint8_t *rk = FWD(key); + uint8x16_t s[1]; + + EACH1(LOAD_IN); + ENC_ROUNDS(EACH1); + EACH1(STORE_OUT); +} + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_decrypt_block)(aes_block *output, aes_key *key, aes_block *input) +{ + const uint8_t *fwd = FWD(key); + const uint8_t *inv = INV(key); + uint8x16_t s[1]; + + EACH1(LOAD_IN); + DEC_ROUNDS(EACH1); + EACH1(STORE_OUT); +} + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_encrypt_ecb)(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks) +{ + const uint8_t *rk = FWD(key); + uint8x16_t s[WAY]; + + for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) { + EACH8(LOAD_IN); + ENC_ROUNDS(EACH8); + EACH8(STORE_OUT); + } + for (; nb_blocks > 0; nb_blocks--, input++, output++) { + EACH1(LOAD_IN); + ENC_ROUNDS(EACH1); + EACH1(STORE_OUT); + } +} + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_decrypt_ecb)(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks) +{ + const uint8_t *fwd = FWD(key); + const uint8_t *inv = INV(key); + uint8x16_t s[WAY]; + + for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) { + EACH8(LOAD_IN); + DEC_ROUNDS(EACH8); + EACH8(STORE_OUT); + } + for (; nb_blocks > 0; nb_blocks--, input++, output++) { + EACH1(LOAD_IN); + DEC_ROUNDS(EACH1); + EACH1(STORE_OUT); + } +} + +/* CBC encryption chains, so there is nothing to interleave. It still gains + * the round keys staying put. */ +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_encrypt_cbc)(aes_block *output, aes_key *key, aes_block *_iv, aes_block *input, uint32_t nb_blocks) +{ + const uint8_t *rk = FWD(key); + uint8x16_t iv = vld1q_u8((const uint8_t *) _iv); + uint8x16_t s[1]; + + for (; nb_blocks-- > 0; input++, output++) { + s[0] = veorq_u8(iv, vld1q_u8((const uint8_t *) input)); + ENC_ROUNDS(EACH1); + iv = s[0]; + EACH1(STORE_OUT); + } +} + +/* Decryption does not chain: each block is deciphered on its own and then + * XORed with the ciphertext before it, so it interleaves like ECB. */ +/* c[] holds the previous block at index 0 and this group's ciphertext after + * it, so block i is XORed with c[i] and the next group starts from c[WAY]. */ +#define CBC_KEEP(i) c[(i) + 1] = s[i]; +#define CBC_XOR(i) vst1q_u8((uint8_t *) (output + (i)), veorq_u8(s[i], c[i])); + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_decrypt_cbc)(aes_block *output, aes_key *key, aes_block *_iv, aes_block *input, uint32_t nb_blocks) +{ + const uint8_t *fwd = FWD(key); + const uint8_t *inv = INV(key); + uint8x16_t iv = vld1q_u8((const uint8_t *) _iv); + uint8x16_t s[WAY], c[WAY + 1]; + + for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) { + EACH8(LOAD_IN); + c[0] = iv; + EACH8(CBC_KEEP); + DEC_ROUNDS(EACH8); + EACH8(CBC_XOR); + iv = c[WAY]; + } + for (; nb_blocks > 0; nb_blocks--, input++, output++) { + EACH1(LOAD_IN); + c[1] = s[0]; + DEC_ROUNDS(EACH1); + vst1q_u8((uint8_t *) output, veorq_u8(s[0], iv)); + iv = c[1]; + } +} + +/* + * CTR counts the whole 128 bits big-endian, with the carry crossing the + * halves. The arithmetic is kept identical to + * crypton_aes_generic_encrypt_ctr, which also leaves the caller's IV alone. + */ +#define CTR_SET(i) s[i] = vreinterpretq_u8_u64(vsetq_lane_u64(cpu_to_be64(lo + (i)), base, 1)); +#define CTR_XOR(i) vst1q_u8(output + 16 * (i), \ + veorq_u8(s[i], vld1q_u8(input + 16 * (i)))); + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_encrypt_ctr)(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len) +{ + const uint8_t *rk = FWD(key); + uint32_t nb_blocks = len / 16; + uint32_t remaining = len % 16; + aes_block ctr; + uint8x16_t s[WAY]; + uint32_t i; + + block128_copy(&ctr, iv); + + /* + * The counter goes through memory only when its low half is about to + * wrap. Otherwise it stays in registers: the top eight bytes do not + * change and the bottom eight are one add away. That matters -- with + * a store and a reload for every block, CTR ran at the same speed for + * 128-bit and 256-bit keys, which is the giveaway that the cipher was + * not what it was waiting for. + */ + for (; nb_blocks >= WAY; nb_blocks -= WAY, input += 16 * WAY, output += 16 * WAY) { + uint64_t lo = be64_to_cpu(ctr.q[1]); + + if (lo + (WAY - 1) < lo) { + /* a block in this group carries into the top half; + * let the scalar increment deal with it */ + for (i = 0; i < WAY; i++, block128_inc_be(&ctr)) + s[i] = vld1q_u8((const uint8_t *) &ctr); + } else { + const uint64x2_t base = + vreinterpretq_u64_u8(vld1q_u8((const uint8_t *) &ctr)); + + EACH8(CTR_SET); + + /* no block above needed a carry, but the counter left + * for the next group still can */ + ctr.q[1] = cpu_to_be64(lo + WAY); + if (lo + WAY < lo) + ctr.q[0] = cpu_to_be64(be64_to_cpu(ctr.q[0]) + 1); + } + ENC_ROUNDS(EACH8); + EACH8(CTR_XOR); + } + for (; nb_blocks > 0; nb_blocks--, input += 16, output += 16) { + s[0] = vld1q_u8((const uint8_t *) &ctr); + block128_inc_be(&ctr); + ENC_ROUNDS(EACH1); + vst1q_u8(output, veorq_u8(s[0], vld1q_u8(input))); + } + if (remaining) { + aes_block o; + + s[0] = vld1q_u8((const uint8_t *) &ctr); + ENC_ROUNDS(EACH1); + vst1q_u8((uint8_t *) &o, s[0]); + for (i = 0; i < remaining; i++) + output[i] = o.b[i] ^ input[i]; + } +} + + +/* + * GCM, rather than the generic loop calling the block function once per + * block through the branch table. Eight counter blocks go through the + * rounds together, and their GHASH folds into a single reduction with + * H^8 .. H^1, so a group costs one reduction instead of eight. The tag + * and the counter stay in registers across the whole run. + * + * GCM's counter is the low 32 bits only and wraps there, so unlike CTR + * there is no carry to chase: the top twelve bytes never move. + */ +#define GCM_CTR(i) s[i] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c + 1 + (i)), base, 3)); +#define GCM_ENC(i) { const uint8x16_t m_ = vld1q_u8(input + 16 * (i)); \ + s[i] = veorq_u8(s[i], m_); \ + vst1q_u8(output + 16 * (i), s[i]); } +#define GCM_DEC(i) { const uint8x16_t m_ = vld1q_u8(input + 16 * (i)); \ + vst1q_u8(output + 16 * (i), veorq_u8(s[i], m_)); \ + s[i] = m_; } +#define GCM_GHASH(i) { uint8x16_t l_, h_; \ + clmul_pmull(s[i], vld1q_u8((const uint8_t *) &ht[WAY - 1 - (i)]), \ + &l_, &h_); \ + glo = veorq_u8(glo, l_); ghi = veorq_u8(ghi, h_); } + +/* the eight blocks now in s[] are the ciphertext; fold them into the tag */ +#define GCM_FOLD() \ + do { \ + uint8x16_t glo, ghi; \ + clmul_pmull(veorq_u8(tag, s[0]), \ + vld1q_u8((const uint8_t *) &ht[WAY - 1]), \ + &glo, &ghi); \ + EACH7(GCM_GHASH) \ + tag = gfred_pmull(glo, ghi); \ + } while (0) + +#define GCM_PROLOGUE \ + const uint8_t *rk = FWD(key); \ + const block128 *ht = gcm->htable; \ + uint8x16_t s[WAY]; \ + uint8x16_t tag = vld1q_u8((const uint8_t *) &gcm->tag); \ + uint32_t c = be32_to_cpu(gcm->civ.d[3]); \ + uint32x4_t base = vreinterpretq_u32_u8(vld1q_u8((const uint8_t *) &gcm->civ)) + +/* one block, for what is left after the last group of eight */ +#define GCM_ONE(load_m, store_c, ghash_of) \ + do { \ + const uint8x16_t m_ = (load_m); \ + c++; \ + s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3)); \ + ENC_ROUNDS(EACH1); \ + s[0] = veorq_u8(s[0], m_); \ + (store_c); \ + tag = gfmul_pmull(veorq_u8(tag, (ghash_of)), (const uint8_t *) ht); \ + } while (0) + +#define GCM_EPILOGUE \ + do { \ + gcm->civ.d[3] = cpu_to_be32(c); \ + vst1q_u8((uint8_t *) &gcm->tag, tag); \ + } while (0) + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_gcm_encrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length) +{ + GCM_PROLOGUE; + uint32_t i; + + gcm->length_input += length; + + for (; length >= 16 * WAY; input += 16 * WAY, output += 16 * WAY, length -= 16 * WAY) { + EACH8(GCM_CTR); + c += WAY; + ENC_ROUNDS(EACH8); + EACH8(GCM_ENC); + GCM_FOLD(); + } + for (; length >= 16; input += 16, output += 16, length -= 16) { + GCM_ONE(vld1q_u8(input), vst1q_u8(output, s[0]), s[0]); + } + if (length) { + aes_block m, o; + + block128_zero(&m); + block128_copy_bytes(&m, input, length); + c++; + s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3)); + ENC_ROUNDS(EACH1); + s[0] = veorq_u8(s[0], vld1q_u8((const uint8_t *) &m)); + vst1q_u8((uint8_t *) &o, s[0]); + block128_zero(&m); + for (i = 0; i < length; i++) + output[i] = m.b[i] = o.b[i]; + tag = gfmul_pmull(veorq_u8(tag, vld1q_u8((const uint8_t *) &m)), + (const uint8_t *) ht); + } + GCM_EPILOGUE; +} + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_gcm_decrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length) +{ + GCM_PROLOGUE; + uint32_t i; + + gcm->length_input += length; + + for (; length >= 16 * WAY; input += 16 * WAY, output += 16 * WAY, length -= 16 * WAY) { + EACH8(GCM_CTR); + c += WAY; + ENC_ROUNDS(EACH8); + EACH8(GCM_DEC); + GCM_FOLD(); + } + for (; length >= 16; input += 16, output += 16, length -= 16) { + const uint8x16_t ct = vld1q_u8(input); + + GCM_ONE(ct, vst1q_u8(output, s[0]), ct); + } + if (length) { + aes_block m, o; + + block128_zero(&m); + block128_copy_bytes(&m, input, length); + c++; + s[0] = vreinterpretq_u8_u32(vsetq_lane_u32(cpu_to_be32(c), base, 3)); + ENC_ROUNDS(EACH1); + s[0] = veorq_u8(s[0], vld1q_u8((const uint8_t *) &m)); + vst1q_u8((uint8_t *) &o, s[0]); + for (i = 0; i < length; i++) + output[i] = o.b[i]; + tag = gfmul_pmull(veorq_u8(tag, vld1q_u8((const uint8_t *) &m)), + (const uint8_t *) ht); + } + GCM_EPILOGUE; +} + + +/* + * XTS. The tweak for each block is the one before it doubled, so a group's + * eight tweaks are a short chain that runs while the eight AES chains are in + * flight. The first tweak is the data unit number enciphered under the + * second key; spoint skips that many blocks into the unit. + */ +#define XTS_IN(i) s[i] = veorq_u8(vld1q_u8((const uint8_t *) (input + (i))), t[i]); +#define XTS_OUT(i) vst1q_u8((uint8_t *) (output + (i)), veorq_u8(s[i], t[i])); +/* + * The tweak is kept in general-purpose registers and moved into a vector + * one per block. Doubling it costs three integer operations, and the + * integer units have nothing else to do here, where the vector ones are + * busy with the rounds and the exclusive ors: done in vector registers, + * which is what this did, the eight doublings of a group take about as + * long as the eight blocks of AES they are for. + */ +#define XTS_TWEAK(i) do { \ + t[i] = vreinterpretq_u8_u64( \ + vcombine_u64(vcreate_u64(tlo), vcreate_u64(thi))); \ + { \ + const uint64_t _c = thi >> 63; \ + thi = (thi << 1) | (tlo >> 63); \ + tlo = (tlo << 1) ^ (_c ? 0x87 : 0); \ + } \ +} while (0); +/* + * The group after this one's. Doubling is a chain -- each tweak waits for + * the one before it -- and eight of them in front of the rounds that want + * them is time in which nothing else happens, which on a processor whose + * AES is this fast is most of the block. Worked out a group early they + * have nothing to wait for and go through the rounds of the group before, + * which do not want the same units. There are registers enough here for + * both groups at once. + */ +#define XTS_TWEAK_NEXT(i) do { \ + tn[i] = vreinterpretq_u8_u64( \ + vcombine_u64(vcreate_u64(tlo), vcreate_u64(thi))); \ + { \ + const uint64_t _c = thi >> 63; \ + thi = (thi << 1) | (tlo >> 63); \ + tlo = (tlo << 1) ^ (_c ? 0x87 : 0); \ + } \ +} while (0); +#define XTS_TWEAK_ROLL(i) do { t[i] = tn[i]; } while (0); + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_encrypt_xts)(aes_block *output, aes_key *key, aes_key *key2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks) +{ + const uint8_t *rk = FWD(key); + uint8x16_t s[WAY], t[WAY], tn[WAY]; + uint64_t tlo, thi; + + { + aes_block first; + + SIZED(crypton_aes_armv8_encrypt_block)(&first, key2, dataunit); + tlo = first.q[0]; + thi = first.q[1]; + } + while (spoint-- > 0) { + const uint64_t c = thi >> 63; + + thi = (thi << 1) | (tlo >> 63); + tlo = (tlo << 1) ^ (c ? 0x87 : 0); + } + + EACH8(XTS_TWEAK); + for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) { + EACH8(XTS_IN); + EACH8(XTS_TWEAK_NEXT); + ENC_ROUNDS(EACH8); + EACH8(XTS_OUT); + EACH8(XTS_TWEAK_ROLL); + } + /* the group that was made ready and not used */ + { + const uint64x2_t back = vreinterpretq_u64_u8(t[0]); + + tlo = vgetq_lane_u64(back, 0); + thi = vgetq_lane_u64(back, 1); + } + for (; nb_blocks > 0; nb_blocks--, input++, output++) { + EACH1(XTS_TWEAK); + EACH1(XTS_IN); + ENC_ROUNDS(EACH1); + EACH1(XTS_OUT); + } +} + +TARGET_ARMV8_CRYPTO +void SIZED(crypton_aes_armv8_decrypt_xts)(aes_block *output, aes_key *key, aes_key *key2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks) +{ + const uint8_t *fwd = FWD(key); + const uint8_t *inv = INV(key); + uint8x16_t s[WAY], t[WAY], tn[WAY]; + uint64_t tlo, thi; + + { + aes_block first; + + /* the tweak is always enciphered, whichever way the data goes */ + SIZED(crypton_aes_armv8_encrypt_block)(&first, key2, dataunit); + tlo = first.q[0]; + thi = first.q[1]; + } + while (spoint-- > 0) { + const uint64_t c = thi >> 63; + + thi = (thi << 1) | (tlo >> 63); + tlo = (tlo << 1) ^ (c ? 0x87 : 0); + } + + EACH8(XTS_TWEAK); + for (; nb_blocks >= WAY; nb_blocks -= WAY, input += WAY, output += WAY) { + EACH8(XTS_IN); + EACH8(XTS_TWEAK_NEXT); + DEC_ROUNDS(EACH8); + EACH8(XTS_OUT); + EACH8(XTS_TWEAK_ROLL); + } + /* the group that was made ready and not used */ + { + const uint64x2_t back = vreinterpretq_u64_u8(t[0]); + + tlo = vgetq_lane_u64(back, 0); + thi = vgetq_lane_u64(back, 1); + } + for (; nb_blocks > 0; nb_blocks--, input++, output++) { + EACH1(XTS_TWEAK); + EACH1(XTS_IN); + DEC_ROUNDS(EACH1); + EACH1(XTS_OUT); + } +} + +#undef WAY +#undef EACH1 +#undef EACH7 +#undef EACH8 +#undef LOAD_IN +#undef STORE_OUT +#undef ENC_STEP +#undef ENC_LAST +#undef DEC_STEP +#undef DEC_LAST +#undef ENC_ROUNDS +#undef DEC_ROUNDS +#undef CBC_KEEP +#undef CBC_XOR +#undef CTR_SET +#undef CTR_XOR +#undef XTS_IN +#undef XTS_OUT +#undef XTS_TWEAK +#undef GCM_CTR +#undef GCM_ENC +#undef GCM_DEC +#undef GCM_GHASH +#undef GCM_FOLD +#undef GCM_PROLOGUE +#undef GCM_ONE +#undef GCM_EPILOGUE diff --git a/cbits/aes/gcm_x86_asm.c b/cbits/aes/gcm_x86_asm.c new file mode 100644 --- /dev/null +++ b/cbits/aes/gcm_x86_asm.c @@ -0,0 +1,256 @@ +/* + * Copyright (c) 2026 Kazu Yamamoto + * + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the author nor the names of his contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + * + * What the stitched AES-GCM assembly in cbits/asm needs in order to be + * called: the two pieces of state it reads are laid out the way OpenSSL + * lays them out, which is not the way crypton does, and neither is worth + * changing the rest of the library for. Both are built here, per message, + * from the key schedule and the H that crypton already has. + */ + +#include "crypton_cpu.h" + +#ifdef WITH_X86_GCM_ASM + +#include +#include +#include +#include +#include +#include + +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_PCLMUL __attribute__((target("sse4.1,pclmul"))) +#else +#define TARGET_PCLMUL +#endif + +#define ALIGNMENT(n) __attribute__((aligned(n))) + +/* + * cbits/asm/aesni-gcm-x86_64-*.S. Both answer how many bytes they got + * through, which is a multiple of six blocks and is zero if the message is + * shorter than they are willing to start on. + */ +size_t crypton_gcm_asm_encrypt(const void *in, void *out, size_t len, + const void *key, uint8_t ivec[16], void *Xi); +size_t crypton_gcm_asm_decrypt(const void *in, void *out, size_t len, + const void *key, uint8_t ivec[16], void *Xi); + +/* + * The key schedule as the assembly reads it: the encryption round keys, + * and at offset 240 the number of rounds less one, which is the count + * OpenSSL's AES-NI key setup leaves there -- 9, 11 and 13 -- and what the + * assembly compares against to tell the three key sizes apart. + */ +struct asm_key { + uint8_t rd_key[240]; + uint32_t rounds; +}; + +/* + * The assembly reads the running tag from the front of this and the powers + * of H from 32 bytes in, which is where they sit in OpenSSL's GCM context + * -- the 16 bytes between them hold H itself there and nothing here. + * Powers up to the sixth are used, since the loop takes six blocks at a + * time, and each pair of them is followed by the halves the Karatsuba + * multiplication would otherwise have to add up again. + */ +struct asm_gcm { + block128 xi; + block128 unused; + block128 htable[9]; +}; + +/* + * H, and every power of it, is kept shifted up by one bit: GCM numbers the + * bits of a field element the other way round from the way the carry-less + * multiply does, and pre-shifting the operand is what saves the correction + * after each multiply. The bit that falls off the top is the one that the + * polynomial reduces. + */ +TARGET_PCLMUL +static __m128i twist(__m128i h) +{ + const __m128i poly = _mm_set_epi64x(0xc200000000000000ULL, 1); + __m128i carried = _mm_slli_si128(_mm_srli_epi64(h, 63), 8); + __m128i top = _mm_shuffle_epi32(h, 0xff); + __m128i reduce = _mm_cmpgt_epi32(_mm_setzero_si128(), top); + + h = _mm_or_si128(_mm_slli_epi64(h, 1), carried); + return _mm_xor_si128(h, _mm_and_si128(reduce, poly)); +} + +/* the two halves of a value added together, which is the term Karatsuba + * needs and which does not depend on what it is multiplied by */ +TARGET_PCLMUL +static __m128i fold(__m128i a) +{ + return _mm_xor_si128(a, _mm_unpackhi_epi64(a, a)); +} + +/* + * The table the assembly reads: the first six powers of H, each shifted up + * by one, and after each pair the two halves of both of them added + * together, which is the term the Karatsuba multiplication would otherwise + * work out for itself every time. + * + * The powers are not computed here. crypton's own table already holds + * H^1 to H^8, in the byte order the multiply wants and unshifted, so + * twisting each one is the whole of the work -- which is why this is worth + * doing per message rather than keeping a second table in the context. + */ +TARGET_PCLMUL +static void init_htable(struct asm_gcm *st, const aes_gcm *gcm) +{ + int i; + + for (i = 0; i < 3; i++) { + __m128i odd = twist(_mm_loadu_si128( + (const __m128i *) &gcm->htable[2 * i])); + __m128i even = twist(_mm_loadu_si128( + (const __m128i *) &gcm->htable[2 * i + 1])); + + _mm_storeu_si128((__m128i *) &st->htable[3 * i + 0], odd); + _mm_storeu_si128((__m128i *) &st->htable[3 * i + 1], even); + _mm_storeu_si128((__m128i *) &st->htable[3 * i + 2], + _mm_unpacklo_epi64(fold(odd), fold(even))); + } +} + +/* + * The counter block, whose bottom 32 bits are what counts, as GCM has it. + * crypton keeps the value it last used and the assembly wants the one it + * is to use next, so this steps between the two conventions at each end. + */ +static void ctr32_bump(uint8_t ivec[16], uint32_t delta) +{ + uint32_t c = ((uint32_t) ivec[12] << 24) | ((uint32_t) ivec[13] << 16) + | ((uint32_t) ivec[14] << 8) | (uint32_t) ivec[15]; + + c += delta; + ivec[12] = (uint8_t) (c >> 24); + ivec[13] = (uint8_t) (c >> 16); + ivec[14] = (uint8_t) (c >> 8); + ivec[15] = (uint8_t) c; +} + +/* + * How much of the message to hand over. The assembly works in groups of + * six blocks, and what it leaves behind goes to a loop that works in groups + * of eight and then one at a time. Handing over every group it could take + * often leaves two or four blocks to go through one at a time, which at a + * multiply apiece costs more than the three groups it takes to line the + * remainder up on eight. So the length is rounded down to whichever number + * of six-block groups within reach leaves the least behind, modulo eight. + */ +static uint32_t handover(uint32_t blocks) +{ + uint32_t groups = blocks / 6; + uint32_t best = groups; + uint32_t least = (blocks - 6 * groups) % 8; + uint32_t i; + + for (i = 1; i <= 3 && groups >= i; i++) { + uint32_t left = (blocks - 6 * (groups - i)) % 8; + + if (left < least) { + least = left; + best = groups - i; + } + } + return best * 6 * 16; +} + +int crypton_gcm_asm_usable(void) +{ + static int resolved = 0; + static int usable = 0; + + if (!resolved) { + const uint32_t need = CRYPTON_X86_AVX | CRYPTON_X86_MOVBE + | CRYPTON_X86_PCLMUL; + + usable = (crypton_x86_simd_features() & need) == need; + resolved = 1; + } + return usable; +} + +TARGET_PCLMUL +static uint32_t bulk(int encrypt, uint8_t *output, aes_gcm *gcm, aes_key *key, + const uint8_t *input, uint32_t length) +{ + struct asm_gcm st ALIGNMENT(16); + struct asm_key k ALIGNMENT(16); + uint8_t ivec[16] ALIGNMENT(16); + uint32_t hand; + size_t done; + + if (!crypton_gcm_asm_usable()) + return 0; + + /* below its own minimum the assembly does nothing, so in that case + * give it everything and let it decide */ + hand = handover(length / 16); + if (hand < (encrypt ? 0x60 * 3 : 0x60)) + hand = length; + + memcpy(k.rd_key, key->data, 16 * (size_t) (key->nbr + 1)); + k.rounds = (uint32_t) key->nbr - 1; + memcpy(&st.xi, &gcm->tag, 16); + memcpy(ivec, &gcm->civ, 16); + ctr32_bump(ivec, 1); + init_htable(&st, gcm); + + done = encrypt + ? crypton_gcm_asm_encrypt(input, output, hand, &k, ivec, &st.xi) + : crypton_gcm_asm_decrypt(input, output, hand, &k, ivec, &st.xi); + + if (done > 0) { + ctr32_bump(ivec, 0xffffffff); + memcpy(&gcm->tag, &st.xi, 16); + memcpy(&gcm->civ, ivec, 16); + } + return (uint32_t) done; +} + +uint32_t crypton_gcm_asm_bulk_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, + const uint8_t *input, uint32_t length) +{ + return bulk(1, output, gcm, key, input, length); +} + +uint32_t crypton_gcm_asm_bulk_decrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, + const uint8_t *input, uint32_t length) +{ + return bulk(0, output, gcm, key, input, length); +} + +#endif diff --git a/cbits/aes/gcm_x86_asm.h b/cbits/aes/gcm_x86_asm.h new file mode 100644 --- /dev/null +++ b/cbits/aes/gcm_x86_asm.h @@ -0,0 +1,72 @@ +/* + * Copyright (c) 2026 Kazu Yamamoto + * + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * 3. Neither the name of the author nor the names of his contributors + * may be used to endorse or promote products derived from this software + * without specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHORS OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ +#ifndef CRYPTON_AES_GCM_X86_ASM_H +#define CRYPTON_AES_GCM_X86_ASM_H + +#ifdef WITH_X86_GCM_ASM + +#include +#include + +/* + * How long a message has to be before it is handed over. The assembly + * needs the powers of H in a layout of its own, and what does not fill six + * blocks is left to the loop that would otherwise have taken eight at a + * time, so a short message pays for the setup and for a tail that goes + * through one block at a time. Encryption also spends its first twelve + * blocks in plain counter mode before the stitched loop starts, which is + * why it has to be given a good deal more before it comes out ahead. + * + * Measured on a Haswell-generation x86-64: decryption is ahead from 288 + * bytes up, by 5 to 20 per cent, and below that loses by about as much. + * Encryption between 288 and 1024 bytes is a wash -- it swings either way + * by up to ten per cent depending on how the length divides into groups -- + * and from 1152 bytes it is ahead by 9 per cent or more, reaching 25 to 40 + * per cent once the message is a few kilobytes. + */ +#define GCM_ASM_MIN_BLOCKS_ENC 72 +#define GCM_ASM_MIN_BLOCKS_DEC 18 + +/* whether the processor has what cbits/asm/aesni-gcm-x86_64-*.S needs */ +int crypton_gcm_asm_usable(void); + +/* + * Encrypt or decrypt from the front of the message, hashing as it goes, and + * answer how much was done -- a multiple of 96 bytes, possibly none of it. + * The counter and the running tag in *gcm are brought forward by that much. + */ +uint32_t crypton_gcm_asm_bulk_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, + const uint8_t *input, uint32_t length); +uint32_t crypton_gcm_asm_bulk_decrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, + const uint8_t *input, uint32_t length); + +#endif + +#endif diff --git a/cbits/aes/gf.c b/cbits/aes/gf.c --- a/cbits/aes/gf.c +++ b/cbits/aes/gf.c @@ -144,3 +144,19 @@ block128_cpu_swap_be(a, &b); /* restore BE order when done */ } } + +/* + * Four GHASH steps at once. The generic table-driven multiply has no cheaper + * way to do this than one block at a time; the point of the entry is that the + * PMULL and PCLMUL versions can fold the four products into one reduction, so + * the GCM loops hand over four blocks whenever they have them. + */ +void crypton_aes_generic_gf_mul4(block128 *a, const block128 *blocks, const table_4bit htable) +{ + int i; + + for (i = 0; i < 4; i++) { + block128_xor(a, &blocks[i]); + crypton_aes_generic_gf_mul(a, htable); + } +} diff --git a/cbits/aes/gf.h b/cbits/aes/gf.h --- a/cbits/aes/gf.h +++ b/cbits/aes/gf.h @@ -38,5 +38,6 @@ void crypton_aes_generic_hinit(table_4bit htable, const block128 *h); void crypton_aes_generic_gf_mul(block128 *a, const table_4bit htable); +void crypton_aes_generic_gf_mul4(block128 *a, const block128 *blocks, const table_4bit htable); #endif diff --git a/cbits/aes/x86ni.c b/cbits/aes/x86ni.c --- a/cbits/aes/x86ni.c +++ b/cbits/aes/x86ni.c @@ -38,6 +38,7 @@ #include #include #include +#include #ifdef ARCH_X86 #define ALIGN_UP(addr, size) (((addr) + ((size) - 1)) & (~((size) - 1))) @@ -56,7 +57,23 @@ return _mm_xor_si128(key, keygened); } +/* + * SubWord(RotWord(w)), which is the one part of a key schedule that would + * otherwise want the S-box out of a table. AESKEYGENASSIST computes it for + * the words in lanes 1 and 3 and exclusive-ors the round constant into the + * result; the constant is an immediate, so it is left at zero here and + * applied by the caller, which keeps the 192-bit schedule a loop. + */ TARGET_AESNI +static uint32_t key_sub_rot(uint32_t w) +{ + const __m128i t = + _mm_aeskeygenassist_si128(_mm_setr_epi32(0, (int) w, 0, 0), 0x00); + + return (uint32_t) _mm_cvtsi128_si32(_mm_srli_si128(t, 4)); +} + +TARGET_AESNI static __m128i aes_128_key_expansion_aa(__m128i key, __m128i keygened) { keygened = _mm_shuffle_epi32(keygened, 0xaa); @@ -105,6 +122,34 @@ for (i = 0; i < 20; i++) _mm_storeu_si128(((__m128i *) out) + i, k[i]); break; + case 24: { + /* + * The 192-bit schedule takes six words at a time where a round + * key is four, so it does not fall into 128-bit pieces the way + * the other two do; it is built a word at a time instead. + * Thirteen round keys, then the eleven inverted ones. + */ + static const uint32_t rcon[8] = { + 0x01, 0x02, 0x04, 0x08, 0x10, 0x20, 0x40, 0x80, + }; + uint32_t w[52]; + + memcpy(w, ikey, 24); + for (i = 6; i < 52; i++) { + uint32_t t = w[i - 1]; + + if (i % 6 == 0) + t = key_sub_rot(t) ^ rcon[i / 6 - 1]; + w[i] = w[i - 6] ^ t; + } + memcpy(out, w, sizeof(w)); + + for (i = 1; i < 12; i++) + _mm_storeu_si128(((__m128i *) out) + 12 + i, + _mm_aesimc_si128(_mm_loadu_si128( + ((const __m128i *) w) + (12 - i)))); + break; + } case 32: #define AES_256_key_exp_1(K1, K2, RCON) aes_128_key_expansion_ff(K1, _mm_aeskeygenassist_si128(K2, RCON)) #define AES_256_key_exp_2(K1, K2) aes_128_key_expansion_aa(K1, _mm_aeskeygenassist_si128(K2, 0x00)) @@ -172,36 +217,89 @@ return tag; } +/* Four or eight GHASH steps. The table-driven multiply gains nothing from + * seeing them together; the PCLMUL versions below fold them into one + * reduction. */ +TARGET_AESNI +static __m128i gfmul4_generic(__m128i tag, const table_4bit htable, const __m128i *m) +{ + int i; + + for (i = 0; i < 4; i++) + tag = gfmul_generic(_mm_xor_si128(tag, m[i]), htable); + return tag; +} + +TARGET_AESNI +static __m128i gfmul8_generic(__m128i tag, const table_4bit htable, const __m128i *m) +{ + int i; + + for (i = 0; i < 8; i++) + tag = gfmul_generic(_mm_xor_si128(tag, m[i]), htable); + return tag; +} + #ifdef WITH_PCLMUL __m128i (*crypton_gfmul_branch_ptr)(__m128i a, const table_4bit t) = gfmul_generic; #define gfmul(a,t) ((*crypton_gfmul_branch_ptr)(a,t)) +__m128i (*crypton_gfmul4_branch_ptr)(__m128i a, const table_4bit t, const __m128i *m) = gfmul4_generic; +#define gfmul4(a,t,m) ((*crypton_gfmul4_branch_ptr)(a,t,m)) + +__m128i (*crypton_gfmul8_branch_ptr)(__m128i a, const table_4bit t, const __m128i *m) = gfmul8_generic; +#define gfmul8(a,t,m) ((*crypton_gfmul8_branch_ptr)(a,t,m)) + /* See Intel carry-less-multiplication-instruction-in-gcm-mode-paper.pdf * * Adapted from figure 5, with additional byte swapping so that interface * is simimar to crypton_aes_generic_gf_mul. */ +/* + * The 256-bit carry-less product, before the reflection fixup and the + * reduction. Split out from the reduction because both of those are linear + * over XOR: several products can be added together and fixed up just once, + * which is what gf_mul4 below does. + */ TARGET_AESNI_PCLMUL -static __m128i gfmul_pclmuldq(__m128i a, const table_4bit htable) +static inline void clmul_pclmuldq(__m128i a, __m128i b, __m128i *lo, __m128i *hi) { - __m128i b, tmp2, tmp3, tmp4, tmp5, tmp6, tmp7, tmp8, tmp9; + __m128i tmp3, tmp4, tmp5, tmp6; __m128i bswap_mask = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15); a = _mm_shuffle_epi8(a, bswap_mask); - b = _mm_loadu_si128((__m128i *) htable); + /* + * Karatsuba: the middle term of the product is + * (a0^a1)(b0^b1) ^ a0b0 ^ a1b1, which is one carry-less multiply + * where the direct form needs two. Three PCLMULQDQ rather than + * four, at the cost of a few shuffles and exclusive ors -- worth it + * wherever the multiply is the narrower port, which is every part + * this has been measured on. + */ tmp3 = _mm_clmulepi64_si128(a, b, 0x00); - tmp4 = _mm_clmulepi64_si128(a, b, 0x10); - tmp5 = _mm_clmulepi64_si128(a, b, 0x01); tmp6 = _mm_clmulepi64_si128(a, b, 0x11); + tmp4 = _mm_clmulepi64_si128(_mm_xor_si128(a, _mm_shuffle_epi32(a, 0x4e)), + _mm_xor_si128(b, _mm_shuffle_epi32(b, 0x4e)), + 0x00); + tmp4 = _mm_xor_si128(tmp4, _mm_xor_si128(tmp3, tmp6)); - tmp4 = _mm_xor_si128(tmp4, tmp5); tmp5 = _mm_slli_si128(tmp4, 8); tmp4 = _mm_srli_si128(tmp4, 8); - tmp3 = _mm_xor_si128(tmp3, tmp5); - tmp6 = _mm_xor_si128(tmp6, tmp4); + *lo = _mm_xor_si128(tmp3, tmp5); + *hi = _mm_xor_si128(tmp6, tmp4); +} + +/* Shift the 256-bit product left by one to undo GCM's bit reflection, then + * reduce modulo the GCM polynomial. This is the expensive half. */ +TARGET_AESNI_PCLMUL +static inline __m128i gfred_pclmuldq(__m128i tmp3, __m128i tmp6) +{ + __m128i tmp2, tmp4, tmp5, tmp7, tmp8, tmp9; + __m128i bswap_mask = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15); + tmp7 = _mm_srli_epi32(tmp3, 31); tmp8 = _mm_srli_epi32(tmp6, 31); tmp3 = _mm_slli_epi32(tmp3, 1); @@ -236,14 +334,38 @@ return _mm_shuffle_epi8(tmp6, bswap_mask); } +TARGET_AESNI_PCLMUL +static __m128i gfmul_pclmuldq(__m128i a, const table_4bit htable) +{ + __m128i lo, hi; + + clmul_pclmuldq(a, _mm_loadu_si128((__m128i *) htable), &lo, &hi); + return gfred_pclmuldq(lo, hi); +} + +TARGET_AESNI_PCLMUL void crypton_aesni_hinit_pclmul(table_4bit htable, const block128 *h) { + __m128i bswap_mask = _mm_set_epi8(0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15); + __m128i p; + int i; + /* When pclmul is active we don't need to fill the table. Instead we just * store H at index 0. It is written in reverse order, so function * gfmul_pclmuldq will not byte-swap this value. */ - htable->q[0] = bitfn_swap64(h->q[1]); - htable->q[1] = bitfn_swap64(h->q[0]); + htable[0].q[0] = bitfn_swap64(h->q[1]); + htable[0].q[1] = bitfn_swap64(h->q[0]); + + /* Indices 1..7 get H^2 .. H^8, which is what lets a group of blocks + * fold into one reduction: gf_mul4 uses the first four, the GCM loop + * all eight. The table has sixteen slots. */ + p = _mm_loadu_si128((const __m128i *) h); + for (i = 1; i < 8; i++) { + p = gfmul_pclmuldq(p, htable); + _mm_storeu_si128((__m128i *) &htable[i], + _mm_shuffle_epi8(p, bswap_mask)); + } } TARGET_AESNI_PCLMUL @@ -255,13 +377,74 @@ _mm_storeu_si128((__m128i *) a, _b); } +/* + * Four GHASH steps -- ((((a^b0)H ^ b1)H ^ b2)H ^ b3)H -- with a single + * reduction. Expanded that is (a^b0)H^4 ^ b1*H^3 ^ b2*H^2 ^ b3*H, so the + * four products can be summed first and reduced once, which is where the + * time goes. Aggregated reduction, from the Intel GCM paper. + */ +TARGET_AESNI_PCLMUL +static __m128i gfmul4_pclmul(__m128i tag, const table_4bit htable, const __m128i *m) +{ + __m128i lo, hi, l, h; + int i; + + clmul_pclmuldq(_mm_xor_si128(tag, m[0]), + _mm_loadu_si128((const __m128i *) &htable[3]), &lo, &hi); + + for (i = 1; i < 4; i++) { + clmul_pclmuldq(m[i], _mm_loadu_si128((const __m128i *) &htable[3 - i]), + &l, &h); + lo = _mm_xor_si128(lo, l); + hi = _mm_xor_si128(hi, h); + } + + return gfred_pclmuldq(lo, hi); +} + +TARGET_AESNI_PCLMUL +static __m128i gfmul8_pclmul(__m128i tag, const table_4bit htable, const __m128i *m) +{ + __m128i lo, hi, l, h; + int i; + + clmul_pclmuldq(_mm_xor_si128(tag, m[0]), + _mm_loadu_si128((const __m128i *) &htable[7]), &lo, &hi); + + for (i = 1; i < 8; i++) { + clmul_pclmuldq(m[i], _mm_loadu_si128((const __m128i *) &htable[7 - i]), + &l, &h); + lo = _mm_xor_si128(lo, l); + hi = _mm_xor_si128(hi, h); + } + + return gfred_pclmuldq(lo, hi); +} + +TARGET_AESNI_PCLMUL +void crypton_aesni_gf_mul4_pclmul(block128 *a, const block128 *blocks, const table_4bit htable) +{ + __m128i m[4]; + int i; + + for (i = 0; i < 4; i++) + m[i] = _mm_loadu_si128((const __m128i *) &blocks[i]); + + _mm_storeu_si128((__m128i *) a, + gfmul4_pclmul(_mm_loadu_si128((const __m128i *) a), htable, m)); +} + void crypton_aesni_init_pclmul(void) { crypton_gfmul_branch_ptr = gfmul_pclmuldq; + crypton_gfmul4_branch_ptr = gfmul4_pclmul; + crypton_gfmul8_branch_ptr = gfmul8_pclmul; } #else #define gfmul(a,t) (gfmul_generic(a,t)) +#define gfmul4(a,t,m) (gfmul4_generic(a,t,m)) +#define gfmul8(a,t,m) (gfmul8_generic(a,t,m)) #endif TARGET_AESNI @@ -271,6 +454,164 @@ return gfmul(tag, htable); } +TARGET_AESNI +static inline __m128i ghash_add4(__m128i tag, const table_4bit htable, const __m128i *m) +{ + return gfmul4(tag, htable, m); +} + +TARGET_AESNI +static inline __m128i ghash_add8(__m128i tag, const table_4bit htable, const __m128i *m) +{ + return gfmul8(tag, htable, m); +} + +/* + * Eight blocks through the rounds with the round keys read from memory rather + * than held in registers. + * + * There are sixteen vector registers. Eight blocks and eleven to fifteen + * round keys do not fit in them, and when the GCM loop preloaded the keys the + * compiler spilled: ninety-six stack accesses around a hundred AESENCs, which + * cost more than half the loop's throughput. AESENC takes a memory operand, + * and the round keys are in L1 from one group to the next, so reading them + * each round costs nothing and leaves the registers for the blocks. + */ +/* + * Eight blocks through the rounds with the round keys read from memory rather + * than held in registers. + * + * There are sixteen vector registers. Eight blocks and eleven to fifteen + * round keys do not fit in them, and when the GCM loop preloaded the keys the + * compiler spilled: ninety-six stack accesses around a hundred AESENCs. + * AESENC takes a memory operand and the round keys stay in L1 from one group + * to the next, so reading them costs nothing and leaves the registers for the + * blocks. + * + * The rounds are written out rather than looped: the loop cost a fifth of the + * throughput, which is what -funroll-loops was recovering. + */ +#define K_(r) _mm_loadu_si128(k_ + (r)) + +/* the rounds beyond the tenth, which only a longer key has */ +#define ROUNDS8_EXTRA_128 +#define ROUNDS8_EXTRA_192 AESENC8(K_(10)) AESENC8(K_(11)) +#define ROUNDS8_EXTRA_256 \ + AESENC8(K_(10)) AESENC8(K_(11)) AESENC8(K_(12)) AESENC8(K_(13)) + +#define DO_ENC_BLOCK8_MEM(m, k, nbr, EXTRA) \ + do { \ + const __m128i *k_ = (const __m128i *) (k); \ + XOR8(K_(0)) \ + AESENC8(K_(1)) AESENC8(K_(2)) AESENC8(K_(3)) \ + AESENC8(K_(4)) AESENC8(K_(5)) AESENC8(K_(6)) \ + AESENC8(K_(7)) AESENC8(K_(8)) AESENC8(K_(9)) \ + EXTRA \ + AESENCLAST8(K_(nbr)) \ + } while (0) + +#define DO_ENC_BLOCK_MEM(m, k, nbr) \ + do { \ + const __m128i *k_ = (const __m128i *) (k); \ + int r_; \ + m = _mm_xor_si128(m, K_(0)); \ + for (r_ = 1; r_ < (nbr); r_++) \ + m = _mm_aesenc_si128(m, K_(r_)); \ + m = _mm_aesenclast_si128(m, K_(nbr)); \ + } while (0) + +/* + * GCM's GHASH, called directly rather than through the branch pointer the + * other callers use: the pointer is a call the compiler cannot see through, + * and these want to be scheduled against the rounds around them. The cost is + * that the GCM loops are compiled with the instruction and so may only be + * installed where the processor has it, which crypton_aes.c sees to, as it + * already does for the AArch64 ones. + */ +#ifdef WITH_PCLMUL + +#define GCM_TARGET TARGET_AESNI_PCLMUL + +TARGET_AESNI_PCLMUL +static inline __m128i gcm_ghash_add(__m128i tag, const table_4bit htable, __m128i m) +{ + return gfmul_pclmuldq(_mm_xor_si128(tag, m), htable); +} + +TARGET_AESNI_PCLMUL +static inline __m128i gcm_ghash_add8(__m128i tag, const table_4bit htable, const __m128i *m) +{ + return gfmul8_pclmul(tag, htable, m); +} + +/* + * One block's carry-less multiply, accumulated rather than reduced, so that + * the eight of a group can be spread between the rounds of the next group's + * AES. + */ +TARGET_AESNI_PCLMUL +static inline void ghash_fold(__m128i *lo, __m128i *hi, __m128i b, + const table_4bit htable, int i) +{ + __m128i l, h; + + clmul_pclmuldq(b, _mm_loadu_si128((const __m128i *) &htable[i]), &l, &h); + *lo = _mm_xor_si128(*lo, l); + *hi = _mm_xor_si128(*hi, h); +} + +#else + +#define GCM_TARGET TARGET_AESNI +#define gcm_ghash_add(t, h, m) ghash_add((t), (h), (m)) +#define gcm_ghash_add8(t, h, m) ghash_add8((t), (h), (m)) + +#endif + +/* + * A group of eight encrypted, with the previous group's GHASH folded in + * between the rounds where the build has the carry-less multiply: GH(j) after + * round j + 1, and the reduction after round nine, which every key size + * reaches. The names are the ones the GCM loops use. + */ +#ifdef WITH_PCLMUL + +#define GCM_GH(j) \ + ghash_fold(&glo_, &ghi_, \ + (j) == 0 ? _mm_xor_si128(tag, pending[0]) : pending[j], \ + gcm->htable, 7 - (j)); + +#define GCM_GHRED tag = gfred_pclmuldq(glo_, ghi_); + +#define GCM_GROUP8(m, k, nbr, EXTRA) \ + do { \ + const __m128i *k_ = (const __m128i *) (k); \ + __m128i glo_ = _mm_setzero_si128(); \ + __m128i ghi_ = _mm_setzero_si128(); \ + XOR8(K_(0)) \ + AESENC8(K_(1)) GCM_GH(0) \ + AESENC8(K_(2)) GCM_GH(1) \ + AESENC8(K_(3)) GCM_GH(2) \ + AESENC8(K_(4)) GCM_GH(3) \ + AESENC8(K_(5)) GCM_GH(4) \ + AESENC8(K_(6)) GCM_GH(5) \ + AESENC8(K_(7)) GCM_GH(6) \ + AESENC8(K_(8)) GCM_GH(7) \ + AESENC8(K_(9)) GCM_GHRED \ + EXTRA \ + AESENCLAST8(K_(nbr)) \ + } while (0) + +#else + +#define GCM_GROUP8(m, k, nbr, EXTRA) \ + do { \ + DO_ENC_BLOCK8_MEM(m, k, nbr, EXTRA); \ + tag = ghash_add8(tag, gcm->htable, pending); \ + } while (0) + +#endif + #define PRELOAD_ENC_KEYS128(k) \ __m128i K0 = _mm_loadu_si128(((__m128i *) k)+0); \ __m128i K1 = _mm_loadu_si128(((__m128i *) k)+1); \ @@ -284,6 +625,11 @@ __m128i K9 = _mm_loadu_si128(((__m128i *) k)+9); \ __m128i K10 = _mm_loadu_si128(((__m128i *) k)+10); +#define PRELOAD_ENC_KEYS192(k) \ + PRELOAD_ENC_KEYS128(k) \ + __m128i K11 = _mm_loadu_si128(((__m128i *) k)+11); \ + __m128i K12 = _mm_loadu_si128(((__m128i *) k)+12); + #define PRELOAD_ENC_KEYS256(k) \ PRELOAD_ENC_KEYS128(k) \ __m128i K11 = _mm_loadu_si128(((__m128i *) k)+11); \ @@ -304,6 +650,21 @@ m = _mm_aesenc_si128(m, K9); \ m = _mm_aesenclast_si128(m, K10); +#define DO_ENC_BLOCK192(m) \ + m = _mm_xor_si128(m, K0); \ + m = _mm_aesenc_si128(m, K1); \ + m = _mm_aesenc_si128(m, K2); \ + m = _mm_aesenc_si128(m, K3); \ + m = _mm_aesenc_si128(m, K4); \ + m = _mm_aesenc_si128(m, K5); \ + m = _mm_aesenc_si128(m, K6); \ + m = _mm_aesenc_si128(m, K7); \ + m = _mm_aesenc_si128(m, K8); \ + m = _mm_aesenc_si128(m, K9); \ + m = _mm_aesenc_si128(m, K10); \ + m = _mm_aesenc_si128(m, K11); \ + m = _mm_aesenclast_si128(m, K12); + #define DO_ENC_BLOCK256(m) \ m = _mm_xor_si128(m, K0); \ m = _mm_aesenc_si128(m, K1); \ @@ -334,10 +695,55 @@ __m128i K8 = _mm_loadu_si128(((__m128i *) k)+at+8); \ __m128i K9 = _mm_loadu_si128(((__m128i *) k)+at+9); \ +/* + * Eight blocks through the rounds together, which is what covers the + * latency of AESENC. Written out one line per block rather than left to a + * loop over m[i]: a loop is only as good as the compiler's willingness to + * unroll it, and when it declines the blocks go to the stack and each + * round becomes a load and a store. + */ +#define XOR8(KK) \ + m[0] = _mm_xor_si128(m[0], KK); m[1] = _mm_xor_si128(m[1], KK); \ + m[2] = _mm_xor_si128(m[2], KK); m[3] = _mm_xor_si128(m[3], KK); \ + m[4] = _mm_xor_si128(m[4], KK); m[5] = _mm_xor_si128(m[5], KK); \ + m[6] = _mm_xor_si128(m[6], KK); m[7] = _mm_xor_si128(m[7], KK); + +#define AESENC8(KK) \ + m[0] = _mm_aesenc_si128(m[0], KK); m[1] = _mm_aesenc_si128(m[1], KK); \ + m[2] = _mm_aesenc_si128(m[2], KK); m[3] = _mm_aesenc_si128(m[3], KK); \ + m[4] = _mm_aesenc_si128(m[4], KK); m[5] = _mm_aesenc_si128(m[5], KK); \ + m[6] = _mm_aesenc_si128(m[6], KK); m[7] = _mm_aesenc_si128(m[7], KK); + +#define AESENCLAST8(KK) \ + m[0] = _mm_aesenclast_si128(m[0], KK); m[1] = _mm_aesenclast_si128(m[1], KK); \ + m[2] = _mm_aesenclast_si128(m[2], KK); m[3] = _mm_aesenclast_si128(m[3], KK); \ + m[4] = _mm_aesenclast_si128(m[4], KK); m[5] = _mm_aesenclast_si128(m[5], KK); \ + m[6] = _mm_aesenclast_si128(m[6], KK); m[7] = _mm_aesenclast_si128(m[7], KK); + +#define DO_ENC_BLOCK8_128(m) \ + XOR8(K0) AESENC8(K1) AESENC8(K2) AESENC8(K3) AESENC8(K4) AESENC8(K5) \ + AESENC8(K6) AESENC8(K7) AESENC8(K8) AESENC8(K9) AESENCLAST8(K10) + +#define DO_ENC_BLOCK8_192(m) \ + XOR8(K0) AESENC8(K1) AESENC8(K2) AESENC8(K3) AESENC8(K4) AESENC8(K5) \ + AESENC8(K6) AESENC8(K7) AESENC8(K8) AESENC8(K9) AESENC8(K10) \ + AESENC8(K11) AESENCLAST8(K12) + +#define DO_ENC_BLOCK8_256(m) \ + XOR8(K0) AESENC8(K1) AESENC8(K2) AESENC8(K3) AESENC8(K4) AESENC8(K5) \ + AESENC8(K6) AESENC8(K7) AESENC8(K8) AESENC8(K9) AESENC8(K10) \ + AESENC8(K11) AESENC8(K12) AESENC8(K13) AESENCLAST8(K14) + #define PRELOAD_DEC_KEYS128(k) \ PRELOAD_DEC_KEYS_AT(k, 10) \ __m128i K10 = _mm_loadu_si128(((__m128i *) k)+0); +#define PRELOAD_DEC_KEYS192(k) \ + PRELOAD_DEC_KEYS_AT(k, 12) \ + __m128i K10 = _mm_loadu_si128(((__m128i *) k)+12+10); \ + __m128i K11 = _mm_loadu_si128(((__m128i *) k)+12+11); \ + __m128i K12 = _mm_loadu_si128(((__m128i *) k)+0); + #define PRELOAD_DEC_KEYS256(k) \ PRELOAD_DEC_KEYS_AT(k, 14) \ __m128i K10 = _mm_loadu_si128(((__m128i *) k)+14+10); \ @@ -346,6 +752,76 @@ __m128i K13 = _mm_loadu_si128(((__m128i *) k)+14+13); \ __m128i K14 = _mm_loadu_si128(((__m128i *) k)+0); +#define AESDEC8(KK) \ + m[0] = _mm_aesdec_si128(m[0], KK); m[1] = _mm_aesdec_si128(m[1], KK); \ + m[2] = _mm_aesdec_si128(m[2], KK); m[3] = _mm_aesdec_si128(m[3], KK); \ + m[4] = _mm_aesdec_si128(m[4], KK); m[5] = _mm_aesdec_si128(m[5], KK); \ + m[6] = _mm_aesdec_si128(m[6], KK); m[7] = _mm_aesdec_si128(m[7], KK); + +#define AESDECLAST8(KK) \ + m[0] = _mm_aesdeclast_si128(m[0], KK); m[1] = _mm_aesdeclast_si128(m[1], KK); \ + m[2] = _mm_aesdeclast_si128(m[2], KK); m[3] = _mm_aesdeclast_si128(m[3], KK); \ + m[4] = _mm_aesdeclast_si128(m[4], KK); m[5] = _mm_aesdeclast_si128(m[5], KK); \ + m[6] = _mm_aesdeclast_si128(m[6], KK); m[7] = _mm_aesdeclast_si128(m[7], KK); + +#define DO_DEC_BLOCK8_128(m) \ + XOR8(K0) AESDEC8(K1) AESDEC8(K2) AESDEC8(K3) AESDEC8(K4) AESDEC8(K5) \ + AESDEC8(K6) AESDEC8(K7) AESDEC8(K8) AESDEC8(K9) AESDECLAST8(K10) + +#define DO_DEC_BLOCK8_192(m) \ + XOR8(K0) AESDEC8(K1) AESDEC8(K2) AESDEC8(K3) AESDEC8(K4) AESDEC8(K5) \ + AESDEC8(K6) AESDEC8(K7) AESDEC8(K8) AESDEC8(K9) AESDEC8(K10) \ + AESDEC8(K11) AESDECLAST8(K12) + +#define DO_DEC_BLOCK8_256(m) \ + XOR8(K0) AESDEC8(K1) AESDEC8(K2) AESDEC8(K3) AESDEC8(K4) AESDEC8(K5) \ + AESDEC8(K6) AESDEC8(K7) AESDEC8(K8) AESDEC8(K9) AESDEC8(K10) \ + AESDEC8(K11) AESDEC8(K12) AESDEC8(K13) AESDECLAST8(K14) + +/* + * The XTS tweak advances by doubling in GF(2^128). gfmulx above does that + * through memory; this keeps it in a register, which matters once eight + * tweaks are wanted per group. The block is little-endian, so the low + * 64-bit half is first. + */ +TARGET_AESNI +static inline __m128i gfmulx_sse(__m128i v) +{ + const __m128i poly = _mm_set_epi64x(0, 0x87); + const __m128i carry = _mm_srli_epi64(v, 63); + /* the low half's carry becomes the high half's bit 0 */ + const __m128i into_hi = _mm_slli_si128(carry, 8); + /* and the high half's becomes all ones, or nothing, in the low half */ + const __m128i out = _mm_sub_epi64(_mm_setzero_si128(), _mm_srli_si128(carry, 8)); + + return _mm_xor_si128(_mm_or_si128(_mm_slli_epi64(v, 1), into_hi), + _mm_and_si128(out, poly)); +} + +/* + * The tweak doubles in a pair of general-purpose registers and is moved + * into a vector one per block. The doubling is three integer operations, + * and the integer units have nothing else to do here, where there are only + * sixteen vector registers and the rounds want as many of them as they can + * get: done in vector registers, which is what this did, the eight + * doublings of a group both lengthen the critical path and push the round + * keys out to memory. + */ +#define XTS_TWEAK_STEP(lo, hi) do { \ + const uint64_t _c = (hi) >> 63; \ + (hi) = ((hi) << 1) | ((lo) >> 63); \ + (lo) = ((lo) << 1) ^ (_c ? 0x87 : 0); \ +} while (0) + +/* the eight tweaks a group needs, from the one it starts at */ +#define XTS_TWEAKS8(dst, lo, hi) do { \ + int _i; \ + for (_i = 0; _i < 8; _i++) { \ + (dst)[_i] = _mm_set_epi64x((long long) (hi), (long long) (lo)); \ + XTS_TWEAK_STEP(lo, hi); \ + } \ +} while (0) + #define DO_DEC_BLOCK128(m) \ m = _mm_xor_si128(m, K0); \ m = _mm_aesdec_si128(m, K1); \ @@ -359,6 +835,21 @@ m = _mm_aesdec_si128(m, K9); \ m = _mm_aesdeclast_si128(m, K10); +#define DO_DEC_BLOCK192(m) \ + m = _mm_xor_si128(m, K0); \ + m = _mm_aesdec_si128(m, K1); \ + m = _mm_aesdec_si128(m, K2); \ + m = _mm_aesdec_si128(m, K3); \ + m = _mm_aesdec_si128(m, K4); \ + m = _mm_aesdec_si128(m, K5); \ + m = _mm_aesdec_si128(m, K6); \ + m = _mm_aesdec_si128(m, K7); \ + m = _mm_aesdec_si128(m, K8); \ + m = _mm_aesdec_si128(m, K9); \ + m = _mm_aesdec_si128(m, K10); \ + m = _mm_aesdec_si128(m, K11); \ + m = _mm_aesdeclast_si128(m, K12); + #define DO_DEC_BLOCK256(m) \ m = _mm_xor_si128(m, K0); \ m = _mm_aesdec_si128(m, K1); \ @@ -377,34 +868,73 @@ m = _mm_aesdeclast_si128(m, K14); #define SIZE 128 +#define NBR 10 +#define ROUNDS8_EXTRA ROUNDS8_EXTRA_128 #define SIZED(m) m##128 #define PRELOAD_ENC PRELOAD_ENC_KEYS128 #define DO_ENC_BLOCK DO_ENC_BLOCK128 +#define DO_ENC_BLOCK8 DO_ENC_BLOCK8_128 #define PRELOAD_DEC PRELOAD_DEC_KEYS128 #define DO_DEC_BLOCK DO_DEC_BLOCK128 +#define DO_DEC_BLOCK8 DO_DEC_BLOCK8_128 #include #undef SIZE +#undef NBR +#undef ROUNDS8_EXTRA #undef SIZED #undef PRELOAD_ENC #undef PRELOAD_DEC #undef DO_ENC_BLOCK +#undef DO_ENC_BLOCK8 #undef DO_DEC_BLOCK +#undef DO_DEC_BLOCK8 +#define SIZED(m) m##192 +#define SIZE 192 +#define NBR 12 +#define ROUNDS8_EXTRA ROUNDS8_EXTRA_192 +#define PRELOAD_ENC PRELOAD_ENC_KEYS192 +#define DO_ENC_BLOCK DO_ENC_BLOCK192 +#define DO_ENC_BLOCK8 DO_ENC_BLOCK8_192 +#define PRELOAD_DEC PRELOAD_DEC_KEYS192 +#define DO_DEC_BLOCK DO_DEC_BLOCK192 +#define DO_DEC_BLOCK8 DO_DEC_BLOCK8_192 +#include + +#undef SIZE +#undef NBR +#undef ROUNDS8_EXTRA +#undef SIZED +#undef PRELOAD_ENC +#undef PRELOAD_DEC +#undef DO_ENC_BLOCK +#undef DO_ENC_BLOCK8 +#undef DO_DEC_BLOCK +#undef DO_DEC_BLOCK8 + #define SIZED(m) m##256 #define SIZE 256 +#define NBR 14 +#define ROUNDS8_EXTRA ROUNDS8_EXTRA_256 #define PRELOAD_ENC PRELOAD_ENC_KEYS256 #define DO_ENC_BLOCK DO_ENC_BLOCK256 +#define DO_ENC_BLOCK8 DO_ENC_BLOCK8_256 #define PRELOAD_DEC PRELOAD_DEC_KEYS256 #define DO_DEC_BLOCK DO_DEC_BLOCK256 +#define DO_DEC_BLOCK8 DO_DEC_BLOCK8_256 #include #undef SIZE +#undef NBR +#undef ROUNDS8_EXTRA #undef SIZED #undef PRELOAD_ENC #undef PRELOAD_DEC #undef DO_ENC_BLOCK +#undef DO_ENC_BLOCK8 #undef DO_DEC_BLOCK +#undef DO_DEC_BLOCK8 #endif diff --git a/cbits/aes/x86ni.h b/cbits/aes/x86ni.h --- a/cbits/aes/x86ni.h +++ b/cbits/aes/x86ni.h @@ -59,34 +59,30 @@ #endif void crypton_aesni_init(aes_key *key, uint8_t *origkey, uint8_t size); -void crypton_aesni_encrypt_block128(aes_block *out, aes_key *key, aes_block *in); -void crypton_aesni_encrypt_block256(aes_block *out, aes_key *key, aes_block *in); -void crypton_aesni_decrypt_block128(aes_block *out, aes_key *key, aes_block *in); -void crypton_aesni_decrypt_block256(aes_block *out, aes_key *key, aes_block *in); -void crypton_aesni_encrypt_ecb128(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); -void crypton_aesni_encrypt_ecb256(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); -void crypton_aesni_decrypt_ecb128(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); -void crypton_aesni_decrypt_ecb256(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); -void crypton_aesni_encrypt_cbc128(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); -void crypton_aesni_encrypt_cbc256(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); -void crypton_aesni_decrypt_cbc128(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); -void crypton_aesni_decrypt_cbc256(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); -void crypton_aesni_encrypt_ctr128(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); -void crypton_aesni_encrypt_ctr256(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); -void crypton_aesni_encrypt_c32_128(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); -void crypton_aesni_encrypt_c32_256(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); -void crypton_aesni_encrypt_xts128(aes_block *out, aes_key *key1, aes_key *key2, - aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks); -void crypton_aesni_encrypt_xts256(aes_block *out, aes_key *key1, aes_key *key2, - aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks); - -void crypton_aesni_gcm_encrypt128(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length); -void crypton_aesni_gcm_encrypt256(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length); +#define AESNI_DECLS(sz) \ + void crypton_aesni_encrypt_block##sz(aes_block *out, aes_key *key, aes_block *in); \ + void crypton_aesni_decrypt_block##sz(aes_block *out, aes_key *key, aes_block *in); \ + void crypton_aesni_encrypt_ecb##sz(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); \ + void crypton_aesni_decrypt_ecb##sz(aes_block *out, aes_key *key, aes_block *in, uint32_t blocks); \ + void crypton_aesni_encrypt_cbc##sz(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); \ + void crypton_aesni_decrypt_cbc##sz(aes_block *out, aes_key *key, aes_block *_iv, aes_block *in, uint32_t blocks); \ + void crypton_aesni_encrypt_ctr##sz(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); \ + void crypton_aesni_encrypt_c32_##sz(uint8_t *out, aes_key *key, aes_block *_iv, uint8_t *in, uint32_t length); \ + void crypton_aesni_encrypt_xts##sz(aes_block *out, aes_key *key1, aes_key *key2, \ + aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks); \ + void crypton_aesni_decrypt_xts##sz(aes_block *out, aes_key *key1, aes_key *key2, \ + aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks); \ + void crypton_aesni_gcm_encrypt##sz(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length); \ + void crypton_aesni_gcm_decrypt##sz(uint8_t *out, aes_gcm *gcm, aes_key *key, uint8_t *in, uint32_t length); +AESNI_DECLS(128) +AESNI_DECLS(192) +AESNI_DECLS(256) #ifdef WITH_PCLMUL void crypton_aesni_init_pclmul(void); void crypton_aesni_hinit_pclmul(table_4bit htable, const block128 *h); void crypton_aesni_gf_mul_pclmul(block128 *a, const table_4bit htable); +void crypton_aesni_gf_mul4_pclmul(block128 *a, const block128 *blocks, const table_4bit htable); #endif #endif diff --git a/cbits/aes/x86ni_impl.c b/cbits/aes/x86ni_impl.c --- a/cbits/aes/x86ni_impl.c +++ b/cbits/aes/x86ni_impl.c @@ -204,34 +204,124 @@ void SIZED(crypton_aesni_encrypt_xts)(aes_block *out, aes_key *key1, aes_key *key2, aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks) { - __m128i tweak = _mm_loadu_si128((__m128i *) _tweak); + uint64_t tlo, thi; do { __m128i *k2 = (__m128i *) key2->data; + __m128i tweak = _mm_loadu_si128((__m128i *) _tweak); + aes_block first ALIGNMENT(16); + PRELOAD_ENC(k2); DO_ENC_BLOCK(tweak); + _mm_storeu_si128((__m128i *) &first, tweak); + tlo = first.q[0]; + thi = first.q[1]; while (spoint-- > 0) - tweak = gfmulx(tweak); + XTS_TWEAK_STEP(tlo, thi); } while (0) ; do { __m128i *k1 = (__m128i *) key1->data; - PRELOAD_ENC(k1); - for ( ; blocks-- > 0; in += 1, out += 1, tweak = gfmulx(tweak)) { + /* + * Eight at a time. The eight tweaks are kept from one group + * to the next and each is advanced by eight doublings at + * once, which is a single multiplication and does not wait + * for the other seven; doubling along the group instead, + * which is what this did, puts a chain of eight in front of + * every set of rounds, and on a processor whose AES is fast + * that chain is most of the block. + */ + for ( ; blocks >= 8; blocks -= 8, in += 8, out += 8) { + __m128i m[8], t[8]; + int i; + + XTS_TWEAKS8(t, tlo, thi); + for (i = 0; i < 8; i++) + m[i] = _mm_xor_si128( + _mm_loadu_si128((__m128i *) (in + i)), t[i]); + DO_ENC_BLOCK8_MEM(m, k1, NBR, ROUNDS8_EXTRA); + for (i = 0; i < 8; i++) + _mm_storeu_si128((__m128i *) (out + i), + _mm_xor_si128(m[i], t[i])); + } + for ( ; blocks-- > 0; in += 1, out += 1) { + const __m128i tweak = + _mm_set_epi64x((long long) thi, (long long) tlo); __m128i m = _mm_loadu_si128((__m128i *) in); m = _mm_xor_si128(m, tweak); - DO_ENC_BLOCK(m); + DO_ENC_BLOCK_MEM(m, k1, NBR); m = _mm_xor_si128(m, tweak); _mm_storeu_si128((__m128i *) out, m); + XTS_TWEAK_STEP(tlo, thi); } } while (0); } +/* + * XTS the other way, which until now fell to the generic loop -- and which + * nothing reached at all, since crypton_aes_decrypt_xts called the generic + * function directly rather than through the branch table. The tweak is + * enciphered whichever way the data goes; only the data is deciphered. + */ TARGET_AESNI +void SIZED(crypton_aesni_decrypt_xts)(aes_block *out, aes_key *key1, aes_key *key2, + aes_block *_tweak, uint32_t spoint, aes_block *in, uint32_t blocks) +{ + uint64_t tlo, thi; + + do { + __m128i *k2 = (__m128i *) key2->data; + __m128i tweak = _mm_loadu_si128((__m128i *) _tweak); + aes_block first ALIGNMENT(16); + + PRELOAD_ENC(k2); + DO_ENC_BLOCK(tweak); + _mm_storeu_si128((__m128i *) &first, tweak); + tlo = first.q[0]; + thi = first.q[1]; + + while (spoint-- > 0) + XTS_TWEAK_STEP(tlo, thi); + } while (0) ; + + do { + __m128i *k1 = (__m128i *) key1->data; + PRELOAD_DEC(k1); + + /* the tweaks kept and advanced, as encryption has them */ + for ( ; blocks >= 8; blocks -= 8, in += 8, out += 8) { + __m128i m[8], t[8]; + int i; + + XTS_TWEAKS8(t, tlo, thi); + for (i = 0; i < 8; i++) + m[i] = _mm_xor_si128( + _mm_loadu_si128((__m128i *) (in + i)), t[i]); + DO_DEC_BLOCK8(m); + for (i = 0; i < 8; i++) + _mm_storeu_si128((__m128i *) (out + i), + _mm_xor_si128(m[i], t[i])); + } + for ( ; blocks-- > 0; in += 1, out += 1) { + const __m128i tweak = + _mm_set_epi64x((long long) thi, (long long) tlo); + __m128i m = _mm_loadu_si128((__m128i *) in); + + m = _mm_xor_si128(m, tweak); + DO_DEC_BLOCK(m); + m = _mm_xor_si128(m, tweak); + + _mm_storeu_si128((__m128i *) out, m); + XTS_TWEAK_STEP(tlo, thi); + } + } while (0); +} + +GCM_TARGET void SIZED(crypton_aesni_gcm_encrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length) { __m128i *k = (__m128i *) key->data; @@ -239,15 +329,73 @@ __m128i one = _mm_set_epi32(0,1,0,0); uint32_t nb_blocks = length / 16; uint32_t part_block_len = length % 16; + /* the group of ciphertext whose GHASH has not been taken yet */ + __m128i pending[8]; + int held = 0; gcm->length_input += length; +#if defined(WITH_X86_GCM_ASM) && defined(WITH_PCLMUL) + /* + * The stitched assembly first, which takes whole groups of six + * blocks off the front of the message and leaves the counter and the + * running tag where the loop below expects to find them. It wants + * eighteen blocks before it will start, and answers with what it did. + */ + if (nb_blocks >= GCM_ASM_MIN_BLOCKS_ENC) { + uint32_t done = crypton_gcm_asm_bulk_encrypt(output, gcm, key, + input, nb_blocks * 16); + + output += done; + input += done; + nb_blocks -= done / 16; + } +#endif + __m128i tag = _mm_loadu_si128((__m128i *) &gcm->tag); __m128i iv = _mm_loadu_si128((__m128i *) &gcm->civ); iv = _mm_shuffle_epi8(iv, bswap_mask); - PRELOAD_ENC(k); + /* + * Eight blocks at a time: the counters go through the rounds together + * so the pipeline has something to do while AESENC is in flight, and + * their GHASH folds into one reduction against H^8 .. H^1 rather than + * eight. + * + * The GHASH is of the group before, not this one. Taken in step the + * two halves cannot overlap at all: the multiply of a block waits for + * the rounds that produced it, and on this processor they do not even + * want the same port -- AESENC and PCLMULQDQ issue to different ones, + * so held a group apart they run through each other. It costs one + * group's worth of ciphertext kept aside and a last GHASH after the + * loop. + */ + for (; nb_blocks >= 8; nb_blocks -= 8, output += 128, input += 128) { + __m128i m[8]; + int i; + + for (i = 0; i < 8; i++) { + /* iv += 1, put back in big endian */ + iv = _mm_add_epi32(iv, one); + m[i] = _mm_shuffle_epi8(iv, bswap_mask); + } + if (held) + GCM_GROUP8(m, k, NBR, ROUNDS8_EXTRA); + else + DO_ENC_BLOCK8_MEM(m, k, NBR, ROUNDS8_EXTRA); + + for (i = 0; i < 8; i++) { + m[i] = _mm_xor_si128(m[i], + _mm_loadu_si128((__m128i *) (input + 16 * i))); + _mm_storeu_si128((__m128i *) (output + 16 * i), m[i]); + } + for (i = 0; i < 8; i++) + pending[i] = m[i]; + held = 1; + } + if (held) + tag = gcm_ghash_add8(tag, gcm->htable, pending); for (; nb_blocks-- > 0; output += 16, input += 16) { /* iv += 1 */ iv = _mm_add_epi32(iv, one); @@ -255,11 +403,11 @@ /* put back iv in big endian, encrypt it, * and xor it to input */ __m128i tmp = _mm_shuffle_epi8(iv, bswap_mask); - DO_ENC_BLOCK(tmp); + DO_ENC_BLOCK_MEM(tmp, k, NBR); __m128i m = _mm_loadu_si128((__m128i *) input); m = _mm_xor_si128(m, tmp); - tag = ghash_add(tag, gcm->htable, m); + tag = gcm_ghash_add(tag, gcm->htable, m); /* store it out */ _mm_storeu_si128((__m128i *) output, m); @@ -294,16 +442,120 @@ /* put back iv in big endian mode, encrypt it and xor it with input */ __m128i tmp = _mm_shuffle_epi8(iv, bswap_mask); - DO_ENC_BLOCK(tmp); + DO_ENC_BLOCK_MEM(tmp, k, NBR); __m128i m = _mm_loadu_si128((__m128i *) &block); m = _mm_xor_si128(m, tmp); m = _mm_shuffle_epi8(m, mask); - tag = ghash_add(tag, gcm->htable, m); + tag = gcm_ghash_add(tag, gcm->htable, m); /* make output */ _mm_storeu_si128((__m128i *) &block.b, m); + memcpy(output, &block.b, part_block_len); + } + /* store back IV & tag */ + __m128i tmp = _mm_shuffle_epi8(iv, bswap_mask); + _mm_storeu_si128((__m128i *) &gcm->civ, tmp); + _mm_storeu_si128((__m128i *) &gcm->tag, tag); +} + +/* + * GCM decryption, which until now fell to the generic loop: that advances + * the counter and calls the block function once per block through the + * branch table, and measured a quarter the speed of encryption on the same + * machine. The shape is the encryption loop with two differences -- the + * tag is taken over the ciphertext, which is the input rather than the + * output, and the ciphertext is read before anything is written, since + * output may be input. + */ +GCM_TARGET +void SIZED(crypton_aesni_gcm_decrypt)(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length) +{ + __m128i *k = (__m128i *) key->data; + __m128i bswap_mask = _mm_setr_epi8(7,6,5,4,3,2,1,0,15,14,13,12,11,10,9,8); + __m128i one = _mm_set_epi32(0,1,0,0); + uint32_t nb_blocks = length / 16; + uint32_t part_block_len = length % 16; + /* the group of ciphertext whose GHASH has not been taken yet */ + __m128i pending[8]; + int held = 0; + + gcm->length_input += length; + +#if defined(WITH_X86_GCM_ASM) && defined(WITH_PCLMUL) + /* the same as encryption, except that decryption has nothing to + * hold back and so will start on six blocks */ + if (nb_blocks >= GCM_ASM_MIN_BLOCKS_DEC) { + uint32_t done = crypton_gcm_asm_bulk_decrypt(output, gcm, key, + input, nb_blocks * 16); + + output += done; + input += done; + nb_blocks -= done / 16; + } +#endif + + __m128i tag = _mm_loadu_si128((__m128i *) &gcm->tag); + __m128i iv = _mm_loadu_si128((__m128i *) &gcm->civ); + iv = _mm_shuffle_epi8(iv, bswap_mask); + + + /* the group before's GHASH, alongside this group's rounds, as + * encryption does it */ + for (; nb_blocks >= 8; nb_blocks -= 8, output += 128, input += 128) { + __m128i m[8], c[8]; + int i; + + for (i = 0; i < 8; i++) { + /* iv += 1, put back in big endian */ + iv = _mm_add_epi32(iv, one); + m[i] = _mm_shuffle_epi8(iv, bswap_mask); + } + for (i = 0; i < 8; i++) + c[i] = _mm_loadu_si128((__m128i *) (input + 16 * i)); + if (held) + GCM_GROUP8(m, k, NBR, ROUNDS8_EXTRA); + else + DO_ENC_BLOCK8_MEM(m, k, NBR, ROUNDS8_EXTRA); + + for (i = 0; i < 8; i++) + _mm_storeu_si128((__m128i *) (output + 16 * i), + _mm_xor_si128(m[i], c[i])); + for (i = 0; i < 8; i++) + pending[i] = c[i]; + held = 1; + } + if (held) + tag = gcm_ghash_add8(tag, gcm->htable, pending); + for (; nb_blocks-- > 0; output += 16, input += 16) { + __m128i c = _mm_loadu_si128((__m128i *) input); + + iv = _mm_add_epi32(iv, one); + __m128i tmp = _mm_shuffle_epi8(iv, bswap_mask); + DO_ENC_BLOCK_MEM(tmp, k, NBR); + + tag = gcm_ghash_add(tag, gcm->htable, c); + _mm_storeu_si128((__m128i *) output, _mm_xor_si128(tmp, c)); + } + if (part_block_len > 0) { + aes_block block; + + /* the ciphertext padded with zeros is what the tag is taken + * over, so no mask is needed the way encryption needs one */ + block128_zero(&block); + block128_copy_bytes(&block, input, part_block_len); + __m128i c = _mm_loadu_si128((__m128i *) &block); + + /* iv += 1 */ + iv = _mm_add_epi32(iv, one); + + __m128i tmp = _mm_shuffle_epi8(iv, bswap_mask); + DO_ENC_BLOCK_MEM(tmp, k, NBR); + + tag = gcm_ghash_add(tag, gcm->htable, c); + + _mm_storeu_si128((__m128i *) &block.b, _mm_xor_si128(tmp, c)); memcpy(output, &block.b, part_block_len); } /* store back IV & tag */ diff --git a/cbits/asm/LICENSE.cryptogams b/cbits/asm/LICENSE.cryptogams new file mode 100644 --- /dev/null +++ b/cbits/asm/LICENSE.cryptogams @@ -0,0 +1,36 @@ +Copyright (c) 2006, CRYPTOGAMS by +All rights reserved. + +Redistribution and use in source and binary forms, with or without +modification, are permitted provided that the following conditions +are met: + + * Redistributions of source code must retain copyright notices, + this list of conditions and the following disclaimer. + + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following + disclaimer in the documentation and/or other materials + provided with the distribution. + + * Neither the name of the CRYPTOGAMS nor the names of its + copyright holder and contributors may be used to endorse or + promote products derived from this software without specific + prior written permission. + +ALTERNATIVELY, provided that this notice is retained in full, this +product may be distributed under the terms of the GNU General Public +License (GPL), in which case the provisions of the GPL apply INSTEAD OF +those given above. + +THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDER AND CONTRIBUTORS +"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. diff --git a/cbits/asm/README.md b/cbits/asm/README.md new file mode 100644 --- /dev/null +++ b/cbits/asm/README.md @@ -0,0 +1,171 @@ +# Vendored assembly + +## What is here + +Two modules from [CRYPTOGAMS](https://github.com/dot-asm/cryptogams), by Andy +Polyakov, checked in unmodified together with the translators they need: + +| generator | what it is | +| --- | --- | +| `aesni-gcm-x86_64.pl` | AES-NI/PCLMULQDQ stitched AES-GCM for x86-64 | +| `chacha-x86_64.pl` | ChaCha20 for x86-64 | +| `poly1305-x86_64.pl` | Poly1305 for x86-64 | +| `sha512-x86_64.pl` | SHA-256 and SHA-512 for x86-64 (one generator, two outputs, as on AArch64) | +| `keccak1600-x86_64.pl` | Keccak for x86-64 | +| `chacha-armv8.pl` | ChaCha20 for AArch64 | +| `poly1305-armv8.pl` | Poly1305 for AArch64 | +| `sha1-armv8.pl` | SHA-1 for AArch64 | +| `sha512-armv8.pl` | SHA-256 for AArch64 (the generator emits SHA-512 or SHA-256 according to the name it is given, and only the latter is wanted) | +| `keccak1600-armv8.pl` | Keccak for AArch64 | + +`x86_64-xlate.pl`, `arm-xlate.pl` and `arm_arch.h` are the machinery those +modules use. `generate.sh` runs the generators to produce the `.S` files, which are +what crypton actually compiles -- one per object format, since the calling +convention and the assembler syntax differ. The names of the entry points are +changed on the way through, and the ELF output is given the note that says the +code does not want an executable stack. + +Keeping the generated files in the tree means building crypton needs no perl. + +## Why + +Both do something the compiler will not do with intrinsics. + +**AES-GCM.** Counter-mode AES and GHASH do not compete for the same execution +ports, so a loop that interleaves them at instruction granularity runs both in +the time one of them would take. Written in C the interleaving does not +survive: given the AES rounds and the multiplies of a group of blocks, GCC and +Clang schedule all the multiplies after all the rounds, which is the sum of the +two rather than the maximum. + +**ChaCha20.** On AArch64 the vector registers hold four ChaCha states and there +is no room for a fifth, so further parallelism has to come from the integer +side: that module runs a fifth block through the general registers alongside +four in the vector ones, and above 512 bytes two alongside six. Which register +holds which word is the whole trick, and that is not something C says. The +x86-64 module is worth taking for a different reason -- it has vector code for +lengths the C here still takes a block at a time, so a 256-byte message more +than doubles -- and is a few per cent ahead in bulk besides. + +**Poly1305.** One multiplication modulo 2^130 - 5 depends on the one before it, +so what there is to win is in how the multiplies and the carries are laid +against each other, and in keeping the accumulator in whichever base costs +less: these modules work in base 2^64 while the message is short and switch to +base 2^26 for the vector loop, which is a decision no compiler will make for +you. On AArch64 that is twice the speed of the C here at 16 KiB and three and +a half times at 64 bytes; on x86-64, a quarter faster at 16 KiB and nearly +three times at 64. + +The x86-64 module also has paths for AVX-512, which are **not** taken. What +the generator emits is chosen from the version of the assembler it is told +about, and `generate.sh` tells it one that predates AVX-512: no machine here +can run those paths, an assembler old enough to be in use cannot always +assemble them, and a path nothing has executed is not worth the few per cent +it might be worth. + +**SHA-1, SHA-256, SHA-512 and Keccak.** On AArch64 the instructions are the same ones the +intrinsics here already use. What the module does is schedule them across a +whole run of blocks instead of one at a time, and keep the message schedule of +the next block moving while the rounds of this one are still going, which a +per-block C function cannot do at all. A quarter faster, and it needs no +alignment and no copy since it reads the message as bytes. The x86-64 module +is the same idea with more paths to choose from -- the SHA extensions, AVX2, +AVX, SSSE3 -- and is about a fifth faster than the C on a machine with AVX2 +and no SHA extensions. The AArch64 SHA-1 and Keccak modules are the same +story again -- the instructions are the ones the intrinsics here use, and what +the modules add is the arrangement: the schedule of the next four SHA-1 rounds +against the rounds of this one, and one Keccak round against the next. The +x86-64 Keccak is there for a different reason: nothing on that side has +instructions for this permutation, and what the module has over the C is that +its twenty-five lanes stay in registers across a round, where a compiler given +the C spills them. Two and a half times, and level with openssl. + +## Interfaces + + size_t crypton_gcm_asm_encrypt(const void *in, void *out, size_t len, + const void *key, unsigned char ivec[16], + void *Xi); + size_t crypton_gcm_asm_decrypt(... the same ...); + +Both return the number of bytes processed, which is a multiple of 96 and may be +zero: encryption wants at least 288 bytes to start, decryption at least 96. +Whatever is left over is the caller's to finish. + +`key` is the AES key schedule in the layout the OpenSSL assembly expects -- the +round keys, and at offset 240 one less than the number of rounds, which is what +OpenSSL's own AES-NI key setup puts there -- and `Xi` points at the running +GHASH state, with the table of powers of H, in the layout `gcm_init_avx` leaves +behind, 32 bytes past it. `cbits/aes/gcm_x86_asm.c` builds both, and the +multiplication that fills that table is written there in C rather than taken +from `ghash-x86_64.pl`: it runs once per message, so it is not worth a second +vendored file, and the one in OpenSSL is under a licence this package does not +use. + +The code needs AES-NI, PCLMULQDQ, AVX and MOVBE, which +`crypton_x86_simd_features()` is asked about before any of it is called. + + void crypton_chacha20_ctr32(unsigned char *out, const unsigned char *in, + size_t len, const unsigned int key[8], + const unsigned int counter[4]); + +Twenty rounds, the constants that go with a 256-bit key, and a 32-bit counter +which it does not write back: the caller advances it by the number of blocks. +Any length is accepted; the AArch64 module's vector path starts at 192 bytes, +the x86-64 one's rather lower. They ask `crypton_armcap_P` and +`crypton_ia32cap_P` respectively what the processor has, and +`cbits/crypton_chacha.c` calls them only for the states they fit -- twenty +rounds, a 256-bit key, and only as many blocks as the 32-bit counter has room +for. + + int crypton_poly1305_asm_init(void *ctx, const unsigned char key[16], + void *func[2]); + void crypton_poly1305_asm_blocks(void *ctx, const unsigned char *inp, + size_t len, unsigned int padbit); + void crypton_poly1305_asm_emit(void *ctx, unsigned char mac[16], + const unsigned int nonce[4]); + +`ctx` is 192 bytes of state the module keeps for itself -- its accumulator, the +clamped key and the powers of it -- and `key` is the first half of the Poly1305 +key, the second half being handed to `emit` as `nonce`. `padbit` is the bit +above each block, set for the blocks of the message and clear for the padded +last one. `len` is a whole number of blocks. + +Initialisation hands back through `func` the pair of functions its own dispatch +would use, the vector entry point not being exported, and +`cbits/crypton_poly1305.c` calls those. It reads `crypton_armcap_P` to choose +between them; `cbits/crypton_cpu.c` defines that. + +The x86-64 Poly1305 module presents the same three functions, and reads +`crypton_ia32cap_P` -- cpuid's own words, in the order OpenSSL keeps them -- +where the AArch64 one reads `crypton_armcap_P`. `cbits/crypton_cpu.c` fills +it, with the bits for anything the operating system will not preserve cleared, +and the AVX-512 ones cleared whatever the processor says. + + void crypton_sha1_asm_block_data_order(unsigned int state[5], + const void *data, size_t blocks); + void crypton_sha256_asm_block_data_order(unsigned int state[8], + const void *data, size_t blocks); + size_t crypton_keccak_asm_absorb_cext(unsigned long long state[25], + const void *inp, size_t len, + size_t bsz); + +The state is the words of the digest in host order and `blocks` whole blocks of +64 bytes. Each entry point picks between the SHA-2 instructions, NEON and +plain integer code from `crypton_armcap_P`, whose SHA-1 and SHA-256 bits +`cbits/crypton_sha1.c` and `cbits/crypton_sha256.c` set once they have asked +the operating system whether the processor has them -- they are optional in +ARMv8.0. + +Keccak's absorb takes the state as its twenty-five lanes, `bsz` as the rate in +bytes, and answers with what was left over. On AArch64 the `_cext` entry point +is the one that uses the SHA-3 instructions, and `cbits/crypton_sha3.c` calls +it only where its own runtime check has found them; the x86-64 module asks +nothing of the processor beyond the baseline and is called wherever it is +compiled in. + +## Licence + +`LICENSE.cryptogams` is the licence the CRYPTOGAMS files are distributed under. +It is the three-clause BSD licence, with the GNU General Public Licence offered +as an alternative; crypton takes the former, which is the licence of the rest +of this package. diff --git a/cbits/asm/aesni-gcm-x86_64-elf.S b/cbits/asm/aesni-gcm-x86_64-elf.S new file mode 100644 --- /dev/null +++ b/cbits/asm/aesni-gcm-x86_64-elf.S @@ -0,0 +1,814 @@ +.text + +.type _crypton_gcm_asm_ctr32_ghash_6x,@function +.align 32 +_crypton_gcm_asm_ctr32_ghash_6x: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + vmovdqu 32(%r11),%xmm2 + subq $6,%rdx + vpxor %xmm4,%xmm4,%xmm4 + vmovdqu 0-128(%rcx),%xmm15 + vpaddb %xmm2,%xmm1,%xmm10 + vpaddb %xmm2,%xmm10,%xmm11 + vpaddb %xmm2,%xmm11,%xmm12 + vpaddb %xmm2,%xmm12,%xmm13 + vpaddb %xmm2,%xmm13,%xmm14 + vpxor %xmm15,%xmm1,%xmm9 + vmovdqu %xmm4,16+8(%rsp) + jmp .Loop6x + +.align 32 +.Loop6x: + addl $100663296,%ebx + jc .Lhandle_ctr32 + vmovdqu 0-32(%r9),%xmm3 + vpaddb %xmm2,%xmm14,%xmm1 + vpxor %xmm15,%xmm10,%xmm10 + vpxor %xmm15,%xmm11,%xmm11 + +.Lresume_ctr32: + vmovdqu %xmm1,(%r8) + vpclmulqdq $0x10,%xmm3,%xmm7,%xmm5 + vpxor %xmm15,%xmm12,%xmm12 + vmovups 16-128(%rcx),%xmm2 + vpclmulqdq $0x01,%xmm3,%xmm7,%xmm6 + xorq %r12,%r12 + cmpq %r14,%r15 + + vaesenc %xmm2,%xmm9,%xmm9 + vmovdqu 48+8(%rsp),%xmm0 + vpxor %xmm15,%xmm13,%xmm13 + vpclmulqdq $0x00,%xmm3,%xmm7,%xmm1 + vaesenc %xmm2,%xmm10,%xmm10 + vpxor %xmm15,%xmm14,%xmm14 + setnc %r12b + vpclmulqdq $0x11,%xmm3,%xmm7,%xmm7 + vaesenc %xmm2,%xmm11,%xmm11 + vmovdqu 16-32(%r9),%xmm3 + negq %r12 + vaesenc %xmm2,%xmm12,%xmm12 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm3,%xmm0,%xmm5 + vpxor %xmm4,%xmm8,%xmm8 + vaesenc %xmm2,%xmm13,%xmm13 + vpxor %xmm5,%xmm1,%xmm4 + andq $0x60,%r12 + vmovups 32-128(%rcx),%xmm15 + vpclmulqdq $0x10,%xmm3,%xmm0,%xmm1 + vaesenc %xmm2,%xmm14,%xmm14 + + vpclmulqdq $0x01,%xmm3,%xmm0,%xmm2 + leaq (%r14,%r12,1),%r14 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor 16+8(%rsp),%xmm8,%xmm8 + vpclmulqdq $0x11,%xmm3,%xmm0,%xmm3 + vmovdqu 64+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 88(%r14),%r13 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 80(%r14),%r12 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,32+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,40+8(%rsp) + vmovdqu 48-32(%r9),%xmm5 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 48-128(%rcx),%xmm15 + vpxor %xmm1,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm5,%xmm0,%xmm1 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm2,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm5,%xmm0,%xmm2 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor %xmm3,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm5,%xmm0,%xmm3 + vaesenc %xmm15,%xmm11,%xmm11 + vpclmulqdq $0x11,%xmm5,%xmm0,%xmm5 + vmovdqu 80+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor %xmm1,%xmm4,%xmm4 + vmovdqu 64-32(%r9),%xmm1 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 64-128(%rcx),%xmm15 + vpxor %xmm2,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm1,%xmm0,%xmm2 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm3,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm1,%xmm0,%xmm3 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 72(%r14),%r13 + vpxor %xmm5,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm1,%xmm0,%xmm5 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 64(%r14),%r12 + vpclmulqdq $0x11,%xmm1,%xmm0,%xmm1 + vmovdqu 96+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,48+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,56+8(%rsp) + vpxor %xmm2,%xmm4,%xmm4 + vmovdqu 96-32(%r9),%xmm2 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 80-128(%rcx),%xmm15 + vpxor %xmm3,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm2,%xmm0,%xmm3 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm2,%xmm0,%xmm5 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 56(%r14),%r13 + vpxor %xmm1,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm2,%xmm0,%xmm1 + vpxor 112+8(%rsp),%xmm8,%xmm8 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 48(%r14),%r12 + vpclmulqdq $0x11,%xmm2,%xmm0,%xmm2 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,64+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,72+8(%rsp) + vpxor %xmm3,%xmm4,%xmm4 + vmovdqu 112-32(%r9),%xmm3 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 96-128(%rcx),%xmm15 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm5 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm1,%xmm6,%xmm6 + vpclmulqdq $0x01,%xmm3,%xmm8,%xmm1 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 40(%r14),%r13 + vpxor %xmm2,%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm3,%xmm8,%xmm2 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 32(%r14),%r12 + vpclmulqdq $0x11,%xmm3,%xmm8,%xmm8 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,80+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,88+8(%rsp) + vpxor %xmm5,%xmm6,%xmm6 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor %xmm1,%xmm6,%xmm6 + + vmovups 112-128(%rcx),%xmm15 + vpslldq $8,%xmm6,%xmm5 + vpxor %xmm2,%xmm4,%xmm4 + vmovdqu 16(%r11),%xmm3 + + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm8,%xmm7,%xmm7 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor %xmm5,%xmm4,%xmm4 + movbeq 24(%r14),%r13 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 16(%r14),%r12 + vpalignr $8,%xmm4,%xmm4,%xmm0 + vpclmulqdq $0x10,%xmm3,%xmm4,%xmm4 + movq %r13,96+8(%rsp) + vaesenc %xmm15,%xmm12,%xmm12 + movq %r12,104+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + vmovups 128-128(%rcx),%xmm1 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vmovups 144-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm10,%xmm10 + vpsrldq $8,%xmm6,%xmm6 + vaesenc %xmm1,%xmm11,%xmm11 + vpxor %xmm6,%xmm7,%xmm7 + vaesenc %xmm1,%xmm12,%xmm12 + vpxor %xmm0,%xmm4,%xmm4 + movbeq 8(%r14),%r13 + vaesenc %xmm1,%xmm13,%xmm13 + movbeq 0(%r14),%r12 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 160-128(%rcx),%xmm1 + cmpl $11,%r10d + jb .Lenc_tail + + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vaesenc %xmm1,%xmm10,%xmm10 + vaesenc %xmm1,%xmm11,%xmm11 + vaesenc %xmm1,%xmm12,%xmm12 + vaesenc %xmm1,%xmm13,%xmm13 + vmovups 176-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 192-128(%rcx),%xmm1 + je .Lenc_tail + + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vaesenc %xmm1,%xmm10,%xmm10 + vaesenc %xmm1,%xmm11,%xmm11 + vaesenc %xmm1,%xmm12,%xmm12 + vaesenc %xmm1,%xmm13,%xmm13 + vmovups 208-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 224-128(%rcx),%xmm1 + jmp .Lenc_tail + +.align 32 +.Lhandle_ctr32: + vmovdqu (%r11),%xmm0 + vpshufb %xmm0,%xmm1,%xmm6 + vmovdqu 48(%r11),%xmm5 + vpaddd 64(%r11),%xmm6,%xmm10 + vpaddd %xmm5,%xmm6,%xmm11 + vmovdqu 0-32(%r9),%xmm3 + vpaddd %xmm5,%xmm10,%xmm12 + vpshufb %xmm0,%xmm10,%xmm10 + vpaddd %xmm5,%xmm11,%xmm13 + vpshufb %xmm0,%xmm11,%xmm11 + vpxor %xmm15,%xmm10,%xmm10 + vpaddd %xmm5,%xmm12,%xmm14 + vpshufb %xmm0,%xmm12,%xmm12 + vpxor %xmm15,%xmm11,%xmm11 + vpaddd %xmm5,%xmm13,%xmm1 + vpshufb %xmm0,%xmm13,%xmm13 + vpshufb %xmm0,%xmm14,%xmm14 + vpshufb %xmm0,%xmm1,%xmm1 + jmp .Lresume_ctr32 + +.align 32 +.Lenc_tail: + vaesenc %xmm15,%xmm9,%xmm9 + vmovdqu %xmm7,16+8(%rsp) + vpalignr $8,%xmm4,%xmm4,%xmm8 + vaesenc %xmm15,%xmm10,%xmm10 + vpclmulqdq $0x10,%xmm3,%xmm4,%xmm4 + vpxor 0(%rdi),%xmm1,%xmm2 + vaesenc %xmm15,%xmm11,%xmm11 + vpxor 16(%rdi),%xmm1,%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + vpxor 32(%rdi),%xmm1,%xmm5 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor 48(%rdi),%xmm1,%xmm6 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor 64(%rdi),%xmm1,%xmm7 + vpxor 80(%rdi),%xmm1,%xmm3 + vmovdqu (%r8),%xmm1 + + vaesenclast %xmm2,%xmm9,%xmm9 + vmovdqu 32(%r11),%xmm2 + vaesenclast %xmm0,%xmm10,%xmm10 + vpaddb %xmm2,%xmm1,%xmm0 + movq %r13,112+8(%rsp) + leaq 96(%rdi),%rdi + vaesenclast %xmm5,%xmm11,%xmm11 + vpaddb %xmm2,%xmm0,%xmm5 + movq %r12,120+8(%rsp) + leaq 96(%rsi),%rsi + vmovdqu 0-128(%rcx),%xmm15 + vaesenclast %xmm6,%xmm12,%xmm12 + vpaddb %xmm2,%xmm5,%xmm6 + vaesenclast %xmm7,%xmm13,%xmm13 + vpaddb %xmm2,%xmm6,%xmm7 + vaesenclast %xmm3,%xmm14,%xmm14 + vpaddb %xmm2,%xmm7,%xmm3 + + addq $0x60,%rax + subq $0x6,%rdx + jc .L6x_done + + vmovups %xmm9,-96(%rsi) + vpxor %xmm15,%xmm1,%xmm9 + vmovups %xmm10,-80(%rsi) + vmovdqa %xmm0,%xmm10 + vmovups %xmm11,-64(%rsi) + vmovdqa %xmm5,%xmm11 + vmovups %xmm12,-48(%rsi) + vmovdqa %xmm6,%xmm12 + vmovups %xmm13,-32(%rsi) + vmovdqa %xmm7,%xmm13 + vmovups %xmm14,-16(%rsi) + vmovdqa %xmm3,%xmm14 + vmovdqu 32+8(%rsp),%xmm7 + jmp .Loop6x + +.L6x_done: + vpxor 16+8(%rsp),%xmm8,%xmm8 + vpxor %xmm4,%xmm8,%xmm8 + + .byte 0xf3,0xc3 +.cfi_endproc +.size _crypton_gcm_asm_ctr32_ghash_6x,.-_crypton_gcm_asm_ctr32_ghash_6x +.globl crypton_gcm_asm_decrypt +.type crypton_gcm_asm_decrypt,@function +.align 32 +crypton_gcm_asm_decrypt: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + xorq %rax,%rax + cmpq $0x60,%rdx + jb .Lgcm_dec_abort + + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + + vzeroupper + + vmovdqu (%r8),%xmm1 + addq $-128,%rsp + movl 12(%r8),%ebx + leaq .Lbswap_mask(%rip),%r11 + leaq -128(%rcx),%r14 + movq $0xf80,%r15 + vmovdqu (%r9),%xmm8 + andq $-128,%rsp + vmovdqu (%r11),%xmm0 + leaq 128(%rcx),%rcx + leaq 32+32(%r9),%r9 + movl 240-128(%rcx),%r10d + vpshufb %xmm0,%xmm8,%xmm8 + + andq %r15,%r14 + andq %rsp,%r15 + subq %r14,%r15 + jc .Ldec_no_key_aliasing + cmpq $768,%r15 + jnc .Ldec_no_key_aliasing + subq %r15,%rsp +.Ldec_no_key_aliasing: + + vmovdqu 80(%rdi),%xmm7 + leaq (%rdi),%r14 + vmovdqu 64(%rdi),%xmm4 + leaq -192(%rdi,%rdx,1),%r15 + vmovdqu 48(%rdi),%xmm5 + shrq $4,%rdx + xorq %rax,%rax + vmovdqu 32(%rdi),%xmm6 + vpshufb %xmm0,%xmm7,%xmm7 + vmovdqu 16(%rdi),%xmm2 + vpshufb %xmm0,%xmm4,%xmm4 + vmovdqu (%rdi),%xmm3 + vpshufb %xmm0,%xmm5,%xmm5 + vmovdqu %xmm4,48(%rsp) + vpshufb %xmm0,%xmm6,%xmm6 + vmovdqu %xmm5,64(%rsp) + vpshufb %xmm0,%xmm2,%xmm2 + vmovdqu %xmm6,80(%rsp) + vpshufb %xmm0,%xmm3,%xmm3 + vmovdqu %xmm2,96(%rsp) + vmovdqu %xmm3,112(%rsp) + + call _crypton_gcm_asm_ctr32_ghash_6x + + vmovups %xmm9,-96(%rsi) + vmovups %xmm10,-80(%rsi) + vmovups %xmm11,-64(%rsi) + vmovups %xmm12,-48(%rsi) + vmovups %xmm13,-32(%rsi) + vmovups %xmm14,-16(%rsi) + + vpshufb (%r11),%xmm8,%xmm8 + vmovdqu %xmm8,-64(%r9) + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp +.Lgcm_dec_abort: + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_gcm_asm_decrypt,.-crypton_gcm_asm_decrypt +.type _crypton_gcm_asm_ctr32_6x,@function +.align 32 +_crypton_gcm_asm_ctr32_6x: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + vmovdqu 0-128(%rcx),%xmm4 + vmovdqu 32(%r11),%xmm2 + leaq -1(%r10),%r13 + vmovups 16-128(%rcx),%xmm15 + leaq 32-128(%rcx),%r12 + vpxor %xmm4,%xmm1,%xmm9 + addl $100663296,%ebx + jc .Lhandle_ctr32_2 + vpaddb %xmm2,%xmm1,%xmm10 + vpaddb %xmm2,%xmm10,%xmm11 + vpxor %xmm4,%xmm10,%xmm10 + vpaddb %xmm2,%xmm11,%xmm12 + vpxor %xmm4,%xmm11,%xmm11 + vpaddb %xmm2,%xmm12,%xmm13 + vpxor %xmm4,%xmm12,%xmm12 + vpaddb %xmm2,%xmm13,%xmm14 + vpxor %xmm4,%xmm13,%xmm13 + vpaddb %xmm2,%xmm14,%xmm1 + vpxor %xmm4,%xmm14,%xmm14 + jmp .Loop_ctr32 + +.align 16 +.Loop_ctr32: + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + vmovups (%r12),%xmm15 + leaq 16(%r12),%r12 + decl %r13d + jnz .Loop_ctr32 + + vmovdqu (%r12),%xmm3 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor 0(%rdi),%xmm3,%xmm4 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor 16(%rdi),%xmm3,%xmm5 + vaesenc %xmm15,%xmm11,%xmm11 + vpxor 32(%rdi),%xmm3,%xmm6 + vaesenc %xmm15,%xmm12,%xmm12 + vpxor 48(%rdi),%xmm3,%xmm8 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor 64(%rdi),%xmm3,%xmm2 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor 80(%rdi),%xmm3,%xmm3 + leaq 96(%rdi),%rdi + + vaesenclast %xmm4,%xmm9,%xmm9 + vaesenclast %xmm5,%xmm10,%xmm10 + vaesenclast %xmm6,%xmm11,%xmm11 + vaesenclast %xmm8,%xmm12,%xmm12 + vaesenclast %xmm2,%xmm13,%xmm13 + vaesenclast %xmm3,%xmm14,%xmm14 + vmovups %xmm9,0(%rsi) + vmovups %xmm10,16(%rsi) + vmovups %xmm11,32(%rsi) + vmovups %xmm12,48(%rsi) + vmovups %xmm13,64(%rsi) + vmovups %xmm14,80(%rsi) + leaq 96(%rsi),%rsi + + .byte 0xf3,0xc3 +.align 32 +.Lhandle_ctr32_2: + vpshufb %xmm0,%xmm1,%xmm6 + vmovdqu 48(%r11),%xmm5 + vpaddd 64(%r11),%xmm6,%xmm10 + vpaddd %xmm5,%xmm6,%xmm11 + vpaddd %xmm5,%xmm10,%xmm12 + vpshufb %xmm0,%xmm10,%xmm10 + vpaddd %xmm5,%xmm11,%xmm13 + vpshufb %xmm0,%xmm11,%xmm11 + vpxor %xmm4,%xmm10,%xmm10 + vpaddd %xmm5,%xmm12,%xmm14 + vpshufb %xmm0,%xmm12,%xmm12 + vpxor %xmm4,%xmm11,%xmm11 + vpaddd %xmm5,%xmm13,%xmm1 + vpshufb %xmm0,%xmm13,%xmm13 + vpxor %xmm4,%xmm12,%xmm12 + vpshufb %xmm0,%xmm14,%xmm14 + vpxor %xmm4,%xmm13,%xmm13 + vpshufb %xmm0,%xmm1,%xmm1 + vpxor %xmm4,%xmm14,%xmm14 + jmp .Loop_ctr32 +.cfi_endproc +.size _crypton_gcm_asm_ctr32_6x,.-_crypton_gcm_asm_ctr32_6x + +.globl crypton_gcm_asm_encrypt +.type crypton_gcm_asm_encrypt,@function +.align 32 +crypton_gcm_asm_encrypt: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + xorq %rax,%rax + cmpq $288,%rdx + jb .Lgcm_enc_abort + + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + + vzeroupper + + vmovdqu (%r8),%xmm1 + addq $-128,%rsp + movl 12(%r8),%ebx + leaq .Lbswap_mask(%rip),%r11 + leaq -128(%rcx),%r14 + movq $0xf80,%r15 + leaq 128(%rcx),%rcx + vmovdqu (%r11),%xmm0 + andq $-128,%rsp + movl 240-128(%rcx),%r10d + + andq %r15,%r14 + andq %rsp,%r15 + subq %r14,%r15 + jc .Lenc_no_key_aliasing + cmpq $768,%r15 + jnc .Lenc_no_key_aliasing + subq %r15,%rsp +.Lenc_no_key_aliasing: + + leaq (%rsi),%r14 + leaq -192(%rsi,%rdx,1),%r15 + shrq $4,%rdx + + call _crypton_gcm_asm_ctr32_6x + vpshufb %xmm0,%xmm9,%xmm8 + vpshufb %xmm0,%xmm10,%xmm2 + vmovdqu %xmm8,112(%rsp) + vpshufb %xmm0,%xmm11,%xmm4 + vmovdqu %xmm2,96(%rsp) + vpshufb %xmm0,%xmm12,%xmm5 + vmovdqu %xmm4,80(%rsp) + vpshufb %xmm0,%xmm13,%xmm6 + vmovdqu %xmm5,64(%rsp) + vpshufb %xmm0,%xmm14,%xmm7 + vmovdqu %xmm6,48(%rsp) + + call _crypton_gcm_asm_ctr32_6x + + vmovdqu (%r9),%xmm8 + leaq 32+32(%r9),%r9 + subq $12,%rdx + movq $192,%rax + vpshufb %xmm0,%xmm8,%xmm8 + + call _crypton_gcm_asm_ctr32_ghash_6x + vmovdqu 32(%rsp),%xmm7 + vmovdqu (%r11),%xmm0 + vmovdqu 0-32(%r9),%xmm3 + vpunpckhqdq %xmm7,%xmm7,%xmm1 + vmovdqu 32-32(%r9),%xmm15 + vmovups %xmm9,-96(%rsi) + vpshufb %xmm0,%xmm9,%xmm9 + vpxor %xmm7,%xmm1,%xmm1 + vmovups %xmm10,-80(%rsi) + vpshufb %xmm0,%xmm10,%xmm10 + vmovups %xmm11,-64(%rsi) + vpshufb %xmm0,%xmm11,%xmm11 + vmovups %xmm12,-48(%rsi) + vpshufb %xmm0,%xmm12,%xmm12 + vmovups %xmm13,-32(%rsi) + vpshufb %xmm0,%xmm13,%xmm13 + vmovups %xmm14,-16(%rsi) + vpshufb %xmm0,%xmm14,%xmm14 + vmovdqu %xmm9,16(%rsp) + vmovdqu 48(%rsp),%xmm6 + vmovdqu 16-32(%r9),%xmm0 + vpunpckhqdq %xmm6,%xmm6,%xmm2 + vpclmulqdq $0x00,%xmm3,%xmm7,%xmm5 + vpxor %xmm6,%xmm2,%xmm2 + vpclmulqdq $0x11,%xmm3,%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm15,%xmm1,%xmm1 + + vmovdqu 64(%rsp),%xmm9 + vpclmulqdq $0x00,%xmm0,%xmm6,%xmm4 + vmovdqu 48-32(%r9),%xmm3 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm9,%xmm9,%xmm5 + vpclmulqdq $0x11,%xmm0,%xmm6,%xmm6 + vpxor %xmm9,%xmm5,%xmm5 + vpxor %xmm7,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm15,%xmm2,%xmm2 + vmovdqu 80-32(%r9),%xmm15 + vpxor %xmm1,%xmm2,%xmm2 + + vmovdqu 80(%rsp),%xmm1 + vpclmulqdq $0x00,%xmm3,%xmm9,%xmm7 + vmovdqu 64-32(%r9),%xmm0 + vpxor %xmm4,%xmm7,%xmm7 + vpunpckhqdq %xmm1,%xmm1,%xmm4 + vpclmulqdq $0x11,%xmm3,%xmm9,%xmm9 + vpxor %xmm1,%xmm4,%xmm4 + vpxor %xmm6,%xmm9,%xmm9 + vpclmulqdq $0x00,%xmm15,%xmm5,%xmm5 + vpxor %xmm2,%xmm5,%xmm5 + + vmovdqu 96(%rsp),%xmm2 + vpclmulqdq $0x00,%xmm0,%xmm1,%xmm6 + vmovdqu 96-32(%r9),%xmm3 + vpxor %xmm7,%xmm6,%xmm6 + vpunpckhqdq %xmm2,%xmm2,%xmm7 + vpclmulqdq $0x11,%xmm0,%xmm1,%xmm1 + vpxor %xmm2,%xmm7,%xmm7 + vpxor %xmm9,%xmm1,%xmm1 + vpclmulqdq $0x10,%xmm15,%xmm4,%xmm4 + vmovdqu 128-32(%r9),%xmm15 + vpxor %xmm5,%xmm4,%xmm4 + + vpxor 112(%rsp),%xmm8,%xmm8 + vpclmulqdq $0x00,%xmm3,%xmm2,%xmm5 + vmovdqu 112-32(%r9),%xmm0 + vpunpckhqdq %xmm8,%xmm8,%xmm9 + vpxor %xmm6,%xmm5,%xmm5 + vpclmulqdq $0x11,%xmm3,%xmm2,%xmm2 + vpxor %xmm8,%xmm9,%xmm9 + vpxor %xmm1,%xmm2,%xmm2 + vpclmulqdq $0x00,%xmm15,%xmm7,%xmm7 + vpxor %xmm4,%xmm7,%xmm4 + + vpclmulqdq $0x00,%xmm0,%xmm8,%xmm6 + vmovdqu 0-32(%r9),%xmm3 + vpunpckhqdq %xmm14,%xmm14,%xmm1 + vpclmulqdq $0x11,%xmm0,%xmm8,%xmm8 + vpxor %xmm14,%xmm1,%xmm1 + vpxor %xmm5,%xmm6,%xmm5 + vpclmulqdq $0x10,%xmm15,%xmm9,%xmm9 + vmovdqu 32-32(%r9),%xmm15 + vpxor %xmm2,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm6 + + vmovdqu 16-32(%r9),%xmm0 + vpxor %xmm5,%xmm7,%xmm9 + vpclmulqdq $0x00,%xmm3,%xmm14,%xmm4 + vpxor %xmm9,%xmm6,%xmm6 + vpunpckhqdq %xmm13,%xmm13,%xmm2 + vpclmulqdq $0x11,%xmm3,%xmm14,%xmm14 + vpxor %xmm13,%xmm2,%xmm2 + vpslldq $8,%xmm6,%xmm9 + vpclmulqdq $0x00,%xmm15,%xmm1,%xmm1 + vpxor %xmm9,%xmm5,%xmm8 + vpsrldq $8,%xmm6,%xmm6 + vpxor %xmm6,%xmm7,%xmm7 + + vpclmulqdq $0x00,%xmm0,%xmm13,%xmm5 + vmovdqu 48-32(%r9),%xmm3 + vpxor %xmm4,%xmm5,%xmm5 + vpunpckhqdq %xmm12,%xmm12,%xmm9 + vpclmulqdq $0x11,%xmm0,%xmm13,%xmm13 + vpxor %xmm12,%xmm9,%xmm9 + vpxor %xmm14,%xmm13,%xmm13 + vpalignr $8,%xmm8,%xmm8,%xmm14 + vpclmulqdq $0x10,%xmm15,%xmm2,%xmm2 + vmovdqu 80-32(%r9),%xmm15 + vpxor %xmm1,%xmm2,%xmm2 + + vpclmulqdq $0x00,%xmm3,%xmm12,%xmm4 + vmovdqu 64-32(%r9),%xmm0 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm11,%xmm11,%xmm1 + vpclmulqdq $0x11,%xmm3,%xmm12,%xmm12 + vpxor %xmm11,%xmm1,%xmm1 + vpxor %xmm13,%xmm12,%xmm12 + vxorps 16(%rsp),%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm15,%xmm9,%xmm9 + vpxor %xmm2,%xmm9,%xmm9 + + vpclmulqdq $0x10,16(%r11),%xmm8,%xmm8 + vxorps %xmm14,%xmm8,%xmm8 + + vpclmulqdq $0x00,%xmm0,%xmm11,%xmm5 + vmovdqu 96-32(%r9),%xmm3 + vpxor %xmm4,%xmm5,%xmm5 + vpunpckhqdq %xmm10,%xmm10,%xmm2 + vpclmulqdq $0x11,%xmm0,%xmm11,%xmm11 + vpxor %xmm10,%xmm2,%xmm2 + vpalignr $8,%xmm8,%xmm8,%xmm14 + vpxor %xmm12,%xmm11,%xmm11 + vpclmulqdq $0x10,%xmm15,%xmm1,%xmm1 + vmovdqu 128-32(%r9),%xmm15 + vpxor %xmm9,%xmm1,%xmm1 + + vxorps %xmm7,%xmm14,%xmm14 + vpclmulqdq $0x10,16(%r11),%xmm8,%xmm8 + vxorps %xmm14,%xmm8,%xmm8 + + vpclmulqdq $0x00,%xmm3,%xmm10,%xmm4 + vmovdqu 112-32(%r9),%xmm0 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm8,%xmm8,%xmm9 + vpclmulqdq $0x11,%xmm3,%xmm10,%xmm10 + vpxor %xmm8,%xmm9,%xmm9 + vpxor %xmm11,%xmm10,%xmm10 + vpclmulqdq $0x00,%xmm15,%xmm2,%xmm2 + vpxor %xmm1,%xmm2,%xmm2 + + vpclmulqdq $0x00,%xmm0,%xmm8,%xmm5 + vpclmulqdq $0x11,%xmm0,%xmm8,%xmm7 + vpxor %xmm4,%xmm5,%xmm5 + vpclmulqdq $0x10,%xmm15,%xmm9,%xmm6 + vpxor %xmm10,%xmm7,%xmm7 + vpxor %xmm2,%xmm6,%xmm6 + + vpxor %xmm5,%xmm7,%xmm4 + vpxor %xmm4,%xmm6,%xmm6 + vpslldq $8,%xmm6,%xmm1 + vmovdqu 16(%r11),%xmm3 + vpsrldq $8,%xmm6,%xmm6 + vpxor %xmm1,%xmm5,%xmm8 + vpxor %xmm6,%xmm7,%xmm7 + + vpalignr $8,%xmm8,%xmm8,%xmm2 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm8 + vpxor %xmm2,%xmm8,%xmm8 + + vpalignr $8,%xmm8,%xmm8,%xmm2 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm8 + vpxor %xmm7,%xmm2,%xmm2 + vpxor %xmm2,%xmm8,%xmm8 + vpshufb (%r11),%xmm8,%xmm8 + vmovdqu %xmm8,-64(%r9) + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp +.Lgcm_enc_abort: + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_gcm_asm_encrypt,.-crypton_gcm_asm_encrypt +.align 64 +.Lbswap_mask: +.byte 15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0 +.Lpoly: +.byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2 +.Lone_msb: +.byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1 +.Ltwo_lsb: +.byte 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.Lone_lsb: +.byte 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.byte 65,69,83,45,78,73,32,71,67,77,32,109,111,100,117,108,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 +.align 64 + +.section .note.gnu.property,"a",@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align 8 +2: + +.section .note.GNU-stack,"",@progbits diff --git a/cbits/asm/aesni-gcm-x86_64-macosx.S b/cbits/asm/aesni-gcm-x86_64-macosx.S new file mode 100644 --- /dev/null +++ b/cbits/asm/aesni-gcm-x86_64-macosx.S @@ -0,0 +1,805 @@ +.text + + +.p2align 5 +_crypton_gcm_asm_ctr32_ghash_6x: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + vmovdqu 32(%r11),%xmm2 + subq $6,%rdx + vpxor %xmm4,%xmm4,%xmm4 + vmovdqu 0-128(%rcx),%xmm15 + vpaddb %xmm2,%xmm1,%xmm10 + vpaddb %xmm2,%xmm10,%xmm11 + vpaddb %xmm2,%xmm11,%xmm12 + vpaddb %xmm2,%xmm12,%xmm13 + vpaddb %xmm2,%xmm13,%xmm14 + vpxor %xmm15,%xmm1,%xmm9 + vmovdqu %xmm4,16+8(%rsp) + jmp L$oop6x + +.p2align 5 +L$oop6x: + addl $100663296,%ebx + jc L$handle_ctr32 + vmovdqu 0-32(%r9),%xmm3 + vpaddb %xmm2,%xmm14,%xmm1 + vpxor %xmm15,%xmm10,%xmm10 + vpxor %xmm15,%xmm11,%xmm11 + +L$resume_ctr32: + vmovdqu %xmm1,(%r8) + vpclmulqdq $0x10,%xmm3,%xmm7,%xmm5 + vpxor %xmm15,%xmm12,%xmm12 + vmovups 16-128(%rcx),%xmm2 + vpclmulqdq $0x01,%xmm3,%xmm7,%xmm6 + xorq %r12,%r12 + cmpq %r14,%r15 + + vaesenc %xmm2,%xmm9,%xmm9 + vmovdqu 48+8(%rsp),%xmm0 + vpxor %xmm15,%xmm13,%xmm13 + vpclmulqdq $0x00,%xmm3,%xmm7,%xmm1 + vaesenc %xmm2,%xmm10,%xmm10 + vpxor %xmm15,%xmm14,%xmm14 + setnc %r12b + vpclmulqdq $0x11,%xmm3,%xmm7,%xmm7 + vaesenc %xmm2,%xmm11,%xmm11 + vmovdqu 16-32(%r9),%xmm3 + negq %r12 + vaesenc %xmm2,%xmm12,%xmm12 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm3,%xmm0,%xmm5 + vpxor %xmm4,%xmm8,%xmm8 + vaesenc %xmm2,%xmm13,%xmm13 + vpxor %xmm5,%xmm1,%xmm4 + andq $0x60,%r12 + vmovups 32-128(%rcx),%xmm15 + vpclmulqdq $0x10,%xmm3,%xmm0,%xmm1 + vaesenc %xmm2,%xmm14,%xmm14 + + vpclmulqdq $0x01,%xmm3,%xmm0,%xmm2 + leaq (%r14,%r12,1),%r14 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor 16+8(%rsp),%xmm8,%xmm8 + vpclmulqdq $0x11,%xmm3,%xmm0,%xmm3 + vmovdqu 64+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 88(%r14),%r13 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 80(%r14),%r12 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,32+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,40+8(%rsp) + vmovdqu 48-32(%r9),%xmm5 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 48-128(%rcx),%xmm15 + vpxor %xmm1,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm5,%xmm0,%xmm1 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm2,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm5,%xmm0,%xmm2 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor %xmm3,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm5,%xmm0,%xmm3 + vaesenc %xmm15,%xmm11,%xmm11 + vpclmulqdq $0x11,%xmm5,%xmm0,%xmm5 + vmovdqu 80+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor %xmm1,%xmm4,%xmm4 + vmovdqu 64-32(%r9),%xmm1 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 64-128(%rcx),%xmm15 + vpxor %xmm2,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm1,%xmm0,%xmm2 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm3,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm1,%xmm0,%xmm3 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 72(%r14),%r13 + vpxor %xmm5,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm1,%xmm0,%xmm5 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 64(%r14),%r12 + vpclmulqdq $0x11,%xmm1,%xmm0,%xmm1 + vmovdqu 96+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,48+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,56+8(%rsp) + vpxor %xmm2,%xmm4,%xmm4 + vmovdqu 96-32(%r9),%xmm2 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 80-128(%rcx),%xmm15 + vpxor %xmm3,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm2,%xmm0,%xmm3 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm2,%xmm0,%xmm5 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 56(%r14),%r13 + vpxor %xmm1,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm2,%xmm0,%xmm1 + vpxor 112+8(%rsp),%xmm8,%xmm8 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 48(%r14),%r12 + vpclmulqdq $0x11,%xmm2,%xmm0,%xmm2 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,64+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,72+8(%rsp) + vpxor %xmm3,%xmm4,%xmm4 + vmovdqu 112-32(%r9),%xmm3 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 96-128(%rcx),%xmm15 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm5 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm1,%xmm6,%xmm6 + vpclmulqdq $0x01,%xmm3,%xmm8,%xmm1 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 40(%r14),%r13 + vpxor %xmm2,%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm3,%xmm8,%xmm2 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 32(%r14),%r12 + vpclmulqdq $0x11,%xmm3,%xmm8,%xmm8 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,80+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,88+8(%rsp) + vpxor %xmm5,%xmm6,%xmm6 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor %xmm1,%xmm6,%xmm6 + + vmovups 112-128(%rcx),%xmm15 + vpslldq $8,%xmm6,%xmm5 + vpxor %xmm2,%xmm4,%xmm4 + vmovdqu 16(%r11),%xmm3 + + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm8,%xmm7,%xmm7 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor %xmm5,%xmm4,%xmm4 + movbeq 24(%r14),%r13 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 16(%r14),%r12 + vpalignr $8,%xmm4,%xmm4,%xmm0 + vpclmulqdq $0x10,%xmm3,%xmm4,%xmm4 + movq %r13,96+8(%rsp) + vaesenc %xmm15,%xmm12,%xmm12 + movq %r12,104+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + vmovups 128-128(%rcx),%xmm1 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vmovups 144-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm10,%xmm10 + vpsrldq $8,%xmm6,%xmm6 + vaesenc %xmm1,%xmm11,%xmm11 + vpxor %xmm6,%xmm7,%xmm7 + vaesenc %xmm1,%xmm12,%xmm12 + vpxor %xmm0,%xmm4,%xmm4 + movbeq 8(%r14),%r13 + vaesenc %xmm1,%xmm13,%xmm13 + movbeq 0(%r14),%r12 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 160-128(%rcx),%xmm1 + cmpl $11,%r10d + jb L$enc_tail + + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vaesenc %xmm1,%xmm10,%xmm10 + vaesenc %xmm1,%xmm11,%xmm11 + vaesenc %xmm1,%xmm12,%xmm12 + vaesenc %xmm1,%xmm13,%xmm13 + vmovups 176-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 192-128(%rcx),%xmm1 + je L$enc_tail + + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vaesenc %xmm1,%xmm10,%xmm10 + vaesenc %xmm1,%xmm11,%xmm11 + vaesenc %xmm1,%xmm12,%xmm12 + vaesenc %xmm1,%xmm13,%xmm13 + vmovups 208-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 224-128(%rcx),%xmm1 + jmp L$enc_tail + +.p2align 5 +L$handle_ctr32: + vmovdqu (%r11),%xmm0 + vpshufb %xmm0,%xmm1,%xmm6 + vmovdqu 48(%r11),%xmm5 + vpaddd 64(%r11),%xmm6,%xmm10 + vpaddd %xmm5,%xmm6,%xmm11 + vmovdqu 0-32(%r9),%xmm3 + vpaddd %xmm5,%xmm10,%xmm12 + vpshufb %xmm0,%xmm10,%xmm10 + vpaddd %xmm5,%xmm11,%xmm13 + vpshufb %xmm0,%xmm11,%xmm11 + vpxor %xmm15,%xmm10,%xmm10 + vpaddd %xmm5,%xmm12,%xmm14 + vpshufb %xmm0,%xmm12,%xmm12 + vpxor %xmm15,%xmm11,%xmm11 + vpaddd %xmm5,%xmm13,%xmm1 + vpshufb %xmm0,%xmm13,%xmm13 + vpshufb %xmm0,%xmm14,%xmm14 + vpshufb %xmm0,%xmm1,%xmm1 + jmp L$resume_ctr32 + +.p2align 5 +L$enc_tail: + vaesenc %xmm15,%xmm9,%xmm9 + vmovdqu %xmm7,16+8(%rsp) + vpalignr $8,%xmm4,%xmm4,%xmm8 + vaesenc %xmm15,%xmm10,%xmm10 + vpclmulqdq $0x10,%xmm3,%xmm4,%xmm4 + vpxor 0(%rdi),%xmm1,%xmm2 + vaesenc %xmm15,%xmm11,%xmm11 + vpxor 16(%rdi),%xmm1,%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + vpxor 32(%rdi),%xmm1,%xmm5 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor 48(%rdi),%xmm1,%xmm6 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor 64(%rdi),%xmm1,%xmm7 + vpxor 80(%rdi),%xmm1,%xmm3 + vmovdqu (%r8),%xmm1 + + vaesenclast %xmm2,%xmm9,%xmm9 + vmovdqu 32(%r11),%xmm2 + vaesenclast %xmm0,%xmm10,%xmm10 + vpaddb %xmm2,%xmm1,%xmm0 + movq %r13,112+8(%rsp) + leaq 96(%rdi),%rdi + vaesenclast %xmm5,%xmm11,%xmm11 + vpaddb %xmm2,%xmm0,%xmm5 + movq %r12,120+8(%rsp) + leaq 96(%rsi),%rsi + vmovdqu 0-128(%rcx),%xmm15 + vaesenclast %xmm6,%xmm12,%xmm12 + vpaddb %xmm2,%xmm5,%xmm6 + vaesenclast %xmm7,%xmm13,%xmm13 + vpaddb %xmm2,%xmm6,%xmm7 + vaesenclast %xmm3,%xmm14,%xmm14 + vpaddb %xmm2,%xmm7,%xmm3 + + addq $0x60,%rax + subq $0x6,%rdx + jc L$6x_done + + vmovups %xmm9,-96(%rsi) + vpxor %xmm15,%xmm1,%xmm9 + vmovups %xmm10,-80(%rsi) + vmovdqa %xmm0,%xmm10 + vmovups %xmm11,-64(%rsi) + vmovdqa %xmm5,%xmm11 + vmovups %xmm12,-48(%rsi) + vmovdqa %xmm6,%xmm12 + vmovups %xmm13,-32(%rsi) + vmovdqa %xmm7,%xmm13 + vmovups %xmm14,-16(%rsi) + vmovdqa %xmm3,%xmm14 + vmovdqu 32+8(%rsp),%xmm7 + jmp L$oop6x + +L$6x_done: + vpxor 16+8(%rsp),%xmm8,%xmm8 + vpxor %xmm4,%xmm8,%xmm8 + + .byte 0xf3,0xc3 +.cfi_endproc + +.globl _crypton_gcm_asm_decrypt + +.p2align 5 +_crypton_gcm_asm_decrypt: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + xorq %rax,%rax + cmpq $0x60,%rdx + jb L$gcm_dec_abort + + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + + vzeroupper + + vmovdqu (%r8),%xmm1 + addq $-128,%rsp + movl 12(%r8),%ebx + leaq L$bswap_mask(%rip),%r11 + leaq -128(%rcx),%r14 + movq $0xf80,%r15 + vmovdqu (%r9),%xmm8 + andq $-128,%rsp + vmovdqu (%r11),%xmm0 + leaq 128(%rcx),%rcx + leaq 32+32(%r9),%r9 + movl 240-128(%rcx),%r10d + vpshufb %xmm0,%xmm8,%xmm8 + + andq %r15,%r14 + andq %rsp,%r15 + subq %r14,%r15 + jc L$dec_no_key_aliasing + cmpq $768,%r15 + jnc L$dec_no_key_aliasing + subq %r15,%rsp +L$dec_no_key_aliasing: + + vmovdqu 80(%rdi),%xmm7 + leaq (%rdi),%r14 + vmovdqu 64(%rdi),%xmm4 + leaq -192(%rdi,%rdx,1),%r15 + vmovdqu 48(%rdi),%xmm5 + shrq $4,%rdx + xorq %rax,%rax + vmovdqu 32(%rdi),%xmm6 + vpshufb %xmm0,%xmm7,%xmm7 + vmovdqu 16(%rdi),%xmm2 + vpshufb %xmm0,%xmm4,%xmm4 + vmovdqu (%rdi),%xmm3 + vpshufb %xmm0,%xmm5,%xmm5 + vmovdqu %xmm4,48(%rsp) + vpshufb %xmm0,%xmm6,%xmm6 + vmovdqu %xmm5,64(%rsp) + vpshufb %xmm0,%xmm2,%xmm2 + vmovdqu %xmm6,80(%rsp) + vpshufb %xmm0,%xmm3,%xmm3 + vmovdqu %xmm2,96(%rsp) + vmovdqu %xmm3,112(%rsp) + + call _crypton_gcm_asm_ctr32_ghash_6x + + vmovups %xmm9,-96(%rsi) + vmovups %xmm10,-80(%rsi) + vmovups %xmm11,-64(%rsi) + vmovups %xmm12,-48(%rsi) + vmovups %xmm13,-32(%rsi) + vmovups %xmm14,-16(%rsi) + + vpshufb (%r11),%xmm8,%xmm8 + vmovdqu %xmm8,-64(%r9) + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp +L$gcm_dec_abort: + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +_crypton_gcm_asm_ctr32_6x: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + vmovdqu 0-128(%rcx),%xmm4 + vmovdqu 32(%r11),%xmm2 + leaq -1(%r10),%r13 + vmovups 16-128(%rcx),%xmm15 + leaq 32-128(%rcx),%r12 + vpxor %xmm4,%xmm1,%xmm9 + addl $100663296,%ebx + jc L$handle_ctr32_2 + vpaddb %xmm2,%xmm1,%xmm10 + vpaddb %xmm2,%xmm10,%xmm11 + vpxor %xmm4,%xmm10,%xmm10 + vpaddb %xmm2,%xmm11,%xmm12 + vpxor %xmm4,%xmm11,%xmm11 + vpaddb %xmm2,%xmm12,%xmm13 + vpxor %xmm4,%xmm12,%xmm12 + vpaddb %xmm2,%xmm13,%xmm14 + vpxor %xmm4,%xmm13,%xmm13 + vpaddb %xmm2,%xmm14,%xmm1 + vpxor %xmm4,%xmm14,%xmm14 + jmp L$oop_ctr32 + +.p2align 4 +L$oop_ctr32: + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + vmovups (%r12),%xmm15 + leaq 16(%r12),%r12 + decl %r13d + jnz L$oop_ctr32 + + vmovdqu (%r12),%xmm3 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor 0(%rdi),%xmm3,%xmm4 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor 16(%rdi),%xmm3,%xmm5 + vaesenc %xmm15,%xmm11,%xmm11 + vpxor 32(%rdi),%xmm3,%xmm6 + vaesenc %xmm15,%xmm12,%xmm12 + vpxor 48(%rdi),%xmm3,%xmm8 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor 64(%rdi),%xmm3,%xmm2 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor 80(%rdi),%xmm3,%xmm3 + leaq 96(%rdi),%rdi + + vaesenclast %xmm4,%xmm9,%xmm9 + vaesenclast %xmm5,%xmm10,%xmm10 + vaesenclast %xmm6,%xmm11,%xmm11 + vaesenclast %xmm8,%xmm12,%xmm12 + vaesenclast %xmm2,%xmm13,%xmm13 + vaesenclast %xmm3,%xmm14,%xmm14 + vmovups %xmm9,0(%rsi) + vmovups %xmm10,16(%rsi) + vmovups %xmm11,32(%rsi) + vmovups %xmm12,48(%rsi) + vmovups %xmm13,64(%rsi) + vmovups %xmm14,80(%rsi) + leaq 96(%rsi),%rsi + + .byte 0xf3,0xc3 +.p2align 5 +L$handle_ctr32_2: + vpshufb %xmm0,%xmm1,%xmm6 + vmovdqu 48(%r11),%xmm5 + vpaddd 64(%r11),%xmm6,%xmm10 + vpaddd %xmm5,%xmm6,%xmm11 + vpaddd %xmm5,%xmm10,%xmm12 + vpshufb %xmm0,%xmm10,%xmm10 + vpaddd %xmm5,%xmm11,%xmm13 + vpshufb %xmm0,%xmm11,%xmm11 + vpxor %xmm4,%xmm10,%xmm10 + vpaddd %xmm5,%xmm12,%xmm14 + vpshufb %xmm0,%xmm12,%xmm12 + vpxor %xmm4,%xmm11,%xmm11 + vpaddd %xmm5,%xmm13,%xmm1 + vpshufb %xmm0,%xmm13,%xmm13 + vpxor %xmm4,%xmm12,%xmm12 + vpshufb %xmm0,%xmm14,%xmm14 + vpxor %xmm4,%xmm13,%xmm13 + vpshufb %xmm0,%xmm1,%xmm1 + vpxor %xmm4,%xmm14,%xmm14 + jmp L$oop_ctr32 +.cfi_endproc + + +.globl _crypton_gcm_asm_encrypt + +.p2align 5 +_crypton_gcm_asm_encrypt: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + xorq %rax,%rax + cmpq $288,%rdx + jb L$gcm_enc_abort + + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + + vzeroupper + + vmovdqu (%r8),%xmm1 + addq $-128,%rsp + movl 12(%r8),%ebx + leaq L$bswap_mask(%rip),%r11 + leaq -128(%rcx),%r14 + movq $0xf80,%r15 + leaq 128(%rcx),%rcx + vmovdqu (%r11),%xmm0 + andq $-128,%rsp + movl 240-128(%rcx),%r10d + + andq %r15,%r14 + andq %rsp,%r15 + subq %r14,%r15 + jc L$enc_no_key_aliasing + cmpq $768,%r15 + jnc L$enc_no_key_aliasing + subq %r15,%rsp +L$enc_no_key_aliasing: + + leaq (%rsi),%r14 + leaq -192(%rsi,%rdx,1),%r15 + shrq $4,%rdx + + call _crypton_gcm_asm_ctr32_6x + vpshufb %xmm0,%xmm9,%xmm8 + vpshufb %xmm0,%xmm10,%xmm2 + vmovdqu %xmm8,112(%rsp) + vpshufb %xmm0,%xmm11,%xmm4 + vmovdqu %xmm2,96(%rsp) + vpshufb %xmm0,%xmm12,%xmm5 + vmovdqu %xmm4,80(%rsp) + vpshufb %xmm0,%xmm13,%xmm6 + vmovdqu %xmm5,64(%rsp) + vpshufb %xmm0,%xmm14,%xmm7 + vmovdqu %xmm6,48(%rsp) + + call _crypton_gcm_asm_ctr32_6x + + vmovdqu (%r9),%xmm8 + leaq 32+32(%r9),%r9 + subq $12,%rdx + movq $192,%rax + vpshufb %xmm0,%xmm8,%xmm8 + + call _crypton_gcm_asm_ctr32_ghash_6x + vmovdqu 32(%rsp),%xmm7 + vmovdqu (%r11),%xmm0 + vmovdqu 0-32(%r9),%xmm3 + vpunpckhqdq %xmm7,%xmm7,%xmm1 + vmovdqu 32-32(%r9),%xmm15 + vmovups %xmm9,-96(%rsi) + vpshufb %xmm0,%xmm9,%xmm9 + vpxor %xmm7,%xmm1,%xmm1 + vmovups %xmm10,-80(%rsi) + vpshufb %xmm0,%xmm10,%xmm10 + vmovups %xmm11,-64(%rsi) + vpshufb %xmm0,%xmm11,%xmm11 + vmovups %xmm12,-48(%rsi) + vpshufb %xmm0,%xmm12,%xmm12 + vmovups %xmm13,-32(%rsi) + vpshufb %xmm0,%xmm13,%xmm13 + vmovups %xmm14,-16(%rsi) + vpshufb %xmm0,%xmm14,%xmm14 + vmovdqu %xmm9,16(%rsp) + vmovdqu 48(%rsp),%xmm6 + vmovdqu 16-32(%r9),%xmm0 + vpunpckhqdq %xmm6,%xmm6,%xmm2 + vpclmulqdq $0x00,%xmm3,%xmm7,%xmm5 + vpxor %xmm6,%xmm2,%xmm2 + vpclmulqdq $0x11,%xmm3,%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm15,%xmm1,%xmm1 + + vmovdqu 64(%rsp),%xmm9 + vpclmulqdq $0x00,%xmm0,%xmm6,%xmm4 + vmovdqu 48-32(%r9),%xmm3 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm9,%xmm9,%xmm5 + vpclmulqdq $0x11,%xmm0,%xmm6,%xmm6 + vpxor %xmm9,%xmm5,%xmm5 + vpxor %xmm7,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm15,%xmm2,%xmm2 + vmovdqu 80-32(%r9),%xmm15 + vpxor %xmm1,%xmm2,%xmm2 + + vmovdqu 80(%rsp),%xmm1 + vpclmulqdq $0x00,%xmm3,%xmm9,%xmm7 + vmovdqu 64-32(%r9),%xmm0 + vpxor %xmm4,%xmm7,%xmm7 + vpunpckhqdq %xmm1,%xmm1,%xmm4 + vpclmulqdq $0x11,%xmm3,%xmm9,%xmm9 + vpxor %xmm1,%xmm4,%xmm4 + vpxor %xmm6,%xmm9,%xmm9 + vpclmulqdq $0x00,%xmm15,%xmm5,%xmm5 + vpxor %xmm2,%xmm5,%xmm5 + + vmovdqu 96(%rsp),%xmm2 + vpclmulqdq $0x00,%xmm0,%xmm1,%xmm6 + vmovdqu 96-32(%r9),%xmm3 + vpxor %xmm7,%xmm6,%xmm6 + vpunpckhqdq %xmm2,%xmm2,%xmm7 + vpclmulqdq $0x11,%xmm0,%xmm1,%xmm1 + vpxor %xmm2,%xmm7,%xmm7 + vpxor %xmm9,%xmm1,%xmm1 + vpclmulqdq $0x10,%xmm15,%xmm4,%xmm4 + vmovdqu 128-32(%r9),%xmm15 + vpxor %xmm5,%xmm4,%xmm4 + + vpxor 112(%rsp),%xmm8,%xmm8 + vpclmulqdq $0x00,%xmm3,%xmm2,%xmm5 + vmovdqu 112-32(%r9),%xmm0 + vpunpckhqdq %xmm8,%xmm8,%xmm9 + vpxor %xmm6,%xmm5,%xmm5 + vpclmulqdq $0x11,%xmm3,%xmm2,%xmm2 + vpxor %xmm8,%xmm9,%xmm9 + vpxor %xmm1,%xmm2,%xmm2 + vpclmulqdq $0x00,%xmm15,%xmm7,%xmm7 + vpxor %xmm4,%xmm7,%xmm4 + + vpclmulqdq $0x00,%xmm0,%xmm8,%xmm6 + vmovdqu 0-32(%r9),%xmm3 + vpunpckhqdq %xmm14,%xmm14,%xmm1 + vpclmulqdq $0x11,%xmm0,%xmm8,%xmm8 + vpxor %xmm14,%xmm1,%xmm1 + vpxor %xmm5,%xmm6,%xmm5 + vpclmulqdq $0x10,%xmm15,%xmm9,%xmm9 + vmovdqu 32-32(%r9),%xmm15 + vpxor %xmm2,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm6 + + vmovdqu 16-32(%r9),%xmm0 + vpxor %xmm5,%xmm7,%xmm9 + vpclmulqdq $0x00,%xmm3,%xmm14,%xmm4 + vpxor %xmm9,%xmm6,%xmm6 + vpunpckhqdq %xmm13,%xmm13,%xmm2 + vpclmulqdq $0x11,%xmm3,%xmm14,%xmm14 + vpxor %xmm13,%xmm2,%xmm2 + vpslldq $8,%xmm6,%xmm9 + vpclmulqdq $0x00,%xmm15,%xmm1,%xmm1 + vpxor %xmm9,%xmm5,%xmm8 + vpsrldq $8,%xmm6,%xmm6 + vpxor %xmm6,%xmm7,%xmm7 + + vpclmulqdq $0x00,%xmm0,%xmm13,%xmm5 + vmovdqu 48-32(%r9),%xmm3 + vpxor %xmm4,%xmm5,%xmm5 + vpunpckhqdq %xmm12,%xmm12,%xmm9 + vpclmulqdq $0x11,%xmm0,%xmm13,%xmm13 + vpxor %xmm12,%xmm9,%xmm9 + vpxor %xmm14,%xmm13,%xmm13 + vpalignr $8,%xmm8,%xmm8,%xmm14 + vpclmulqdq $0x10,%xmm15,%xmm2,%xmm2 + vmovdqu 80-32(%r9),%xmm15 + vpxor %xmm1,%xmm2,%xmm2 + + vpclmulqdq $0x00,%xmm3,%xmm12,%xmm4 + vmovdqu 64-32(%r9),%xmm0 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm11,%xmm11,%xmm1 + vpclmulqdq $0x11,%xmm3,%xmm12,%xmm12 + vpxor %xmm11,%xmm1,%xmm1 + vpxor %xmm13,%xmm12,%xmm12 + vxorps 16(%rsp),%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm15,%xmm9,%xmm9 + vpxor %xmm2,%xmm9,%xmm9 + + vpclmulqdq $0x10,16(%r11),%xmm8,%xmm8 + vxorps %xmm14,%xmm8,%xmm8 + + vpclmulqdq $0x00,%xmm0,%xmm11,%xmm5 + vmovdqu 96-32(%r9),%xmm3 + vpxor %xmm4,%xmm5,%xmm5 + vpunpckhqdq %xmm10,%xmm10,%xmm2 + vpclmulqdq $0x11,%xmm0,%xmm11,%xmm11 + vpxor %xmm10,%xmm2,%xmm2 + vpalignr $8,%xmm8,%xmm8,%xmm14 + vpxor %xmm12,%xmm11,%xmm11 + vpclmulqdq $0x10,%xmm15,%xmm1,%xmm1 + vmovdqu 128-32(%r9),%xmm15 + vpxor %xmm9,%xmm1,%xmm1 + + vxorps %xmm7,%xmm14,%xmm14 + vpclmulqdq $0x10,16(%r11),%xmm8,%xmm8 + vxorps %xmm14,%xmm8,%xmm8 + + vpclmulqdq $0x00,%xmm3,%xmm10,%xmm4 + vmovdqu 112-32(%r9),%xmm0 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm8,%xmm8,%xmm9 + vpclmulqdq $0x11,%xmm3,%xmm10,%xmm10 + vpxor %xmm8,%xmm9,%xmm9 + vpxor %xmm11,%xmm10,%xmm10 + vpclmulqdq $0x00,%xmm15,%xmm2,%xmm2 + vpxor %xmm1,%xmm2,%xmm2 + + vpclmulqdq $0x00,%xmm0,%xmm8,%xmm5 + vpclmulqdq $0x11,%xmm0,%xmm8,%xmm7 + vpxor %xmm4,%xmm5,%xmm5 + vpclmulqdq $0x10,%xmm15,%xmm9,%xmm6 + vpxor %xmm10,%xmm7,%xmm7 + vpxor %xmm2,%xmm6,%xmm6 + + vpxor %xmm5,%xmm7,%xmm4 + vpxor %xmm4,%xmm6,%xmm6 + vpslldq $8,%xmm6,%xmm1 + vmovdqu 16(%r11),%xmm3 + vpsrldq $8,%xmm6,%xmm6 + vpxor %xmm1,%xmm5,%xmm8 + vpxor %xmm6,%xmm7,%xmm7 + + vpalignr $8,%xmm8,%xmm8,%xmm2 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm8 + vpxor %xmm2,%xmm8,%xmm8 + + vpalignr $8,%xmm8,%xmm8,%xmm2 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm8 + vpxor %xmm7,%xmm2,%xmm2 + vpxor %xmm2,%xmm8,%xmm8 + vpshufb (%r11),%xmm8,%xmm8 + vmovdqu %xmm8,-64(%r9) + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp +L$gcm_enc_abort: + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + +.p2align 6 +L$bswap_mask: +.byte 15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0 +L$poly: +.byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2 +L$one_msb: +.byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1 +L$two_lsb: +.byte 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +L$one_lsb: +.byte 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.byte 65,69,83,45,78,73,32,71,67,77,32,109,111,100,117,108,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 +.p2align 6 diff --git a/cbits/asm/aesni-gcm-x86_64-mingw64.S b/cbits/asm/aesni-gcm-x86_64-mingw64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/aesni-gcm-x86_64-mingw64.S @@ -0,0 +1,965 @@ +.text + +.def _crypton_gcm_asm_ctr32_ghash_6x; .scl 3; .type 32; .endef +.p2align 5 +_crypton_gcm_asm_ctr32_ghash_6x: + .byte 0xf3,0x0f,0x1e,0xfa + + + vmovdqu 32(%r11),%xmm2 + subq $6,%rdx + vpxor %xmm4,%xmm4,%xmm4 + vmovdqu 0-128(%rcx),%xmm15 + vpaddb %xmm2,%xmm1,%xmm10 + vpaddb %xmm2,%xmm10,%xmm11 + vpaddb %xmm2,%xmm11,%xmm12 + vpaddb %xmm2,%xmm12,%xmm13 + vpaddb %xmm2,%xmm13,%xmm14 + vpxor %xmm15,%xmm1,%xmm9 + vmovdqu %xmm4,16+8(%rsp) + jmp .Loop6x + +.p2align 5 +.Loop6x: + addl $100663296,%ebx + jc .Lhandle_ctr32 + vmovdqu 0-32(%r9),%xmm3 + vpaddb %xmm2,%xmm14,%xmm1 + vpxor %xmm15,%xmm10,%xmm10 + vpxor %xmm15,%xmm11,%xmm11 + +.Lresume_ctr32: + vmovdqu %xmm1,(%r8) + vpclmulqdq $0x10,%xmm3,%xmm7,%xmm5 + vpxor %xmm15,%xmm12,%xmm12 + vmovups 16-128(%rcx),%xmm2 + vpclmulqdq $0x01,%xmm3,%xmm7,%xmm6 + xorq %r12,%r12 + cmpq %r14,%r15 + + vaesenc %xmm2,%xmm9,%xmm9 + vmovdqu 48+8(%rsp),%xmm0 + vpxor %xmm15,%xmm13,%xmm13 + vpclmulqdq $0x00,%xmm3,%xmm7,%xmm1 + vaesenc %xmm2,%xmm10,%xmm10 + vpxor %xmm15,%xmm14,%xmm14 + setnc %r12b + vpclmulqdq $0x11,%xmm3,%xmm7,%xmm7 + vaesenc %xmm2,%xmm11,%xmm11 + vmovdqu 16-32(%r9),%xmm3 + negq %r12 + vaesenc %xmm2,%xmm12,%xmm12 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm3,%xmm0,%xmm5 + vpxor %xmm4,%xmm8,%xmm8 + vaesenc %xmm2,%xmm13,%xmm13 + vpxor %xmm5,%xmm1,%xmm4 + andq $0x60,%r12 + vmovups 32-128(%rcx),%xmm15 + vpclmulqdq $0x10,%xmm3,%xmm0,%xmm1 + vaesenc %xmm2,%xmm14,%xmm14 + + vpclmulqdq $0x01,%xmm3,%xmm0,%xmm2 + leaq (%r14,%r12,1),%r14 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor 16+8(%rsp),%xmm8,%xmm8 + vpclmulqdq $0x11,%xmm3,%xmm0,%xmm3 + vmovdqu 64+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 88(%r14),%r13 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 80(%r14),%r12 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,32+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,40+8(%rsp) + vmovdqu 48-32(%r9),%xmm5 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 48-128(%rcx),%xmm15 + vpxor %xmm1,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm5,%xmm0,%xmm1 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm2,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm5,%xmm0,%xmm2 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor %xmm3,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm5,%xmm0,%xmm3 + vaesenc %xmm15,%xmm11,%xmm11 + vpclmulqdq $0x11,%xmm5,%xmm0,%xmm5 + vmovdqu 80+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor %xmm1,%xmm4,%xmm4 + vmovdqu 64-32(%r9),%xmm1 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 64-128(%rcx),%xmm15 + vpxor %xmm2,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm1,%xmm0,%xmm2 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm3,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm1,%xmm0,%xmm3 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 72(%r14),%r13 + vpxor %xmm5,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm1,%xmm0,%xmm5 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 64(%r14),%r12 + vpclmulqdq $0x11,%xmm1,%xmm0,%xmm1 + vmovdqu 96+8(%rsp),%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,48+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,56+8(%rsp) + vpxor %xmm2,%xmm4,%xmm4 + vmovdqu 96-32(%r9),%xmm2 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 80-128(%rcx),%xmm15 + vpxor %xmm3,%xmm6,%xmm6 + vpclmulqdq $0x00,%xmm2,%xmm0,%xmm3 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm2,%xmm0,%xmm5 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 56(%r14),%r13 + vpxor %xmm1,%xmm7,%xmm7 + vpclmulqdq $0x01,%xmm2,%xmm0,%xmm1 + vpxor 112+8(%rsp),%xmm8,%xmm8 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 48(%r14),%r12 + vpclmulqdq $0x11,%xmm2,%xmm0,%xmm2 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,64+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,72+8(%rsp) + vpxor %xmm3,%xmm4,%xmm4 + vmovdqu 112-32(%r9),%xmm3 + vaesenc %xmm15,%xmm14,%xmm14 + + vmovups 96-128(%rcx),%xmm15 + vpxor %xmm5,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm5 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm1,%xmm6,%xmm6 + vpclmulqdq $0x01,%xmm3,%xmm8,%xmm1 + vaesenc %xmm15,%xmm10,%xmm10 + movbeq 40(%r14),%r13 + vpxor %xmm2,%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm3,%xmm8,%xmm2 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 32(%r14),%r12 + vpclmulqdq $0x11,%xmm3,%xmm8,%xmm8 + vaesenc %xmm15,%xmm12,%xmm12 + movq %r13,80+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + movq %r12,88+8(%rsp) + vpxor %xmm5,%xmm6,%xmm6 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor %xmm1,%xmm6,%xmm6 + + vmovups 112-128(%rcx),%xmm15 + vpslldq $8,%xmm6,%xmm5 + vpxor %xmm2,%xmm4,%xmm4 + vmovdqu 16(%r11),%xmm3 + + vaesenc %xmm15,%xmm9,%xmm9 + vpxor %xmm8,%xmm7,%xmm7 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor %xmm5,%xmm4,%xmm4 + movbeq 24(%r14),%r13 + vaesenc %xmm15,%xmm11,%xmm11 + movbeq 16(%r14),%r12 + vpalignr $8,%xmm4,%xmm4,%xmm0 + vpclmulqdq $0x10,%xmm3,%xmm4,%xmm4 + movq %r13,96+8(%rsp) + vaesenc %xmm15,%xmm12,%xmm12 + movq %r12,104+8(%rsp) + vaesenc %xmm15,%xmm13,%xmm13 + vmovups 128-128(%rcx),%xmm1 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vmovups 144-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm10,%xmm10 + vpsrldq $8,%xmm6,%xmm6 + vaesenc %xmm1,%xmm11,%xmm11 + vpxor %xmm6,%xmm7,%xmm7 + vaesenc %xmm1,%xmm12,%xmm12 + vpxor %xmm0,%xmm4,%xmm4 + movbeq 8(%r14),%r13 + vaesenc %xmm1,%xmm13,%xmm13 + movbeq 0(%r14),%r12 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 160-128(%rcx),%xmm1 + cmpl $11,%r10d + jb .Lenc_tail + + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vaesenc %xmm1,%xmm10,%xmm10 + vaesenc %xmm1,%xmm11,%xmm11 + vaesenc %xmm1,%xmm12,%xmm12 + vaesenc %xmm1,%xmm13,%xmm13 + vmovups 176-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 192-128(%rcx),%xmm1 + je .Lenc_tail + + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + + vaesenc %xmm1,%xmm9,%xmm9 + vaesenc %xmm1,%xmm10,%xmm10 + vaesenc %xmm1,%xmm11,%xmm11 + vaesenc %xmm1,%xmm12,%xmm12 + vaesenc %xmm1,%xmm13,%xmm13 + vmovups 208-128(%rcx),%xmm15 + vaesenc %xmm1,%xmm14,%xmm14 + vmovups 224-128(%rcx),%xmm1 + jmp .Lenc_tail + +.p2align 5 +.Lhandle_ctr32: + vmovdqu (%r11),%xmm0 + vpshufb %xmm0,%xmm1,%xmm6 + vmovdqu 48(%r11),%xmm5 + vpaddd 64(%r11),%xmm6,%xmm10 + vpaddd %xmm5,%xmm6,%xmm11 + vmovdqu 0-32(%r9),%xmm3 + vpaddd %xmm5,%xmm10,%xmm12 + vpshufb %xmm0,%xmm10,%xmm10 + vpaddd %xmm5,%xmm11,%xmm13 + vpshufb %xmm0,%xmm11,%xmm11 + vpxor %xmm15,%xmm10,%xmm10 + vpaddd %xmm5,%xmm12,%xmm14 + vpshufb %xmm0,%xmm12,%xmm12 + vpxor %xmm15,%xmm11,%xmm11 + vpaddd %xmm5,%xmm13,%xmm1 + vpshufb %xmm0,%xmm13,%xmm13 + vpshufb %xmm0,%xmm14,%xmm14 + vpshufb %xmm0,%xmm1,%xmm1 + jmp .Lresume_ctr32 + +.p2align 5 +.Lenc_tail: + vaesenc %xmm15,%xmm9,%xmm9 + vmovdqu %xmm7,16+8(%rsp) + vpalignr $8,%xmm4,%xmm4,%xmm8 + vaesenc %xmm15,%xmm10,%xmm10 + vpclmulqdq $0x10,%xmm3,%xmm4,%xmm4 + vpxor 0(%rdi),%xmm1,%xmm2 + vaesenc %xmm15,%xmm11,%xmm11 + vpxor 16(%rdi),%xmm1,%xmm0 + vaesenc %xmm15,%xmm12,%xmm12 + vpxor 32(%rdi),%xmm1,%xmm5 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor 48(%rdi),%xmm1,%xmm6 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor 64(%rdi),%xmm1,%xmm7 + vpxor 80(%rdi),%xmm1,%xmm3 + vmovdqu (%r8),%xmm1 + + vaesenclast %xmm2,%xmm9,%xmm9 + vmovdqu 32(%r11),%xmm2 + vaesenclast %xmm0,%xmm10,%xmm10 + vpaddb %xmm2,%xmm1,%xmm0 + movq %r13,112+8(%rsp) + leaq 96(%rdi),%rdi + vaesenclast %xmm5,%xmm11,%xmm11 + vpaddb %xmm2,%xmm0,%xmm5 + movq %r12,120+8(%rsp) + leaq 96(%rsi),%rsi + vmovdqu 0-128(%rcx),%xmm15 + vaesenclast %xmm6,%xmm12,%xmm12 + vpaddb %xmm2,%xmm5,%xmm6 + vaesenclast %xmm7,%xmm13,%xmm13 + vpaddb %xmm2,%xmm6,%xmm7 + vaesenclast %xmm3,%xmm14,%xmm14 + vpaddb %xmm2,%xmm7,%xmm3 + + addq $0x60,%rax + subq $0x6,%rdx + jc .L6x_done + + vmovups %xmm9,-96(%rsi) + vpxor %xmm15,%xmm1,%xmm9 + vmovups %xmm10,-80(%rsi) + vmovdqa %xmm0,%xmm10 + vmovups %xmm11,-64(%rsi) + vmovdqa %xmm5,%xmm11 + vmovups %xmm12,-48(%rsi) + vmovdqa %xmm6,%xmm12 + vmovups %xmm13,-32(%rsi) + vmovdqa %xmm7,%xmm13 + vmovups %xmm14,-16(%rsi) + vmovdqa %xmm3,%xmm14 + vmovdqu 32+8(%rsp),%xmm7 + jmp .Loop6x + +.L6x_done: + vpxor 16+8(%rsp),%xmm8,%xmm8 + vpxor %xmm4,%xmm8,%xmm8 + + .byte 0xf3,0xc3 + + +.globl crypton_gcm_asm_decrypt +.def crypton_gcm_asm_decrypt; .scl 2; .type 32; .endef +.p2align 5 +crypton_gcm_asm_decrypt: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_gcm_asm_decrypt: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 48(%rsp),%r8 + movq 56(%rsp),%r9 + xorq %rax,%rax + cmpq $0x60,%rdx + jb .Lgcm_dec_abort + + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -168(%rsp),%rsp + + movaps %xmm6,-208(%rbp) + movaps %xmm7,-192(%rbp) + movaps %xmm8,-176(%rbp) + movaps %xmm9,-160(%rbp) + movaps %xmm10,-144(%rbp) + movaps %xmm11,-128(%rbp) + movaps %xmm12,-112(%rbp) + movaps %xmm13,-96(%rbp) + movaps %xmm14,-80(%rbp) + movaps %xmm15,-64(%rbp) + +.LSEH_body_crypton_gcm_asm_decrypt: + + vzeroupper + + vmovdqu (%r8),%xmm1 + addq $-128,%rsp + movl 12(%r8),%ebx + leaq .Lbswap_mask(%rip),%r11 + leaq -128(%rcx),%r14 + movq $0xf80,%r15 + vmovdqu (%r9),%xmm8 + andq $-128,%rsp + vmovdqu (%r11),%xmm0 + leaq 128(%rcx),%rcx + leaq 32+32(%r9),%r9 + movl 240-128(%rcx),%r10d + vpshufb %xmm0,%xmm8,%xmm8 + + andq %r15,%r14 + andq %rsp,%r15 + subq %r14,%r15 + jc .Ldec_no_key_aliasing + cmpq $768,%r15 + jnc .Ldec_no_key_aliasing + subq %r15,%rsp +.Ldec_no_key_aliasing: + + vmovdqu 80(%rdi),%xmm7 + leaq (%rdi),%r14 + vmovdqu 64(%rdi),%xmm4 + leaq -192(%rdi,%rdx,1),%r15 + vmovdqu 48(%rdi),%xmm5 + shrq $4,%rdx + xorq %rax,%rax + vmovdqu 32(%rdi),%xmm6 + vpshufb %xmm0,%xmm7,%xmm7 + vmovdqu 16(%rdi),%xmm2 + vpshufb %xmm0,%xmm4,%xmm4 + vmovdqu (%rdi),%xmm3 + vpshufb %xmm0,%xmm5,%xmm5 + vmovdqu %xmm4,48(%rsp) + vpshufb %xmm0,%xmm6,%xmm6 + vmovdqu %xmm5,64(%rsp) + vpshufb %xmm0,%xmm2,%xmm2 + vmovdqu %xmm6,80(%rsp) + vpshufb %xmm0,%xmm3,%xmm3 + vmovdqu %xmm2,96(%rsp) + vmovdqu %xmm3,112(%rsp) + + call _crypton_gcm_asm_ctr32_ghash_6x + + vmovups %xmm9,-96(%rsi) + vmovups %xmm10,-80(%rsi) + vmovups %xmm11,-64(%rsi) + vmovups %xmm12,-48(%rsi) + vmovups %xmm13,-32(%rsi) + vmovups %xmm14,-16(%rsi) + + vpshufb (%r11),%xmm8,%xmm8 + vmovdqu %xmm8,-64(%r9) + + vzeroupper + movaps -208(%rbp),%xmm6 + movaps -192(%rbp),%xmm7 + movaps -176(%rbp),%xmm8 + movaps -160(%rbp),%xmm9 + movaps -144(%rbp),%xmm10 + movaps -128(%rbp),%xmm11 + movaps -112(%rbp),%xmm12 + movaps -96(%rbp),%xmm13 + movaps -80(%rbp),%xmm14 + movaps -64(%rbp),%xmm15 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + +.Lgcm_dec_abort: + popq %rbp + +.LSEH_epilogue_crypton_gcm_asm_decrypt: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_gcm_asm_decrypt: +.def _crypton_gcm_asm_ctr32_6x; .scl 3; .type 32; .endef +.p2align 5 +_crypton_gcm_asm_ctr32_6x: + .byte 0xf3,0x0f,0x1e,0xfa + + + vmovdqu 0-128(%rcx),%xmm4 + vmovdqu 32(%r11),%xmm2 + leaq -1(%r10),%r13 + vmovups 16-128(%rcx),%xmm15 + leaq 32-128(%rcx),%r12 + vpxor %xmm4,%xmm1,%xmm9 + addl $100663296,%ebx + jc .Lhandle_ctr32_2 + vpaddb %xmm2,%xmm1,%xmm10 + vpaddb %xmm2,%xmm10,%xmm11 + vpxor %xmm4,%xmm10,%xmm10 + vpaddb %xmm2,%xmm11,%xmm12 + vpxor %xmm4,%xmm11,%xmm11 + vpaddb %xmm2,%xmm12,%xmm13 + vpxor %xmm4,%xmm12,%xmm12 + vpaddb %xmm2,%xmm13,%xmm14 + vpxor %xmm4,%xmm13,%xmm13 + vpaddb %xmm2,%xmm14,%xmm1 + vpxor %xmm4,%xmm14,%xmm14 + jmp .Loop_ctr32 + +.p2align 4 +.Loop_ctr32: + vaesenc %xmm15,%xmm9,%xmm9 + vaesenc %xmm15,%xmm10,%xmm10 + vaesenc %xmm15,%xmm11,%xmm11 + vaesenc %xmm15,%xmm12,%xmm12 + vaesenc %xmm15,%xmm13,%xmm13 + vaesenc %xmm15,%xmm14,%xmm14 + vmovups (%r12),%xmm15 + leaq 16(%r12),%r12 + decl %r13d + jnz .Loop_ctr32 + + vmovdqu (%r12),%xmm3 + vaesenc %xmm15,%xmm9,%xmm9 + vpxor 0(%rdi),%xmm3,%xmm4 + vaesenc %xmm15,%xmm10,%xmm10 + vpxor 16(%rdi),%xmm3,%xmm5 + vaesenc %xmm15,%xmm11,%xmm11 + vpxor 32(%rdi),%xmm3,%xmm6 + vaesenc %xmm15,%xmm12,%xmm12 + vpxor 48(%rdi),%xmm3,%xmm8 + vaesenc %xmm15,%xmm13,%xmm13 + vpxor 64(%rdi),%xmm3,%xmm2 + vaesenc %xmm15,%xmm14,%xmm14 + vpxor 80(%rdi),%xmm3,%xmm3 + leaq 96(%rdi),%rdi + + vaesenclast %xmm4,%xmm9,%xmm9 + vaesenclast %xmm5,%xmm10,%xmm10 + vaesenclast %xmm6,%xmm11,%xmm11 + vaesenclast %xmm8,%xmm12,%xmm12 + vaesenclast %xmm2,%xmm13,%xmm13 + vaesenclast %xmm3,%xmm14,%xmm14 + vmovups %xmm9,0(%rsi) + vmovups %xmm10,16(%rsi) + vmovups %xmm11,32(%rsi) + vmovups %xmm12,48(%rsi) + vmovups %xmm13,64(%rsi) + vmovups %xmm14,80(%rsi) + leaq 96(%rsi),%rsi + + .byte 0xf3,0xc3 +.p2align 5 +.Lhandle_ctr32_2: + vpshufb %xmm0,%xmm1,%xmm6 + vmovdqu 48(%r11),%xmm5 + vpaddd 64(%r11),%xmm6,%xmm10 + vpaddd %xmm5,%xmm6,%xmm11 + vpaddd %xmm5,%xmm10,%xmm12 + vpshufb %xmm0,%xmm10,%xmm10 + vpaddd %xmm5,%xmm11,%xmm13 + vpshufb %xmm0,%xmm11,%xmm11 + vpxor %xmm4,%xmm10,%xmm10 + vpaddd %xmm5,%xmm12,%xmm14 + vpshufb %xmm0,%xmm12,%xmm12 + vpxor %xmm4,%xmm11,%xmm11 + vpaddd %xmm5,%xmm13,%xmm1 + vpshufb %xmm0,%xmm13,%xmm13 + vpxor %xmm4,%xmm12,%xmm12 + vpshufb %xmm0,%xmm14,%xmm14 + vpxor %xmm4,%xmm13,%xmm13 + vpshufb %xmm0,%xmm1,%xmm1 + vpxor %xmm4,%xmm14,%xmm14 + jmp .Loop_ctr32 + + + +.globl crypton_gcm_asm_encrypt +.def crypton_gcm_asm_encrypt; .scl 2; .type 32; .endef +.p2align 5 +crypton_gcm_asm_encrypt: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_gcm_asm_encrypt: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 48(%rsp),%r8 + movq 56(%rsp),%r9 + xorq %rax,%rax + cmpq $288,%rdx + jb .Lgcm_enc_abort + + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -168(%rsp),%rsp + + movaps %xmm6,-208(%rbp) + movaps %xmm7,-192(%rbp) + movaps %xmm8,-176(%rbp) + movaps %xmm9,-160(%rbp) + movaps %xmm10,-144(%rbp) + movaps %xmm11,-128(%rbp) + movaps %xmm12,-112(%rbp) + movaps %xmm13,-96(%rbp) + movaps %xmm14,-80(%rbp) + movaps %xmm15,-64(%rbp) + +.LSEH_body_crypton_gcm_asm_encrypt: + + vzeroupper + + vmovdqu (%r8),%xmm1 + addq $-128,%rsp + movl 12(%r8),%ebx + leaq .Lbswap_mask(%rip),%r11 + leaq -128(%rcx),%r14 + movq $0xf80,%r15 + leaq 128(%rcx),%rcx + vmovdqu (%r11),%xmm0 + andq $-128,%rsp + movl 240-128(%rcx),%r10d + + andq %r15,%r14 + andq %rsp,%r15 + subq %r14,%r15 + jc .Lenc_no_key_aliasing + cmpq $768,%r15 + jnc .Lenc_no_key_aliasing + subq %r15,%rsp +.Lenc_no_key_aliasing: + + leaq (%rsi),%r14 + leaq -192(%rsi,%rdx,1),%r15 + shrq $4,%rdx + + call _crypton_gcm_asm_ctr32_6x + vpshufb %xmm0,%xmm9,%xmm8 + vpshufb %xmm0,%xmm10,%xmm2 + vmovdqu %xmm8,112(%rsp) + vpshufb %xmm0,%xmm11,%xmm4 + vmovdqu %xmm2,96(%rsp) + vpshufb %xmm0,%xmm12,%xmm5 + vmovdqu %xmm4,80(%rsp) + vpshufb %xmm0,%xmm13,%xmm6 + vmovdqu %xmm5,64(%rsp) + vpshufb %xmm0,%xmm14,%xmm7 + vmovdqu %xmm6,48(%rsp) + + call _crypton_gcm_asm_ctr32_6x + + vmovdqu (%r9),%xmm8 + leaq 32+32(%r9),%r9 + subq $12,%rdx + movq $192,%rax + vpshufb %xmm0,%xmm8,%xmm8 + + call _crypton_gcm_asm_ctr32_ghash_6x + vmovdqu 32(%rsp),%xmm7 + vmovdqu (%r11),%xmm0 + vmovdqu 0-32(%r9),%xmm3 + vpunpckhqdq %xmm7,%xmm7,%xmm1 + vmovdqu 32-32(%r9),%xmm15 + vmovups %xmm9,-96(%rsi) + vpshufb %xmm0,%xmm9,%xmm9 + vpxor %xmm7,%xmm1,%xmm1 + vmovups %xmm10,-80(%rsi) + vpshufb %xmm0,%xmm10,%xmm10 + vmovups %xmm11,-64(%rsi) + vpshufb %xmm0,%xmm11,%xmm11 + vmovups %xmm12,-48(%rsi) + vpshufb %xmm0,%xmm12,%xmm12 + vmovups %xmm13,-32(%rsi) + vpshufb %xmm0,%xmm13,%xmm13 + vmovups %xmm14,-16(%rsi) + vpshufb %xmm0,%xmm14,%xmm14 + vmovdqu %xmm9,16(%rsp) + vmovdqu 48(%rsp),%xmm6 + vmovdqu 16-32(%r9),%xmm0 + vpunpckhqdq %xmm6,%xmm6,%xmm2 + vpclmulqdq $0x00,%xmm3,%xmm7,%xmm5 + vpxor %xmm6,%xmm2,%xmm2 + vpclmulqdq $0x11,%xmm3,%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm15,%xmm1,%xmm1 + + vmovdqu 64(%rsp),%xmm9 + vpclmulqdq $0x00,%xmm0,%xmm6,%xmm4 + vmovdqu 48-32(%r9),%xmm3 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm9,%xmm9,%xmm5 + vpclmulqdq $0x11,%xmm0,%xmm6,%xmm6 + vpxor %xmm9,%xmm5,%xmm5 + vpxor %xmm7,%xmm6,%xmm6 + vpclmulqdq $0x10,%xmm15,%xmm2,%xmm2 + vmovdqu 80-32(%r9),%xmm15 + vpxor %xmm1,%xmm2,%xmm2 + + vmovdqu 80(%rsp),%xmm1 + vpclmulqdq $0x00,%xmm3,%xmm9,%xmm7 + vmovdqu 64-32(%r9),%xmm0 + vpxor %xmm4,%xmm7,%xmm7 + vpunpckhqdq %xmm1,%xmm1,%xmm4 + vpclmulqdq $0x11,%xmm3,%xmm9,%xmm9 + vpxor %xmm1,%xmm4,%xmm4 + vpxor %xmm6,%xmm9,%xmm9 + vpclmulqdq $0x00,%xmm15,%xmm5,%xmm5 + vpxor %xmm2,%xmm5,%xmm5 + + vmovdqu 96(%rsp),%xmm2 + vpclmulqdq $0x00,%xmm0,%xmm1,%xmm6 + vmovdqu 96-32(%r9),%xmm3 + vpxor %xmm7,%xmm6,%xmm6 + vpunpckhqdq %xmm2,%xmm2,%xmm7 + vpclmulqdq $0x11,%xmm0,%xmm1,%xmm1 + vpxor %xmm2,%xmm7,%xmm7 + vpxor %xmm9,%xmm1,%xmm1 + vpclmulqdq $0x10,%xmm15,%xmm4,%xmm4 + vmovdqu 128-32(%r9),%xmm15 + vpxor %xmm5,%xmm4,%xmm4 + + vpxor 112(%rsp),%xmm8,%xmm8 + vpclmulqdq $0x00,%xmm3,%xmm2,%xmm5 + vmovdqu 112-32(%r9),%xmm0 + vpunpckhqdq %xmm8,%xmm8,%xmm9 + vpxor %xmm6,%xmm5,%xmm5 + vpclmulqdq $0x11,%xmm3,%xmm2,%xmm2 + vpxor %xmm8,%xmm9,%xmm9 + vpxor %xmm1,%xmm2,%xmm2 + vpclmulqdq $0x00,%xmm15,%xmm7,%xmm7 + vpxor %xmm4,%xmm7,%xmm4 + + vpclmulqdq $0x00,%xmm0,%xmm8,%xmm6 + vmovdqu 0-32(%r9),%xmm3 + vpunpckhqdq %xmm14,%xmm14,%xmm1 + vpclmulqdq $0x11,%xmm0,%xmm8,%xmm8 + vpxor %xmm14,%xmm1,%xmm1 + vpxor %xmm5,%xmm6,%xmm5 + vpclmulqdq $0x10,%xmm15,%xmm9,%xmm9 + vmovdqu 32-32(%r9),%xmm15 + vpxor %xmm2,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm6 + + vmovdqu 16-32(%r9),%xmm0 + vpxor %xmm5,%xmm7,%xmm9 + vpclmulqdq $0x00,%xmm3,%xmm14,%xmm4 + vpxor %xmm9,%xmm6,%xmm6 + vpunpckhqdq %xmm13,%xmm13,%xmm2 + vpclmulqdq $0x11,%xmm3,%xmm14,%xmm14 + vpxor %xmm13,%xmm2,%xmm2 + vpslldq $8,%xmm6,%xmm9 + vpclmulqdq $0x00,%xmm15,%xmm1,%xmm1 + vpxor %xmm9,%xmm5,%xmm8 + vpsrldq $8,%xmm6,%xmm6 + vpxor %xmm6,%xmm7,%xmm7 + + vpclmulqdq $0x00,%xmm0,%xmm13,%xmm5 + vmovdqu 48-32(%r9),%xmm3 + vpxor %xmm4,%xmm5,%xmm5 + vpunpckhqdq %xmm12,%xmm12,%xmm9 + vpclmulqdq $0x11,%xmm0,%xmm13,%xmm13 + vpxor %xmm12,%xmm9,%xmm9 + vpxor %xmm14,%xmm13,%xmm13 + vpalignr $8,%xmm8,%xmm8,%xmm14 + vpclmulqdq $0x10,%xmm15,%xmm2,%xmm2 + vmovdqu 80-32(%r9),%xmm15 + vpxor %xmm1,%xmm2,%xmm2 + + vpclmulqdq $0x00,%xmm3,%xmm12,%xmm4 + vmovdqu 64-32(%r9),%xmm0 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm11,%xmm11,%xmm1 + vpclmulqdq $0x11,%xmm3,%xmm12,%xmm12 + vpxor %xmm11,%xmm1,%xmm1 + vpxor %xmm13,%xmm12,%xmm12 + vxorps 16(%rsp),%xmm7,%xmm7 + vpclmulqdq $0x00,%xmm15,%xmm9,%xmm9 + vpxor %xmm2,%xmm9,%xmm9 + + vpclmulqdq $0x10,16(%r11),%xmm8,%xmm8 + vxorps %xmm14,%xmm8,%xmm8 + + vpclmulqdq $0x00,%xmm0,%xmm11,%xmm5 + vmovdqu 96-32(%r9),%xmm3 + vpxor %xmm4,%xmm5,%xmm5 + vpunpckhqdq %xmm10,%xmm10,%xmm2 + vpclmulqdq $0x11,%xmm0,%xmm11,%xmm11 + vpxor %xmm10,%xmm2,%xmm2 + vpalignr $8,%xmm8,%xmm8,%xmm14 + vpxor %xmm12,%xmm11,%xmm11 + vpclmulqdq $0x10,%xmm15,%xmm1,%xmm1 + vmovdqu 128-32(%r9),%xmm15 + vpxor %xmm9,%xmm1,%xmm1 + + vxorps %xmm7,%xmm14,%xmm14 + vpclmulqdq $0x10,16(%r11),%xmm8,%xmm8 + vxorps %xmm14,%xmm8,%xmm8 + + vpclmulqdq $0x00,%xmm3,%xmm10,%xmm4 + vmovdqu 112-32(%r9),%xmm0 + vpxor %xmm5,%xmm4,%xmm4 + vpunpckhqdq %xmm8,%xmm8,%xmm9 + vpclmulqdq $0x11,%xmm3,%xmm10,%xmm10 + vpxor %xmm8,%xmm9,%xmm9 + vpxor %xmm11,%xmm10,%xmm10 + vpclmulqdq $0x00,%xmm15,%xmm2,%xmm2 + vpxor %xmm1,%xmm2,%xmm2 + + vpclmulqdq $0x00,%xmm0,%xmm8,%xmm5 + vpclmulqdq $0x11,%xmm0,%xmm8,%xmm7 + vpxor %xmm4,%xmm5,%xmm5 + vpclmulqdq $0x10,%xmm15,%xmm9,%xmm6 + vpxor %xmm10,%xmm7,%xmm7 + vpxor %xmm2,%xmm6,%xmm6 + + vpxor %xmm5,%xmm7,%xmm4 + vpxor %xmm4,%xmm6,%xmm6 + vpslldq $8,%xmm6,%xmm1 + vmovdqu 16(%r11),%xmm3 + vpsrldq $8,%xmm6,%xmm6 + vpxor %xmm1,%xmm5,%xmm8 + vpxor %xmm6,%xmm7,%xmm7 + + vpalignr $8,%xmm8,%xmm8,%xmm2 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm8 + vpxor %xmm2,%xmm8,%xmm8 + + vpalignr $8,%xmm8,%xmm8,%xmm2 + vpclmulqdq $0x10,%xmm3,%xmm8,%xmm8 + vpxor %xmm7,%xmm2,%xmm2 + vpxor %xmm2,%xmm8,%xmm8 + vpshufb (%r11),%xmm8,%xmm8 + vmovdqu %xmm8,-64(%r9) + + vzeroupper + movaps -208(%rbp),%xmm6 + movaps -192(%rbp),%xmm7 + movaps -176(%rbp),%xmm8 + movaps -160(%rbp),%xmm9 + movaps -144(%rbp),%xmm10 + movaps -128(%rbp),%xmm11 + movaps -112(%rbp),%xmm12 + movaps -96(%rbp),%xmm13 + movaps -80(%rbp),%xmm14 + movaps -64(%rbp),%xmm15 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + +.Lgcm_enc_abort: + popq %rbp + +.LSEH_epilogue_crypton_gcm_asm_encrypt: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_gcm_asm_encrypt: +.p2align 6 +.Lbswap_mask: +.byte 15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0 +.Lpoly: +.byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2 +.Lone_msb: +.byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1 +.Ltwo_lsb: +.byte 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.Lone_lsb: +.byte 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.byte 65,69,83,45,78,73,32,71,67,77,32,109,111,100,117,108,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 +.p2align 6 +.section .pdata +.p2align 2 +.rva .LSEH_begin_crypton_gcm_asm_decrypt +.rva .LSEH_body_crypton_gcm_asm_decrypt +.rva .LSEH_info_crypton_gcm_asm_decrypt_prologue + +.rva .LSEH_body_crypton_gcm_asm_decrypt +.rva .LSEH_epilogue_crypton_gcm_asm_decrypt +.rva .LSEH_info_crypton_gcm_asm_decrypt_body + +.rva .LSEH_epilogue_crypton_gcm_asm_decrypt +.rva .LSEH_end_crypton_gcm_asm_decrypt +.rva .LSEH_info_crypton_gcm_asm_decrypt_epilogue + +.rva .LSEH_begin_crypton_gcm_asm_encrypt +.rva .LSEH_body_crypton_gcm_asm_encrypt +.rva .LSEH_info_crypton_gcm_asm_encrypt_prologue + +.rva .LSEH_body_crypton_gcm_asm_encrypt +.rva .LSEH_epilogue_crypton_gcm_asm_encrypt +.rva .LSEH_info_crypton_gcm_asm_encrypt_body + +.rva .LSEH_epilogue_crypton_gcm_asm_encrypt +.rva .LSEH_end_crypton_gcm_asm_encrypt +.rva .LSEH_info_crypton_gcm_asm_encrypt_epilogue + +.section .xdata +.p2align 3 +.LSEH_info_crypton_gcm_asm_decrypt_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_gcm_asm_decrypt_body: +.byte 1,0,38,213 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0xb8,0x05,0x00 +.byte 0x00,0xc8,0x06,0x00 +.byte 0x00,0xd8,0x07,0x00 +.byte 0x00,0xe8,0x08,0x00 +.byte 0x00,0xf8,0x09,0x00 +.byte 0x00,0xf4,0x15,0x00 +.byte 0x00,0xe4,0x16,0x00 +.byte 0x00,0xd4,0x17,0x00 +.byte 0x00,0xc4,0x18,0x00 +.byte 0x00,0x34,0x19,0x00 +.byte 0x00,0x74,0x1c,0x00 +.byte 0x00,0x64,0x1d,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x1a,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_gcm_asm_decrypt_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_gcm_asm_encrypt_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_gcm_asm_encrypt_body: +.byte 1,0,38,213 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0xb8,0x05,0x00 +.byte 0x00,0xc8,0x06,0x00 +.byte 0x00,0xd8,0x07,0x00 +.byte 0x00,0xe8,0x08,0x00 +.byte 0x00,0xf8,0x09,0x00 +.byte 0x00,0xf4,0x15,0x00 +.byte 0x00,0xe4,0x16,0x00 +.byte 0x00,0xd4,0x17,0x00 +.byte 0x00,0xc4,0x18,0x00 +.byte 0x00,0x34,0x19,0x00 +.byte 0x00,0x74,0x1c,0x00 +.byte 0x00,0x64,0x1d,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x1a,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_gcm_asm_encrypt_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + diff --git a/cbits/asm/aesni-gcm-x86_64.pl b/cbits/asm/aesni-gcm-x86_64.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/aesni-gcm-x86_64.pl @@ -0,0 +1,974 @@ +#! /usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov for the OpenSSL +# project. The module is, however, dual licensed under OpenSSL and +# CRYPTOGAMS licenses depending on where you obtain it. For further +# details see http://www.openssl.org/~appro/cryptogams/. +# ==================================================================== +# +# +# AES-NI-CTR+GHASH stitch. +# +# February 2013 +# +# OpenSSL GCM implementation is organized in such way that its +# performance is rather close to the sum of its streamed components, +# in the context parallelized AES-NI CTR and modulo-scheduled +# PCLMULQDQ-enabled GHASH. Unfortunately, as no stitch implementation +# was observed to perform significantly better than the sum of the +# components on contemporary CPUs, the effort was deemed impossible to +# justify. This module is based on combination of Intel submissions, +# [1] and [2], with MOVBE twist suggested by Ilya Albrekht and Max +# Locktyukhin of Intel Corp. who verified that it reduces shuffles +# pressure with notable relative improvement, achieving 1.0 cycle per +# byte processed with 128-bit key on Haswell processor, 0.74 - on +# Broadwell, 0.63 - on Skylake... [Mentioned results are raw profiled +# measurements for favourable packet size, one divisible by 96. +# Applications using the EVP interface will observe a few percent +# worse performance.] +# +# Knights Landing processes 1 byte in 1.25 cycles (measured with EVP). +# +# [1] http://rt.openssl.org/Ticket/Display.html?id=2900&user=guest&pass=guest +# [2] http://www.intel.com/content/dam/www/public/us/en/documents/software-support/enabling-high-performance-gcm.pdf + +$flavour = shift; +$output = shift; +if ($flavour =~ /\./) { $output = $flavour; undef $flavour; } + +$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/); + +$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; +( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or +( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or +die "can't locate x86_64-xlate.pl"; + +$ENV{CC} //= "cc"; +if (`$ENV{CC} -Wa,-v -c -o /dev/null -x assembler /dev/null 2>&1` + =~ /GNU assembler version ([2-9]\.[0-9]+)/) { + $avx = ($1>=2.20) + ($1>=2.22); +} + +if (!$avx && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) && + `nasm -v 2>&1` =~ /NASM version ([2-9]\.[0-9]+)/) { + $avx = ($1>=2.09) + ($1>=2.10); +} + +if (!$avx && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) && + `ml64 2>&1` =~ /Version ([0-9]+)\./) { + $avx = ($1>=10) + ($1>=11); +} + +if (!$avx && `$ENV{CC} -v 2>&1` =~ /((?:clang|LLVM) version|.*based on LLVM) ([0-9]+\.[0-9]+)/) { + $avx = ($2>=3.0) + ($2>3.0); +} + +open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\""; +*STDOUT=*OUT; + +if ($avx>1) {{{ + +($inp,$out,$len,$key,$ivp,$Xip)=("%rdi","%rsi","%rdx","%rcx","%r8","%r9"); + +($Ii,$T1,$T2,$Hkey, + $Z0,$Z1,$Z2,$Z3,$Xi) = map("%xmm$_",(0..8)); + +($inout0,$inout1,$inout2,$inout3,$inout4,$inout5,$rndkey) = map("%xmm$_",(9..15)); + +($counter,$rounds,$ret,$const,$in0,$end0)=("%ebx","%r10d","%rax","%r11","%r14","%r15"); + +$code=<<___; +.text + +.type _aesni_ctr32_ghash_6x,\@abi-omnipotent +.align 32 +_aesni_ctr32_ghash_6x: +.cfi_startproc + vmovdqu 0x20($const),$T2 # borrow $T2, .Lone_msb + sub \$6,$len + vpxor $Z0,$Z0,$Z0 # $Z0 = 0 + vmovdqu 0x00-0x80($key),$rndkey + vpaddb $T2,$T1,$inout1 + vpaddb $T2,$inout1,$inout2 + vpaddb $T2,$inout2,$inout3 + vpaddb $T2,$inout3,$inout4 + vpaddb $T2,$inout4,$inout5 + vpxor $rndkey,$T1,$inout0 + vmovdqu $Z0,16+8(%rsp) # "$Z3" = 0 + jmp .Loop6x + +.align 32 +.Loop6x: + add \$`6<<24`,$counter + jc .Lhandle_ctr32 # discard $inout[1-5]? + vmovdqu 0x00-0x20($Xip),$Hkey # $Hkey^1 + vpaddb $T2,$inout5,$T1 # next counter value + vpxor $rndkey,$inout1,$inout1 + vpxor $rndkey,$inout2,$inout2 + +.Lresume_ctr32: + vmovdqu $T1,($ivp) # save next counter value + vpclmulqdq \$0x10,$Hkey,$Z3,$Z1 + vpxor $rndkey,$inout3,$inout3 + vmovups 0x10-0x80($key),$T2 # borrow $T2 for $rndkey + vpclmulqdq \$0x01,$Hkey,$Z3,$Z2 + xor %r12,%r12 + cmp $in0,$end0 + + vaesenc $T2,$inout0,$inout0 + vmovdqu 0x30+8(%rsp),$Ii # I[4] + vpxor $rndkey,$inout4,$inout4 + vpclmulqdq \$0x00,$Hkey,$Z3,$T1 + vaesenc $T2,$inout1,$inout1 + vpxor $rndkey,$inout5,$inout5 + setnc %r12b + vpclmulqdq \$0x11,$Hkey,$Z3,$Z3 + vaesenc $T2,$inout2,$inout2 + vmovdqu 0x10-0x20($Xip),$Hkey # $Hkey^2 + neg %r12 + vaesenc $T2,$inout3,$inout3 + vpxor $Z1,$Z2,$Z2 + vpclmulqdq \$0x00,$Hkey,$Ii,$Z1 + vpxor $Z0,$Xi,$Xi # modulo-scheduled + vaesenc $T2,$inout4,$inout4 + vpxor $Z1,$T1,$Z0 + and \$0x60,%r12 + vmovups 0x20-0x80($key),$rndkey + vpclmulqdq \$0x10,$Hkey,$Ii,$T1 + vaesenc $T2,$inout5,$inout5 + + vpclmulqdq \$0x01,$Hkey,$Ii,$T2 + lea ($in0,%r12),$in0 + vaesenc $rndkey,$inout0,$inout0 + vpxor 16+8(%rsp),$Xi,$Xi # modulo-scheduled [vpxor $Z3,$Xi,$Xi] + vpclmulqdq \$0x11,$Hkey,$Ii,$Hkey + vmovdqu 0x40+8(%rsp),$Ii # I[3] + vaesenc $rndkey,$inout1,$inout1 + movbe 0x58($in0),%r13 + vaesenc $rndkey,$inout2,$inout2 + movbe 0x50($in0),%r12 + vaesenc $rndkey,$inout3,$inout3 + mov %r13,0x20+8(%rsp) + vaesenc $rndkey,$inout4,$inout4 + mov %r12,0x28+8(%rsp) + vmovdqu 0x30-0x20($Xip),$Z1 # borrow $Z1 for $Hkey^3 + vaesenc $rndkey,$inout5,$inout5 + + vmovups 0x30-0x80($key),$rndkey + vpxor $T1,$Z2,$Z2 + vpclmulqdq \$0x00,$Z1,$Ii,$T1 + vaesenc $rndkey,$inout0,$inout0 + vpxor $T2,$Z2,$Z2 + vpclmulqdq \$0x10,$Z1,$Ii,$T2 + vaesenc $rndkey,$inout1,$inout1 + vpxor $Hkey,$Z3,$Z3 + vpclmulqdq \$0x01,$Z1,$Ii,$Hkey + vaesenc $rndkey,$inout2,$inout2 + vpclmulqdq \$0x11,$Z1,$Ii,$Z1 + vmovdqu 0x50+8(%rsp),$Ii # I[2] + vaesenc $rndkey,$inout3,$inout3 + vaesenc $rndkey,$inout4,$inout4 + vpxor $T1,$Z0,$Z0 + vmovdqu 0x40-0x20($Xip),$T1 # borrow $T1 for $Hkey^4 + vaesenc $rndkey,$inout5,$inout5 + + vmovups 0x40-0x80($key),$rndkey + vpxor $T2,$Z2,$Z2 + vpclmulqdq \$0x00,$T1,$Ii,$T2 + vaesenc $rndkey,$inout0,$inout0 + vpxor $Hkey,$Z2,$Z2 + vpclmulqdq \$0x10,$T1,$Ii,$Hkey + vaesenc $rndkey,$inout1,$inout1 + movbe 0x48($in0),%r13 + vpxor $Z1,$Z3,$Z3 + vpclmulqdq \$0x01,$T1,$Ii,$Z1 + vaesenc $rndkey,$inout2,$inout2 + movbe 0x40($in0),%r12 + vpclmulqdq \$0x11,$T1,$Ii,$T1 + vmovdqu 0x60+8(%rsp),$Ii # I[1] + vaesenc $rndkey,$inout3,$inout3 + mov %r13,0x30+8(%rsp) + vaesenc $rndkey,$inout4,$inout4 + mov %r12,0x38+8(%rsp) + vpxor $T2,$Z0,$Z0 + vmovdqu 0x60-0x20($Xip),$T2 # borrow $T2 for $Hkey^5 + vaesenc $rndkey,$inout5,$inout5 + + vmovups 0x50-0x80($key),$rndkey + vpxor $Hkey,$Z2,$Z2 + vpclmulqdq \$0x00,$T2,$Ii,$Hkey + vaesenc $rndkey,$inout0,$inout0 + vpxor $Z1,$Z2,$Z2 + vpclmulqdq \$0x10,$T2,$Ii,$Z1 + vaesenc $rndkey,$inout1,$inout1 + movbe 0x38($in0),%r13 + vpxor $T1,$Z3,$Z3 + vpclmulqdq \$0x01,$T2,$Ii,$T1 + vpxor 0x70+8(%rsp),$Xi,$Xi # accumulate I[0] + vaesenc $rndkey,$inout2,$inout2 + movbe 0x30($in0),%r12 + vpclmulqdq \$0x11,$T2,$Ii,$T2 + vaesenc $rndkey,$inout3,$inout3 + mov %r13,0x40+8(%rsp) + vaesenc $rndkey,$inout4,$inout4 + mov %r12,0x48+8(%rsp) + vpxor $Hkey,$Z0,$Z0 + vmovdqu 0x70-0x20($Xip),$Hkey # $Hkey^6 + vaesenc $rndkey,$inout5,$inout5 + + vmovups 0x60-0x80($key),$rndkey + vpxor $Z1,$Z2,$Z2 + vpclmulqdq \$0x10,$Hkey,$Xi,$Z1 + vaesenc $rndkey,$inout0,$inout0 + vpxor $T1,$Z2,$Z2 + vpclmulqdq \$0x01,$Hkey,$Xi,$T1 + vaesenc $rndkey,$inout1,$inout1 + movbe 0x28($in0),%r13 + vpxor $T2,$Z3,$Z3 + vpclmulqdq \$0x00,$Hkey,$Xi,$T2 + vaesenc $rndkey,$inout2,$inout2 + movbe 0x20($in0),%r12 + vpclmulqdq \$0x11,$Hkey,$Xi,$Xi + vaesenc $rndkey,$inout3,$inout3 + mov %r13,0x50+8(%rsp) + vaesenc $rndkey,$inout4,$inout4 + mov %r12,0x58+8(%rsp) + vpxor $Z1,$Z2,$Z2 + vaesenc $rndkey,$inout5,$inout5 + vpxor $T1,$Z2,$Z2 + + vmovups 0x70-0x80($key),$rndkey + vpslldq \$8,$Z2,$Z1 + vpxor $T2,$Z0,$Z0 + vmovdqu 0x10($const),$Hkey # .Lpoly + + vaesenc $rndkey,$inout0,$inout0 + vpxor $Xi,$Z3,$Z3 + vaesenc $rndkey,$inout1,$inout1 + vpxor $Z1,$Z0,$Z0 + movbe 0x18($in0),%r13 + vaesenc $rndkey,$inout2,$inout2 + movbe 0x10($in0),%r12 + vpalignr \$8,$Z0,$Z0,$Ii # 1st phase + vpclmulqdq \$0x10,$Hkey,$Z0,$Z0 + mov %r13,0x60+8(%rsp) + vaesenc $rndkey,$inout3,$inout3 + mov %r12,0x68+8(%rsp) + vaesenc $rndkey,$inout4,$inout4 + vmovups 0x80-0x80($key),$T1 # borrow $T1 for $rndkey + vaesenc $rndkey,$inout5,$inout5 + + vaesenc $T1,$inout0,$inout0 + vmovups 0x90-0x80($key),$rndkey + vaesenc $T1,$inout1,$inout1 + vpsrldq \$8,$Z2,$Z2 + vaesenc $T1,$inout2,$inout2 + vpxor $Z2,$Z3,$Z3 + vaesenc $T1,$inout3,$inout3 + vpxor $Ii,$Z0,$Z0 + movbe 0x08($in0),%r13 + vaesenc $T1,$inout4,$inout4 + movbe 0x00($in0),%r12 + vaesenc $T1,$inout5,$inout5 + vmovups 0xa0-0x80($key),$T1 + cmp \$11,$rounds + jb .Lenc_tail # 128-bit key + + vaesenc $rndkey,$inout0,$inout0 + vaesenc $rndkey,$inout1,$inout1 + vaesenc $rndkey,$inout2,$inout2 + vaesenc $rndkey,$inout3,$inout3 + vaesenc $rndkey,$inout4,$inout4 + vaesenc $rndkey,$inout5,$inout5 + + vaesenc $T1,$inout0,$inout0 + vaesenc $T1,$inout1,$inout1 + vaesenc $T1,$inout2,$inout2 + vaesenc $T1,$inout3,$inout3 + vaesenc $T1,$inout4,$inout4 + vmovups 0xb0-0x80($key),$rndkey + vaesenc $T1,$inout5,$inout5 + vmovups 0xc0-0x80($key),$T1 + je .Lenc_tail # 192-bit key + + vaesenc $rndkey,$inout0,$inout0 + vaesenc $rndkey,$inout1,$inout1 + vaesenc $rndkey,$inout2,$inout2 + vaesenc $rndkey,$inout3,$inout3 + vaesenc $rndkey,$inout4,$inout4 + vaesenc $rndkey,$inout5,$inout5 + + vaesenc $T1,$inout0,$inout0 + vaesenc $T1,$inout1,$inout1 + vaesenc $T1,$inout2,$inout2 + vaesenc $T1,$inout3,$inout3 + vaesenc $T1,$inout4,$inout4 + vmovups 0xd0-0x80($key),$rndkey + vaesenc $T1,$inout5,$inout5 + vmovups 0xe0-0x80($key),$T1 + jmp .Lenc_tail # 256-bit key + +.align 32 +.Lhandle_ctr32: + vmovdqu ($const),$Ii # borrow $Ii for .Lbswap_mask + vpshufb $Ii,$T1,$Z2 # byte-swap counter + vmovdqu 0x30($const),$Z1 # borrow $Z1, .Ltwo_lsb + vpaddd 0x40($const),$Z2,$inout1 # .Lone_lsb + vpaddd $Z1,$Z2,$inout2 + vmovdqu 0x00-0x20($Xip),$Hkey # $Hkey^1 + vpaddd $Z1,$inout1,$inout3 + vpshufb $Ii,$inout1,$inout1 + vpaddd $Z1,$inout2,$inout4 + vpshufb $Ii,$inout2,$inout2 + vpxor $rndkey,$inout1,$inout1 + vpaddd $Z1,$inout3,$inout5 + vpshufb $Ii,$inout3,$inout3 + vpxor $rndkey,$inout2,$inout2 + vpaddd $Z1,$inout4,$T1 # byte-swapped next counter value + vpshufb $Ii,$inout4,$inout4 + vpshufb $Ii,$inout5,$inout5 + vpshufb $Ii,$T1,$T1 # next counter value + jmp .Lresume_ctr32 + +.align 32 +.Lenc_tail: + vaesenc $rndkey,$inout0,$inout0 + vmovdqu $Z3,16+8(%rsp) # postpone vpxor $Z3,$Xi,$Xi + vpalignr \$8,$Z0,$Z0,$Xi # 2nd phase + vaesenc $rndkey,$inout1,$inout1 + vpclmulqdq \$0x10,$Hkey,$Z0,$Z0 + vpxor 0x00($inp),$T1,$T2 + vaesenc $rndkey,$inout2,$inout2 + vpxor 0x10($inp),$T1,$Ii + vaesenc $rndkey,$inout3,$inout3 + vpxor 0x20($inp),$T1,$Z1 + vaesenc $rndkey,$inout4,$inout4 + vpxor 0x30($inp),$T1,$Z2 + vaesenc $rndkey,$inout5,$inout5 + vpxor 0x40($inp),$T1,$Z3 + vpxor 0x50($inp),$T1,$Hkey + vmovdqu ($ivp),$T1 # load next counter value + + vaesenclast $T2,$inout0,$inout0 + vmovdqu 0x20($const),$T2 # borrow $T2, .Lone_msb + vaesenclast $Ii,$inout1,$inout1 + vpaddb $T2,$T1,$Ii + mov %r13,0x70+8(%rsp) + lea 0x60($inp),$inp + vaesenclast $Z1,$inout2,$inout2 + vpaddb $T2,$Ii,$Z1 + mov %r12,0x78+8(%rsp) + lea 0x60($out),$out + vmovdqu 0x00-0x80($key),$rndkey + vaesenclast $Z2,$inout3,$inout3 + vpaddb $T2,$Z1,$Z2 + vaesenclast $Z3, $inout4,$inout4 + vpaddb $T2,$Z2,$Z3 + vaesenclast $Hkey,$inout5,$inout5 + vpaddb $T2,$Z3,$Hkey + + add \$0x60,$ret + sub \$0x6,$len + jc .L6x_done + + vmovups $inout0,-0x60($out) # save output + vpxor $rndkey,$T1,$inout0 + vmovups $inout1,-0x50($out) + vmovdqa $Ii,$inout1 # 0 latency + vmovups $inout2,-0x40($out) + vmovdqa $Z1,$inout2 # 0 latency + vmovups $inout3,-0x30($out) + vmovdqa $Z2,$inout3 # 0 latency + vmovups $inout4,-0x20($out) + vmovdqa $Z3,$inout4 # 0 latency + vmovups $inout5,-0x10($out) + vmovdqa $Hkey,$inout5 # 0 latency + vmovdqu 0x20+8(%rsp),$Z3 # I[5] + jmp .Loop6x + +.L6x_done: + vpxor 16+8(%rsp),$Xi,$Xi # modulo-scheduled + vpxor $Z0,$Xi,$Xi # modulo-scheduled + + ret +.cfi_endproc +.size _aesni_ctr32_ghash_6x,.-_aesni_ctr32_ghash_6x +___ +###################################################################### +# +# size_t aesni_gcm_[en|de]crypt(const void *inp, void *out, size_t len, +# const AES_KEY *key, unsigned char iv[16], +# struct { u128 Xi,H,Htbl[9]; } *Xip); +$code.=<<___; +.globl aesni_gcm_decrypt +.type aesni_gcm_decrypt,\@function,6,"unwind" +.align 32 +aesni_gcm_decrypt: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp + xor $ret,$ret + cmp \$0x60,$len # minimal accepted length + jb .Lgcm_dec_abort + + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 +___ +$code.=<<___ if ($win64); + lea -0xa8(%rsp),%rsp +.cfi_alloca 0xa8 + movaps %xmm6,-0xd0(%rbp) + movaps %xmm7,-0xc0(%rbp) + movaps %xmm8,-0xb0(%rbp) + movaps %xmm9,-0xa0(%rbp) + movaps %xmm10,-0x90(%rbp) + movaps %xmm11,-0x80(%rbp) + movaps %xmm12,-0x70(%rbp) + movaps %xmm13,-0x60(%rbp) + movaps %xmm14,-0x50(%rbp) + movaps %xmm15,-0x40(%rbp) +.cfi_offset %xmm6-%xmm15,-0xe0 +___ +$code.=<<___; +.cfi_end_prologue + vzeroupper + + vmovdqu ($ivp),$T1 # input counter value + add \$-128,%rsp + mov 12($ivp),$counter + lea .Lbswap_mask(%rip),$const + lea -0x80($key),$in0 # borrow $in0 + mov \$0xf80,$end0 # borrow $end0 + vmovdqu ($Xip),$Xi # load Xi + and \$-128,%rsp # ensure stack alignment + vmovdqu ($const),$Ii # borrow $Ii for .Lbswap_mask + lea 0x80($key),$key # size optimization + lea 0x20+0x20($Xip),$Xip # size optimization + mov 0xf0-0x80($key),$rounds + vpshufb $Ii,$Xi,$Xi + + and $end0,$in0 + and %rsp,$end0 + sub $in0,$end0 + jc .Ldec_no_key_aliasing + cmp \$768,$end0 + jnc .Ldec_no_key_aliasing + sub $end0,%rsp # avoid aliasing with key +.Ldec_no_key_aliasing: + + vmovdqu 0x50($inp),$Z3 # I[5] + lea ($inp),$in0 + vmovdqu 0x40($inp),$Z0 + lea -0xc0($inp,$len),$end0 + vmovdqu 0x30($inp),$Z1 + shr \$4,$len + xor $ret,$ret + vmovdqu 0x20($inp),$Z2 + vpshufb $Ii,$Z3,$Z3 # passed to _aesni_ctr32_ghash_6x + vmovdqu 0x10($inp),$T2 + vpshufb $Ii,$Z0,$Z0 + vmovdqu ($inp),$Hkey + vpshufb $Ii,$Z1,$Z1 + vmovdqu $Z0,0x30(%rsp) + vpshufb $Ii,$Z2,$Z2 + vmovdqu $Z1,0x40(%rsp) + vpshufb $Ii,$T2,$T2 + vmovdqu $Z2,0x50(%rsp) + vpshufb $Ii,$Hkey,$Hkey + vmovdqu $T2,0x60(%rsp) + vmovdqu $Hkey,0x70(%rsp) + + call _aesni_ctr32_ghash_6x + + vmovups $inout0,-0x60($out) # save output + vmovups $inout1,-0x50($out) + vmovups $inout2,-0x40($out) + vmovups $inout3,-0x30($out) + vmovups $inout4,-0x20($out) + vmovups $inout5,-0x10($out) + + vpshufb ($const),$Xi,$Xi # .Lbswap_mask + vmovdqu $Xi,-0x40($Xip) # output Xi + + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0xd0(%rbp),%xmm6 + movaps -0xc0(%rbp),%xmm7 + movaps -0xb0(%rbp),%xmm8 + movaps -0xa0(%rbp),%xmm9 + movaps -0x90(%rbp),%xmm10 + movaps -0x80(%rbp),%xmm11 + movaps -0x70(%rbp),%xmm12 + movaps -0x60(%rbp),%xmm13 + movaps -0x50(%rbp),%xmm14 + movaps -0x40(%rbp),%xmm15 +___ +$code.=<<___; + mov -0x28(%rbp),%r15 + mov -0x20(%rbp),%r14 + mov -0x18(%rbp),%r13 + mov -0x10(%rbp),%r12 + mov -0x08(%rbp),%rbx + mov %rbp,%rsp # restore %rsp +.cfi_def_cfa_register %rsp +.Lgcm_dec_abort: + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size aesni_gcm_decrypt,.-aesni_gcm_decrypt +___ + +$code.=<<___; +.type _aesni_ctr32_6x,\@abi-omnipotent +.align 32 +_aesni_ctr32_6x: +.cfi_startproc + vmovdqu 0x00-0x80($key),$Z0 # borrow $Z0 for $rndkey + vmovdqu 0x20($const),$T2 # borrow $T2, .Lone_msb + lea -1($rounds),%r13 + vmovups 0x10-0x80($key),$rndkey + lea 0x20-0x80($key),%r12 + vpxor $Z0,$T1,$inout0 + add \$`6<<24`,$counter + jc .Lhandle_ctr32_2 + vpaddb $T2,$T1,$inout1 + vpaddb $T2,$inout1,$inout2 + vpxor $Z0,$inout1,$inout1 + vpaddb $T2,$inout2,$inout3 + vpxor $Z0,$inout2,$inout2 + vpaddb $T2,$inout3,$inout4 + vpxor $Z0,$inout3,$inout3 + vpaddb $T2,$inout4,$inout5 + vpxor $Z0,$inout4,$inout4 + vpaddb $T2,$inout5,$T1 + vpxor $Z0,$inout5,$inout5 + jmp .Loop_ctr32 + +.align 16 +.Loop_ctr32: + vaesenc $rndkey,$inout0,$inout0 + vaesenc $rndkey,$inout1,$inout1 + vaesenc $rndkey,$inout2,$inout2 + vaesenc $rndkey,$inout3,$inout3 + vaesenc $rndkey,$inout4,$inout4 + vaesenc $rndkey,$inout5,$inout5 + vmovups (%r12),$rndkey + lea 0x10(%r12),%r12 + dec %r13d + jnz .Loop_ctr32 + + vmovdqu (%r12),$Hkey # last round key + vaesenc $rndkey,$inout0,$inout0 + vpxor 0x00($inp),$Hkey,$Z0 + vaesenc $rndkey,$inout1,$inout1 + vpxor 0x10($inp),$Hkey,$Z1 + vaesenc $rndkey,$inout2,$inout2 + vpxor 0x20($inp),$Hkey,$Z2 + vaesenc $rndkey,$inout3,$inout3 + vpxor 0x30($inp),$Hkey,$Xi + vaesenc $rndkey,$inout4,$inout4 + vpxor 0x40($inp),$Hkey,$T2 + vaesenc $rndkey,$inout5,$inout5 + vpxor 0x50($inp),$Hkey,$Hkey + lea 0x60($inp),$inp + + vaesenclast $Z0,$inout0,$inout0 + vaesenclast $Z1,$inout1,$inout1 + vaesenclast $Z2,$inout2,$inout2 + vaesenclast $Xi,$inout3,$inout3 + vaesenclast $T2,$inout4,$inout4 + vaesenclast $Hkey,$inout5,$inout5 + vmovups $inout0,0x00($out) + vmovups $inout1,0x10($out) + vmovups $inout2,0x20($out) + vmovups $inout3,0x30($out) + vmovups $inout4,0x40($out) + vmovups $inout5,0x50($out) + lea 0x60($out),$out + + ret +.align 32 +.Lhandle_ctr32_2: + vpshufb $Ii,$T1,$Z2 # byte-swap counter + vmovdqu 0x30($const),$Z1 # borrow $Z1, .Ltwo_lsb + vpaddd 0x40($const),$Z2,$inout1 # .Lone_lsb + vpaddd $Z1,$Z2,$inout2 + vpaddd $Z1,$inout1,$inout3 + vpshufb $Ii,$inout1,$inout1 + vpaddd $Z1,$inout2,$inout4 + vpshufb $Ii,$inout2,$inout2 + vpxor $Z0,$inout1,$inout1 + vpaddd $Z1,$inout3,$inout5 + vpshufb $Ii,$inout3,$inout3 + vpxor $Z0,$inout2,$inout2 + vpaddd $Z1,$inout4,$T1 # byte-swapped next counter value + vpshufb $Ii,$inout4,$inout4 + vpxor $Z0,$inout3,$inout3 + vpshufb $Ii,$inout5,$inout5 + vpxor $Z0,$inout4,$inout4 + vpshufb $Ii,$T1,$T1 # next counter value + vpxor $Z0,$inout5,$inout5 + jmp .Loop_ctr32 +.cfi_endproc +.size _aesni_ctr32_6x,.-_aesni_ctr32_6x + +.globl aesni_gcm_encrypt +.type aesni_gcm_encrypt,\@function,6,"unwind" +.align 32 +aesni_gcm_encrypt: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp + xor $ret,$ret + cmp \$0x60*3,$len # minimal accepted length + jb .Lgcm_enc_abort + + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 +___ +$code.=<<___ if ($win64); + lea -0xa8(%rsp),%rsp +.cfi_alloca 0xa8 + movaps %xmm6,-0xd0(%rbp) + movaps %xmm7,-0xc0(%rbp) + movaps %xmm8,-0xb0(%rbp) + movaps %xmm9,-0xa0(%rbp) + movaps %xmm10,-0x90(%rbp) + movaps %xmm11,-0x80(%rbp) + movaps %xmm12,-0x70(%rbp) + movaps %xmm13,-0x60(%rbp) + movaps %xmm14,-0x50(%rbp) + movaps %xmm15,-0x40(%rbp) +.cfi_offset %xmm6-%xmm15,-0xe0 +___ +$code.=<<___; +.cfi_end_prologue + vzeroupper + + vmovdqu ($ivp),$T1 # input counter value + add \$-128,%rsp + mov 12($ivp),$counter + lea .Lbswap_mask(%rip),$const + lea -0x80($key),$in0 # borrow $in0 + mov \$0xf80,$end0 # borrow $end0 + lea 0x80($key),$key # size optimization + vmovdqu ($const),$Ii # borrow $Ii for .Lbswap_mask + and \$-128,%rsp # ensure stack alignment + mov 0xf0-0x80($key),$rounds + + and $end0,$in0 + and %rsp,$end0 + sub $in0,$end0 + jc .Lenc_no_key_aliasing + cmp \$768,$end0 + jnc .Lenc_no_key_aliasing + sub $end0,%rsp # avoid aliasing with key +.Lenc_no_key_aliasing: + + lea ($out),$in0 + lea -0xc0($out,$len),$end0 + shr \$4,$len + + call _aesni_ctr32_6x + vpshufb $Ii,$inout0,$Xi # save bswapped output on stack + vpshufb $Ii,$inout1,$T2 + vmovdqu $Xi,0x70(%rsp) + vpshufb $Ii,$inout2,$Z0 + vmovdqu $T2,0x60(%rsp) + vpshufb $Ii,$inout3,$Z1 + vmovdqu $Z0,0x50(%rsp) + vpshufb $Ii,$inout4,$Z2 + vmovdqu $Z1,0x40(%rsp) + vpshufb $Ii,$inout5,$Z3 # passed to _aesni_ctr32_ghash_6x + vmovdqu $Z2,0x30(%rsp) + + call _aesni_ctr32_6x + + vmovdqu ($Xip),$Xi # load Xi + lea 0x20+0x20($Xip),$Xip # size optimization + sub \$12,$len + mov \$0x60*2,$ret + vpshufb $Ii,$Xi,$Xi + + call _aesni_ctr32_ghash_6x + vmovdqu 0x20(%rsp),$Z3 # I[5] + vmovdqu ($const),$Ii # borrow $Ii for .Lbswap_mask + vmovdqu 0x00-0x20($Xip),$Hkey # $Hkey^1 + vpunpckhqdq $Z3,$Z3,$T1 + vmovdqu 0x20-0x20($Xip),$rndkey # borrow $rndkey for $HK + vmovups $inout0,-0x60($out) # save output + vpshufb $Ii,$inout0,$inout0 # but keep bswapped copy + vpxor $Z3,$T1,$T1 + vmovups $inout1,-0x50($out) + vpshufb $Ii,$inout1,$inout1 + vmovups $inout2,-0x40($out) + vpshufb $Ii,$inout2,$inout2 + vmovups $inout3,-0x30($out) + vpshufb $Ii,$inout3,$inout3 + vmovups $inout4,-0x20($out) + vpshufb $Ii,$inout4,$inout4 + vmovups $inout5,-0x10($out) + vpshufb $Ii,$inout5,$inout5 + vmovdqu $inout0,0x10(%rsp) # free $inout0 +___ +{ my ($HK,$T3)=($rndkey,$inout0); + +$code.=<<___; + vmovdqu 0x30(%rsp),$Z2 # I[4] + vmovdqu 0x10-0x20($Xip),$Ii # borrow $Ii for $Hkey^2 + vpunpckhqdq $Z2,$Z2,$T2 + vpclmulqdq \$0x00,$Hkey,$Z3,$Z1 + vpxor $Z2,$T2,$T2 + vpclmulqdq \$0x11,$Hkey,$Z3,$Z3 + vpclmulqdq \$0x00,$HK,$T1,$T1 + + vmovdqu 0x40(%rsp),$T3 # I[3] + vpclmulqdq \$0x00,$Ii,$Z2,$Z0 + vmovdqu 0x30-0x20($Xip),$Hkey # $Hkey^3 + vpxor $Z1,$Z0,$Z0 + vpunpckhqdq $T3,$T3,$Z1 + vpclmulqdq \$0x11,$Ii,$Z2,$Z2 + vpxor $T3,$Z1,$Z1 + vpxor $Z3,$Z2,$Z2 + vpclmulqdq \$0x10,$HK,$T2,$T2 + vmovdqu 0x50-0x20($Xip),$HK + vpxor $T1,$T2,$T2 + + vmovdqu 0x50(%rsp),$T1 # I[2] + vpclmulqdq \$0x00,$Hkey,$T3,$Z3 + vmovdqu 0x40-0x20($Xip),$Ii # borrow $Ii for $Hkey^4 + vpxor $Z0,$Z3,$Z3 + vpunpckhqdq $T1,$T1,$Z0 + vpclmulqdq \$0x11,$Hkey,$T3,$T3 + vpxor $T1,$Z0,$Z0 + vpxor $Z2,$T3,$T3 + vpclmulqdq \$0x00,$HK,$Z1,$Z1 + vpxor $T2,$Z1,$Z1 + + vmovdqu 0x60(%rsp),$T2 # I[1] + vpclmulqdq \$0x00,$Ii,$T1,$Z2 + vmovdqu 0x60-0x20($Xip),$Hkey # $Hkey^5 + vpxor $Z3,$Z2,$Z2 + vpunpckhqdq $T2,$T2,$Z3 + vpclmulqdq \$0x11,$Ii,$T1,$T1 + vpxor $T2,$Z3,$Z3 + vpxor $T3,$T1,$T1 + vpclmulqdq \$0x10,$HK,$Z0,$Z0 + vmovdqu 0x80-0x20($Xip),$HK + vpxor $Z1,$Z0,$Z0 + + vpxor 0x70(%rsp),$Xi,$Xi # accumulate I[0] + vpclmulqdq \$0x00,$Hkey,$T2,$Z1 + vmovdqu 0x70-0x20($Xip),$Ii # borrow $Ii for $Hkey^6 + vpunpckhqdq $Xi,$Xi,$T3 + vpxor $Z2,$Z1,$Z1 + vpclmulqdq \$0x11,$Hkey,$T2,$T2 + vpxor $Xi,$T3,$T3 + vpxor $T1,$T2,$T2 + vpclmulqdq \$0x00,$HK,$Z3,$Z3 + vpxor $Z0,$Z3,$Z0 + + vpclmulqdq \$0x00,$Ii,$Xi,$Z2 + vmovdqu 0x00-0x20($Xip),$Hkey # $Hkey^1 + vpunpckhqdq $inout5,$inout5,$T1 + vpclmulqdq \$0x11,$Ii,$Xi,$Xi + vpxor $inout5,$T1,$T1 + vpxor $Z1,$Z2,$Z1 + vpclmulqdq \$0x10,$HK,$T3,$T3 + vmovdqu 0x20-0x20($Xip),$HK + vpxor $T2,$Xi,$Z3 + vpxor $Z0,$T3,$Z2 + + vmovdqu 0x10-0x20($Xip),$Ii # borrow $Ii for $Hkey^2 + vpxor $Z1,$Z3,$T3 # aggregated Karatsuba post-processing + vpclmulqdq \$0x00,$Hkey,$inout5,$Z0 + vpxor $T3,$Z2,$Z2 + vpunpckhqdq $inout4,$inout4,$T2 + vpclmulqdq \$0x11,$Hkey,$inout5,$inout5 + vpxor $inout4,$T2,$T2 + vpslldq \$8,$Z2,$T3 + vpclmulqdq \$0x00,$HK,$T1,$T1 + vpxor $T3,$Z1,$Xi + vpsrldq \$8,$Z2,$Z2 + vpxor $Z2,$Z3,$Z3 + + vpclmulqdq \$0x00,$Ii,$inout4,$Z1 + vmovdqu 0x30-0x20($Xip),$Hkey # $Hkey^3 + vpxor $Z0,$Z1,$Z1 + vpunpckhqdq $inout3,$inout3,$T3 + vpclmulqdq \$0x11,$Ii,$inout4,$inout4 + vpxor $inout3,$T3,$T3 + vpxor $inout5,$inout4,$inout4 + vpalignr \$8,$Xi,$Xi,$inout5 # 1st phase + vpclmulqdq \$0x10,$HK,$T2,$T2 + vmovdqu 0x50-0x20($Xip),$HK + vpxor $T1,$T2,$T2 + + vpclmulqdq \$0x00,$Hkey,$inout3,$Z0 + vmovdqu 0x40-0x20($Xip),$Ii # borrow $Ii for $Hkey^4 + vpxor $Z1,$Z0,$Z0 + vpunpckhqdq $inout2,$inout2,$T1 + vpclmulqdq \$0x11,$Hkey,$inout3,$inout3 + vpxor $inout2,$T1,$T1 + vpxor $inout4,$inout3,$inout3 + vxorps 0x10(%rsp),$Z3,$Z3 # accumulate $inout0 + vpclmulqdq \$0x00,$HK,$T3,$T3 + vpxor $T2,$T3,$T3 + + vpclmulqdq \$0x10,0x10($const),$Xi,$Xi + vxorps $inout5,$Xi,$Xi + + vpclmulqdq \$0x00,$Ii,$inout2,$Z1 + vmovdqu 0x60-0x20($Xip),$Hkey # $Hkey^5 + vpxor $Z0,$Z1,$Z1 + vpunpckhqdq $inout1,$inout1,$T2 + vpclmulqdq \$0x11,$Ii,$inout2,$inout2 + vpxor $inout1,$T2,$T2 + vpalignr \$8,$Xi,$Xi,$inout5 # 2nd phase + vpxor $inout3,$inout2,$inout2 + vpclmulqdq \$0x10,$HK,$T1,$T1 + vmovdqu 0x80-0x20($Xip),$HK + vpxor $T3,$T1,$T1 + + vxorps $Z3,$inout5,$inout5 + vpclmulqdq \$0x10,0x10($const),$Xi,$Xi + vxorps $inout5,$Xi,$Xi + + vpclmulqdq \$0x00,$Hkey,$inout1,$Z0 + vmovdqu 0x70-0x20($Xip),$Ii # borrow $Ii for $Hkey^6 + vpxor $Z1,$Z0,$Z0 + vpunpckhqdq $Xi,$Xi,$T3 + vpclmulqdq \$0x11,$Hkey,$inout1,$inout1 + vpxor $Xi,$T3,$T3 + vpxor $inout2,$inout1,$inout1 + vpclmulqdq \$0x00,$HK,$T2,$T2 + vpxor $T1,$T2,$T2 + + vpclmulqdq \$0x00,$Ii,$Xi,$Z1 + vpclmulqdq \$0x11,$Ii,$Xi,$Z3 + vpxor $Z0,$Z1,$Z1 + vpclmulqdq \$0x10,$HK,$T3,$Z2 + vpxor $inout1,$Z3,$Z3 + vpxor $T2,$Z2,$Z2 + + vpxor $Z1,$Z3,$Z0 # aggregated Karatsuba post-processing + vpxor $Z0,$Z2,$Z2 + vpslldq \$8,$Z2,$T1 + vmovdqu 0x10($const),$Hkey # .Lpoly + vpsrldq \$8,$Z2,$Z2 + vpxor $T1,$Z1,$Xi + vpxor $Z2,$Z3,$Z3 + + vpalignr \$8,$Xi,$Xi,$T2 # 1st phase + vpclmulqdq \$0x10,$Hkey,$Xi,$Xi + vpxor $T2,$Xi,$Xi + + vpalignr \$8,$Xi,$Xi,$T2 # 2nd phase + vpclmulqdq \$0x10,$Hkey,$Xi,$Xi + vpxor $Z3,$T2,$T2 + vpxor $T2,$Xi,$Xi +___ +} +$code.=<<___; + vpshufb ($const),$Xi,$Xi # .Lbswap_mask + vmovdqu $Xi,-0x40($Xip) # output Xi + + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0xd0(%rbp),%xmm6 + movaps -0xc0(%rbp),%xmm7 + movaps -0xb0(%rbp),%xmm8 + movaps -0xa0(%rbp),%xmm9 + movaps -0x90(%rbp),%xmm10 + movaps -0x80(%rbp),%xmm11 + movaps -0x70(%rbp),%xmm12 + movaps -0x60(%rbp),%xmm13 + movaps -0x50(%rbp),%xmm14 + movaps -0x40(%rbp),%xmm15 +___ +$code.=<<___; + mov -0x28(%rbp),%r15 + mov -0x20(%rbp),%r14 + mov -0x18(%rbp),%r13 + mov -0x10(%rbp),%r12 + mov -0x08(%rbp),%rbx + mov %rbp,%rsp # restore %rsp +.cfi_def_cfa_register %rsp +.Lgcm_enc_abort: + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size aesni_gcm_encrypt,.-aesni_gcm_encrypt +___ + +$code.=<<___; +.align 64 +.Lbswap_mask: + .byte 15,14,13,12,11,10,9,8,7,6,5,4,3,2,1,0 +.Lpoly: + .byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0xc2 +.Lone_msb: + .byte 0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1 +.Ltwo_lsb: + .byte 2,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.Lone_lsb: + .byte 1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0 +.asciz "AES-NI GCM module for x86_64, CRYPTOGAMS by " +.align 64 +___ +}}} else {{{ +$code=<<___; # assembler is too old +.text + +.globl aesni_gcm_encrypt +.type aesni_gcm_encrypt,\@abi-omnipotent +aesni_gcm_encrypt: +.cfi_startproc + xor %eax,%eax + ret +.cfi_endproc +.size aesni_gcm_encrypt,.-aesni_gcm_encrypt + +.globl aesni_gcm_decrypt +.type aesni_gcm_decrypt,\@abi-omnipotent +aesni_gcm_decrypt: +.cfi_startproc + xor %eax,%eax + ret +.cfi_endproc +.size aesni_gcm_decrypt,.-aesni_gcm_decrypt +___ +}}} + +$code =~ s/\`([^\`]*)\`/eval($1)/gem; + +print $code; + +close STDOUT or die "error closing STDOUT: $!"; diff --git a/cbits/asm/arm-xlate.pl b/cbits/asm/arm-xlate.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/arm-xlate.pl @@ -0,0 +1,467 @@ +#! /usr/bin/env perl +# +# ARM assembler distiller/adapter by \@dot-asm. + +use strict; + +################################################################ +# Recognized "flavour"-s are: +# +# linux[32|64] GNU assembler, effectively pass-through +# ios[32|64] global symbols' decorations, PIC tweaks, etc. +# win[32|64] Visual Studio armasm-specific directives +# coff[32|64] e.g. clang --target=arm-windows ... +# cheri64 L64P128 platform +# +my $flavour = shift; + $flavour = "linux" if (!$flavour or $flavour eq "void"); + +my $output = shift; +open STDOUT,">$output" || die "can't open $output: $!"; + +my %GLOBALS; +my $dotinlocallabels = ($flavour !~ /ios/) ? 1 : 0; +my $in_proc; # used with 'windows' flavour + +################################################################ +# directives which need special treatment on different platforms +################################################################ +my $arch = sub { } if ($flavour !~ /linux|coff64/);# omit .arch +my $fpu = sub { } if ($flavour !~ /linux/); # omit .fpu + +my $rodata = sub { + SWITCH: for ($flavour) { + /linux|cheri/ && return ".section\t.rodata"; + /ios/ && return ".section\t__TEXT,__const"; + /coff/ && return ".section\t.rdata,\"dr\""; + /win/ && return "\tAREA\t|.rdata|,DATA,READONLY,ALIGN=8"; + last; + } +}; + +my $hidden = sub { + if ($flavour =~ /ios/) { ".private_extern\t".join(',',@_); } +} if ($flavour !~ /linux|cheri/); + +my $comm = sub { + my @args = split(/,\s*/,shift); + my $name = @args[0]; + my $global = \$GLOBALS{$name}; + my $ret; + + if ($flavour =~ /ios32/) { + $ret = ".comm\t_$name,@args[1]\n"; + $ret .= ".non_lazy_symbol_pointer\n"; + $ret .= "$name:\n"; + $ret .= ".indirect_symbol\t_$name\n"; + $ret .= ".long\t0\n"; + $ret .= ".previous"; + $name = "_$name"; + } elsif ($flavour =~ /ios64/) { + $name = "_$name"; + $ret = ".comm\t$name,@args[1]"; + } elsif ($flavour =~ /win/) { + $ret = "\tCOMMON\t|$name|,@args[1]"; + } elsif ($flavour =~ /coff/) { + $ret = ".comm\t$name,@args[1]"; + } else { + $ret = ".comm\t".join(',',@args); + } + + $$global = $name; + $ret; +}; + +my $globl = sub { + my $name = shift; + my $global = \$GLOBALS{$name}; + my $ret; + + SWITCH: for ($flavour) { + /ios/ && do { $name = "_$name"; last; }; + /win/ && do { $ret = ""; last; }; + } + + $ret = ".globl $name" if (!defined($ret)); + $$global = $name; + $ret; +}; +my $global = $globl; + +my $extern = sub { + &$globl(@_); + if ($flavour =~ /win/) { + return "\tEXTERN\t@_"; + } + return; # return nothing +}; + +my $type = sub { + my $arg = join(',',@_); + my $ret; + + SWITCH: for ($flavour) { + /ios32/ && do { if ($arg =~ /(\w+),\s*%function/) { + $ret = "#ifdef __thumb2__\n" . + ".thumb_func $1\n" . + "#endif"; + } + last; + }; + /win/ && do { if ($arg =~ /(\w+),\s*%(function|object)/) { + my $type = "[DATA]"; + if ($2 eq "function") { + $in_proc = $1; + $type = "[FUNC]"; + } + $ret = $GLOBALS{$1} ? "\tEXPORT\t|$1|$type" + : ""; + } + last; + }; + /coff/ && do { if ($arg =~ /(\w+),\s*%function/) { + $ret = ".def $1;\n". + ".type 32;\n". + ".endef"; + } + last; + }; + } + return $ret; +} if ($flavour !~ /linux|cheri/); + +my $size = sub { + if ($in_proc && $flavour =~ /win/) { + $in_proc = undef; + return "\tENDP"; + } +} if ($flavour !~ /linux|cheri/); + +my $inst = sub { + if ($flavour =~ /win/) { "\tDCDU\t".join(',',@_); } + else { ".long\t".join(',',@_); } +} if ($flavour !~ /linux|cheri/); + +my $asciz = sub { + my $line = join(",",@_); + if ($line =~ /^"(.*)"$/) + { if ($flavour =~ /win/) { + "\tDCB\t$line,0\n\tALIGN\t4"; + } else { + ".byte " . join(",",unpack("C*",$1),0) . "\n.align 2"; + } + } else { ""; } +}; + +my $align = sub { + "\tALIGN\t".2**@_[0]; +} if ($flavour =~ /win/); + $align = sub { + ".p2align\t".@_[0]; +} if ($flavour =~ /coff/); + +my $byte = sub { + "\tDCB\t".join(',',@_); +} if ($flavour =~ /win/); + +my $short = sub { + "\tDCWU\t".join(',',@_); +} if ($flavour =~ /win/); + +my $word = sub { + "\tDCDU\t".join(',',@_); +} if ($flavour =~ /win/); + +my $long = $word if ($flavour =~ /win/); + +my $quad = sub { + "\tDCQU\t".join(',',@_); +} if ($flavour =~ /win/); + +my $skip = sub { + "\tSPACE\t".shift; +} if ($flavour =~ /win/); + +my $code = sub { + "\tCODE@_[0]"; +} if ($flavour =~ /win/); + +my $thumb = sub { # .thumb should appear prior .text in source + "# define ARM THUMB\n" . + "\tTHUMB"; +} if ($flavour =~ /win/); + +my $text = sub { + "\tAREA\t|.text|,CODE,ALIGN=8,".($flavour =~ /64/ ? "ARM64" : "ARM"); +} if ($flavour =~ /win/); + +my $syntax = sub {} if ($flavour =~ /win/); # omit .syntax + +my $rva = sub { + # .rva directive comes in handy only on 32-bit Windows, i.e. it can + # be used only in '#if defined(_WIN32) && !defined(_WIN64)' sections. + # However! Corresponding compilers don't seem to bet on PIC, which + # raises the question why would assembler programmer have to jump + # through the hoops? But just in case, it would go as following: + # + # ldr r1,.LOPENSSL_armcap + # ldr r2,.LOPENSSL_armcap+4 + # adr r0,.LOPENSSL_armcap + # bic r1,r1,#1 ; de-thumb-ify link.exe's ideas + # sub r0,r0,r1 ; r0 is image base now + # ldr r0,[r0,r2] + # ... + #.LOPENSSL_armcap: + # .rva .LOPENSSL_armcap ; self-reference + # .rva OPENSSL_armcap_P ; real target + # + # Non-position-independent [and ISA-neutral] alternative is so much + # simpler: + # + # ldr r0,.LOPENSSL_armcap + # ldr r0,[r0] + # ... + #.LOPENSSL_armcap: + # .long OPENSSL_armcap_P + # + "\tDCDU\t@_[0]\n\tRELOC\t2" +} if ($flavour =~ /win(?!64)/); + +################################################################ +# some broken instructions in Visual Studio armasm[64]... + +my $it = sub {} if ($flavour =~ /win32/); # omit 'it' + +my $ext = sub { + "\text8\t".join(',',@_); +} if ($flavour =~ /win64/); + +my $csel = sub { + my ($args,$comment) = split(m|\s*//|,shift); + my @regs = split(m|,\s*|,$args); + my $cond = pop(@regs); + + "\tcsel$cond\t".join(',',@regs); +} if ($flavour =~ /win64/); + +my $csetm = sub { + my ($args,$comment) = split(m|\s*//|,shift); + my @regs = split(m|,\s*|,$args); + my $cond = pop(@regs); + + "\tcsetm$cond\t".join(',',@regs); +} if ($flavour =~ /win64/); + +# ... then conditional branch instructions are also broken, but +# maintaining all the variants is tedious, so I kludge-fix it +# elsewhere... + +################################################################ +# CHERI-specific synthetic instructions +my $scvalue = sub { + my ($args,$comment) = split(m|\s*//|,shift); + $args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g; + my @regs = split(m|,\s*|,$args); + @regs[2] =~ s/\bc([0-9])\b/x$1/; + + "\tscvalue\t".join(',',@regs); +}; + +my $cadd = sub { + my ($args,$comment) = split(m|\s*//|,shift); + if ($flavour =~ /cheri/) { + $args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g; + } else { + $args =~ s/\bc([0-9]+)\b/x$1/g; + } + my @regs = split(m|,\s*|,$args); + @regs[2] =~ s/c([0-9])/x$1/; + + "\tadd\t".join(',',@regs); +}; + +my $csub = sub { + my ($args,$comment) = split(m|\s*//|,shift); + if ($flavour =~ /cheri/) { + $args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g; + } else { + $args =~ s/\bc([0-9]+)\b/x$1/g; + } + my @regs = split(m|,\s*|,$args); + @regs[2] =~ s/c([0-9])/x$1/; + + "\tsub\t".join(',',@regs); +}; + +my $cmov = sub { + my $args = shift; + if ($flavour =~ /cheri/) { + $args =~ s/\b(?:x([0-9]+)|(sp))\b/c$1$2/g; + } else { + $args =~ s/\bc([0-9]+)\b/x$1/g; + } + + "\tmov\t".$args; +}; + +my $adr = sub { + my $args = shift; + $args =~ s/\bx([0-9]+)\b/c$1/g; + + "\tadr\t".$args; +} if ($flavour =~ /cheri/); + +################################################################ +my $adrp = sub { + my ($args,$comment) = split(m|\s*//|,shift); + "\tadrp\t$args\@PAGE"; +} if ($flavour =~ /ios64/); + +my $paciasp = sub { + ($flavour =~ /linux|cheri/) ? "\t.inst\t0xd503233f" + : &$inst(0xd503233f); +}; + +my $autiasp = sub { + ($flavour =~ /linux|cheri/) ? "\t.inst\t0xd50323bf" + : &$inst(0xd50323bf); +}; + +sub range { + my ($r,$sfx,$start,$end) = @_; + + join(",",map("$r$_$sfx",($start..$end))); +} + +sub expand_line { + my $line = shift; + my @ret = (); + + pos($line)=0; + + while ($line =~ m/\G[^@\/\{\"]*/g) { + if ($line =~ m/\G(@|\/\/|$)/gc) { + last; + } + elsif ($line =~ m/\G\{/gc) { + my $saved_pos = pos($line); + $line =~ s/\G([rdqv])([0-9]+)([^\-]*)\-\1([0-9]+)\3/range($1,$3,$2,$4)/e; + pos($line) = $saved_pos; + $line =~ m/\G[^\}]*\}/g; + } + elsif ($line =~ m/\G\"/gc) { + $line =~ m/\G[^\"]*\"/g; + } + } + + $line =~ s/\b(\w+)/$GLOBALS{$1} or $1/ge; + + if ($flavour =~ /cheri/) { + $line =~ s/\[\s*(?:x([0-9]+)|(sp))\s*(,?.*)\]/[c$1$2$3]/; + } else { + $line =~ s/\bc((?:[0-9]+|zr))\b/x$1/g; + $line =~ s/\bcsp\b/sp/g; + } + + if ($flavour =~ /win/) { + # adjust alignment hints, "[rN,:32]" -> "[rN@32]" + $line =~ s/(\[\s*(?:r[0-9]+|sp))\s*,?\s*:([0-9]+\s*\])/$1\@$2/; + # adjust local labels, ".Lwhatever" -> "|$Lwhatever|" + $line =~ s/\.(L\w{2,})/|\$$1|/g; + # omit "#:lo12:" on win64 + $line =~ s/#:lo12://; + } elsif ($flavour =~ /coff(?!64)/) { + $line =~ s/\.L(\w{2,})/(\$ML$1)/g; + } elsif ($flavour =~ /ios64/) { + $line =~ s/#:lo12:(\w+)/$1\@PAGEOFF/; + } + + if ($flavour =~ /64/) { + # "vX.Md[N]" -> "vX.d[N] + $line =~ s/\b(v[0-9]+)\.[1-9]+([bhsd]\[[0-9]+\])/$1.$2/; + } + + return $line; +} + +if ($flavour =~ /win(32|64)/) { + print<<___; + GBLA __SIZEOF_POINTER__ +__SIZEOF_POINTER__ SETA $1/8 +___ +} + +while(my $line=<>) { + + if ($flavour =~ /win/) { + if ($line =~ m/^#\s*(ifdef|ifndef|else|endif)\b(.*)/) { + my ($op, $arg) = ($1, $2); + $op = "if :def:" if ($op eq "ifdef"); + $op = "if :lnot::def:" if ($op eq "ifndef"); + print " ".$op.$arg."\n"; + next; + } + $line =~ s|//.*||; + } + + # fix up assembler-specific commentary delimiter + $line =~ s/@(?=[\s@])/\;/g if ($flavour =~ /win|coff/); + + if ($line =~ m/^\s*(#|@|;|\/\/)/) { print $line; next; } + + $line =~ s|/\*.*\*/||; # get rid of C-style comments... + $line =~ s|^\s+||; # ... and skip white spaces in beginning... + $line =~ s|\s+$||; # ... and at the end + + { + $line =~ s|[\b\.]L(\w{2,})|L$1|g; # common denominator for Locallabel + $line =~ s|\bL(\w{2,})|\.L$1|g if ($dotinlocallabels); + } + + { + $line =~ s|(^[\.\w]+)\:\s*||; + my $label = $1; + if ($label) { + $label = ($GLOBALS{$label} or $label); + if ($flavour =~ /win/) { + $label =~ s|^\.L(?=\w)|\$L|; + printf "|%s|%s", $label, ($label eq $in_proc ? " PROC" : ""); + } else { + $label =~ s|^\.L(?=\w)|\$ML| if ($flavour =~ /coff(?!64)/); + printf "%s:", $label; + } + } + } + + if ($line !~ m/^[#@;]/) { + $line =~ s|^\s*(\.?)(\S+)\s*||; + my $c = $1; $c = "\t" if ($c eq ""); + my $mnemonic = $2; + my $opcode; + if ($mnemonic =~ m/([^\.]+)\.([^\.]+)/) { + $opcode = eval("\$$1_$2"); + } else { + $opcode = eval("\$$mnemonic"); + } + + my $arg=expand_line($line); + + if (ref($opcode) eq 'CODE') { + $line = &$opcode($arg); + } elsif ($mnemonic) { + if ($flavour =~ /win64/) { + # "b.cond" -> "bcond", kludge-fix:-( + $mnemonic =~ s/^b\.([a-z]{2}$)/b$1/; + } + $line = $c.$mnemonic; + $line.= "\t$arg" if ($arg ne ""); + } + } + + print $line if ($line); + print "\n"; +} + +print "\tEND\n" if ($flavour =~ /win/); + +close STDOUT; diff --git a/cbits/asm/arm_arch.h b/cbits/asm/arm_arch.h new file mode 100644 --- /dev/null +++ b/cbits/asm/arm_arch.h @@ -0,0 +1,101 @@ +#ifndef __ARM_ARCH_H__ +#define __ARM_ARCH_H__ + +#if !defined(__ARM_ARCH__) +# if defined(__CC_ARM) +# if __TARGET_ARCH_THUMB +# define __thumb__ +# if __TARGET_ARCH_THUMB >= 4 +# define __thumb2__ +# endif +# endif +# if __TARGET_ARCH_ARM +# define __ARM_ARCH__ __TARGET_ARCH_ARM +# else +# define __ARM_ARCH__ (__TARGET_ARCH_THUMB + 3) +# endif +# if defined(__BIG_ENDIAN) +# define __ARMEB__ +# else +# define __ARMEL__ +# endif +# elif defined(__GNUC__) || defined(__clang__) +# if defined(__aarch64__) +# define __ARM_ARCH__ 8 +# ifdef __AARCH64EB__ +# define __ARMEB__ +# else +# define __ARMEL__ +# endif +# elif defined(__ARM_ARCH) +# define __ARM_ARCH__ __ARM_ARCH + /* + * Why didn't gcc define __ARM_ARCH from start? Instead it defined + * bunch of below macros. See all_architectures[] table in + * gcc/config/arm/arm.c. On a side note it defines + * __ARMEL__/__ARMEB__ for little-/big-endian. + */ +# elif defined(__ARM_ARCH_8A__) +# define __ARM_ARCH__ 8 +# elif defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_7A__) || \ + defined(__ARM_ARCH_7R__)|| defined(__ARM_ARCH_7M__) || \ + defined(__ARM_ARCH_7EM__) +# define __ARM_ARCH__ 7 +# elif defined(__ARM_ARCH_6__) || defined(__ARM_ARCH_6J__) || \ + defined(__ARM_ARCH_6K__)|| defined(__ARM_ARCH_6M__) || \ + defined(__ARM_ARCH_6Z__)|| defined(__ARM_ARCH_6ZK__) || \ + defined(__ARM_ARCH_6T2__) +# define __ARM_ARCH__ 6 +# elif defined(__ARM_ARCH_5__) || defined(__ARM_ARCH_5T__) || \ + defined(__ARM_ARCH_5E__)|| defined(__ARM_ARCH_5TE__) || \ + defined(__ARM_ARCH_5TEJ__) +# define __ARM_ARCH__ 5 +# elif defined(__ARM_ARCH_4__) || defined(__ARM_ARCH_4T__) +# define __ARM_ARCH__ 4 +# else +# error "unsupported ARM architecture" +# endif +# elif defined(_MSC_VER) +# define __ARMEL__ +# if defined(_M_ARM) +# define __ARM_ARCH__ _M_ARM +# if defined(_M_THUMB) +# define __thumb__ +# if _M_THUMB >= 7 +# define __thumb2__ +# endif +# endif +# elif defined(_M_ARM64) +# define __AARCH64EL__ +# define __ARM_ARCH__ 8 +# else +# error "unsupported ARM architecture" +# endif +# endif +#endif + +#if !defined(__ARM_MAX_ARCH__) +# define __ARM_MAX_ARCH__ __ARM_ARCH__ +#endif + +#if __ARM_MAX_ARCH__<__ARM_ARCH__ +# error "__ARM_MAX_ARCH__ can't be less than __ARM_ARCH__" +#elif __ARM_MAX_ARCH__!=__ARM_ARCH__ +# if __ARM_ARCH__<7 && __ARM_MAX_ARCH__>=7 && defined(__ARMEB__) +# error "can't build universal big-endian binary" +# endif +#endif + +#ifndef __ASSEMBLER__ +extern unsigned int OPENSSL_armcap_P; +#endif + +#define ARMV7_NEON (1<<0) +#define ARMV7_TICK (1<<1) +#define ARMV8_AES (1<<2) +#define ARMV8_SHA1 (1<<3) +#define ARMV8_SHA256 (1<<4) +#define ARMV8_PMULL (1<<5) +#define ARMV8_SHA512 (1<<6) + +#endif diff --git a/cbits/asm/chacha-armv8-ios64.S b/cbits/asm/chacha-armv8-ios64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-armv8-ios64.S @@ -0,0 +1,2053 @@ +#ifndef __KERNEL__ +# include "arm_arch.h" + +#endif + +.text + +.align 5 +Lsigma: +.quad 0x3320646e61707865,0x6b20657479622d32 // endian-neutral +Lone: +.long 1,2,3,4 +Lrot24: +.long 0x02010003,0x06050407,0x0a09080b,0x0e0d0c0f +.byte 67,104,97,67,104,97,50,48,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 + +.globl _crypton_chacha20_asm_ctr32 + +.align 5 +_crypton_chacha20_asm_ctr32: + cbz x2,Labort + cmp x2,#192 + b.lo Lshort + +#ifndef __KERNEL__ + adrp x17,_crypton_armcap_P@PAGE + ldr w17,[x17,_crypton_armcap_P@PAGEOFF] + tst w17,#ARMV7_NEON + b.ne Lcrypton_chacha20_asm_neon +#endif + +Lshort: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + + adr x5,Lsigma + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#64 + + ldp x22,x23,[x5] // load sigma + ldp x24,x25,[x3] // load key + ldp x26,x27,[x3,#16] + ldp x28,x30,[x4] // load counter +#ifdef __AARCH64EB__ + ror x24,x24,#32 + ror x25,x25,#32 + ror x26,x26,#32 + ror x27,x27,#32 + ror x28,x28,#32 + ror x30,x30,#32 +#endif + +Loop_outer: + mov w5,w22 // unpack key block + lsr x6,x22,#32 + mov w7,w23 + lsr x8,x23,#32 + mov w9,w24 + lsr x10,x24,#32 + mov w11,w25 + lsr x12,x25,#32 + mov w13,w26 + lsr x14,x26,#32 + mov w15,w27 + lsr x16,x27,#32 + mov w17,w28 + lsr x19,x28,#32 + mov w20,w30 + lsr x21,x30,#32 + + mov x4,#10 + subs x2,x2,#64 +Loop: + sub x4,x4,#1 + add w5,w5,w9 + add w6,w6,w10 + add w7,w7,w11 + add w8,w8,w12 + eor w17,w17,w5 + eor w19,w19,w6 + eor w20,w20,w7 + eor w21,w21,w8 + ror w17,w17,#16 + ror w19,w19,#16 + ror w20,w20,#16 + ror w21,w21,#16 + add w13,w13,w17 + add w14,w14,w19 + add w15,w15,w20 + add w16,w16,w21 + eor w9,w9,w13 + eor w10,w10,w14 + eor w11,w11,w15 + eor w12,w12,w16 + ror w9,w9,#20 + ror w10,w10,#20 + ror w11,w11,#20 + ror w12,w12,#20 + add w5,w5,w9 + add w6,w6,w10 + add w7,w7,w11 + add w8,w8,w12 + eor w17,w17,w5 + eor w19,w19,w6 + eor w20,w20,w7 + eor w21,w21,w8 + ror w17,w17,#24 + ror w19,w19,#24 + ror w20,w20,#24 + ror w21,w21,#24 + add w13,w13,w17 + add w14,w14,w19 + add w15,w15,w20 + add w16,w16,w21 + eor w9,w9,w13 + eor w10,w10,w14 + eor w11,w11,w15 + eor w12,w12,w16 + ror w9,w9,#25 + ror w10,w10,#25 + ror w11,w11,#25 + ror w12,w12,#25 + add w5,w5,w10 + add w6,w6,w11 + add w7,w7,w12 + add w8,w8,w9 + eor w21,w21,w5 + eor w17,w17,w6 + eor w19,w19,w7 + eor w20,w20,w8 + ror w21,w21,#16 + ror w17,w17,#16 + ror w19,w19,#16 + ror w20,w20,#16 + add w15,w15,w21 + add w16,w16,w17 + add w13,w13,w19 + add w14,w14,w20 + eor w10,w10,w15 + eor w11,w11,w16 + eor w12,w12,w13 + eor w9,w9,w14 + ror w10,w10,#20 + ror w11,w11,#20 + ror w12,w12,#20 + ror w9,w9,#20 + add w5,w5,w10 + add w6,w6,w11 + add w7,w7,w12 + add w8,w8,w9 + eor w21,w21,w5 + eor w17,w17,w6 + eor w19,w19,w7 + eor w20,w20,w8 + ror w21,w21,#24 + ror w17,w17,#24 + ror w19,w19,#24 + ror w20,w20,#24 + add w15,w15,w21 + add w16,w16,w17 + add w13,w13,w19 + add w14,w14,w20 + eor w10,w10,w15 + eor w11,w11,w16 + eor w12,w12,w13 + eor w9,w9,w14 + ror w10,w10,#25 + ror w11,w11,#25 + ror w12,w12,#25 + ror w9,w9,#25 + cbnz x4,Loop + + add w5,w5,w22 // accumulate key block + add x6,x6,x22,lsr#32 + add w7,w7,w23 + add x8,x8,x23,lsr#32 + add w9,w9,w24 + add x10,x10,x24,lsr#32 + add w11,w11,w25 + add x12,x12,x25,lsr#32 + add w13,w13,w26 + add x14,x14,x26,lsr#32 + add w15,w15,w27 + add x16,x16,x27,lsr#32 + add w17,w17,w28 + add x19,x19,x28,lsr#32 + add w20,w20,w30 + add x21,x21,x30,lsr#32 + + b.lo Ltail + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor x15,x15,x16 + eor x17,x17,x19 + eor x20,x20,x21 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#1 // increment counter + stp x9,x11,[x0,#16] + stp x13,x15,[x0,#32] + stp x17,x20,[x0,#48] + add x0,x0,#64 + + b.hi Loop_outer + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp +Labort: + ret + +.align 4 +Ltail: + add x2,x2,#64 +Less_than_64: + sub x0,x0,#1 + add x1,x1,x2 + add x0,x0,x2 + add x4,sp,x2 + neg x2,x2 + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + stp x5,x7,[sp,#0] // off-load complete block + stp x9,x11,[sp,#16] + stp x13,x15,[sp,#32] + stp x17,x20,[sp,#48] + +Loop_tail: + ldrb w10,[x1,x2] + ldrb w11,[x4,x2] + add x2,x2,#1 + eor w10,w10,w11 + strb w10,[x0,x2] + cbnz x2,Loop_tail + + stp xzr,xzr,[sp,#0] // wipe off-load area + stp xzr,xzr,[sp,#16] + stp xzr,xzr,[sp,#32] + stp xzr,xzr,[sp,#48] + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + + +#ifdef __KERNEL__ +.globl _crypton_chacha20_asm_neon +#endif + +.align 5 +_crypton_chacha20_asm_neon: +Lcrypton_chacha20_asm_neon: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + + adr x5,Lsigma + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + cmp x2,#512 + b.hs L512_or_more_neon + + sub sp,sp,#64 + + ldp x22,x23,[x5] // load sigma + ld1 {v0.4s},[x5],#16 + ldp x24,x25,[x3] // load key + ldp x26,x27,[x3,#16] + ld1 {v1.4s,v2.4s},[x3] + ldp x28,x30,[x4] // load counter + ld1 {v3.4s},[x4] + stp d8,d9,[sp] // meet ABI requirements + ld1 {v8.4s,v9.4s},[x5] +#ifdef __AARCH64EB__ + rev64 v0.4s,v0.4s + ror x24,x24,#32 + ror x25,x25,#32 + ror x26,x26,#32 + ror x27,x27,#32 + ror x28,x28,#32 + ror x30,x30,#32 +#endif + +Loop_outer_neon: + dup v16.4s,v0.s[0] // unpack key block + mov w5,w22 + dup v20.4s,v0.s[1] + lsr x6,x22,#32 + dup v24.4s,v0.s[2] + mov w7,w23 + dup v28.4s,v0.s[3] + lsr x8,x23,#32 + dup v17.4s,v1.s[0] + mov w9,w24 + dup v21.4s,v1.s[1] + lsr x10,x24,#32 + dup v25.4s,v1.s[2] + mov w11,w25 + dup v29.4s,v1.s[3] + lsr x12,x25,#32 + dup v19.4s,v3.s[0] + mov w13,w26 + dup v23.4s,v3.s[1] + lsr x14,x26,#32 + dup v27.4s,v3.s[2] + mov w15,w27 + dup v31.4s,v3.s[3] + lsr x16,x27,#32 + add v19.4s,v19.4s,v8.4s + mov w17,w28 + dup v18.4s,v2.s[0] + lsr x19,x28,#32 + dup v22.4s,v2.s[1] + mov w20,w30 + dup v26.4s,v2.s[2] + lsr x21,x30,#32 + dup v30.4s,v2.s[3] + + mov x4,#10 + subs x2,x2,#320 +Loop_neon: + sub x4,x4,#1 + add v16.4s,v16.4s,v17.4s + add v20.4s,v20.4s,v21.4s + add v24.4s,v24.4s,v25.4s + add v28.4s,v28.4s,v29.4s + eor v19.16b,v19.16b,v16.16b + eor v23.16b,v23.16b,v20.16b + eor v27.16b,v27.16b,v24.16b + eor v31.16b,v31.16b,v28.16b + add w5,w5,w9 + rev32 v19.8h,v19.8h + add w6,w6,w10 + rev32 v23.8h,v23.8h + add w7,w7,w11 + rev32 v27.8h,v27.8h + add w8,w8,w12 + rev32 v31.8h,v31.8h + eor w17,w17,w5 + add v18.4s,v18.4s,v19.4s + eor w19,w19,w6 + add v22.4s,v22.4s,v23.4s + eor w20,w20,w7 + add v26.4s,v26.4s,v27.4s + eor w21,w21,w8 + add v30.4s,v30.4s,v31.4s + ror w17,w17,#16 + eor v4.16b,v17.16b,v18.16b + ror w19,w19,#16 + eor v5.16b,v21.16b,v22.16b + ror w20,w20,#16 + eor v6.16b,v25.16b,v26.16b + ror w21,w21,#16 + eor v7.16b,v29.16b,v30.16b + add w13,w13,w17 + ushr v17.4s,v4.4s,#20 + add w14,w14,w19 + ushr v21.4s,v5.4s,#20 + add w15,w15,w20 + ushr v25.4s,v6.4s,#20 + add w16,w16,w21 + ushr v29.4s,v7.4s,#20 + eor w9,w9,w13 + sli v17.4s,v4.4s,#12 + eor w10,w10,w14 + sli v21.4s,v5.4s,#12 + eor w11,w11,w15 + sli v25.4s,v6.4s,#12 + eor w12,w12,w16 + sli v29.4s,v7.4s,#12 + ror w9,w9,#20 + add v16.4s,v16.4s,v17.4s + ror w10,w10,#20 + add v20.4s,v20.4s,v21.4s + ror w11,w11,#20 + add v24.4s,v24.4s,v25.4s + ror w12,w12,#20 + add v28.4s,v28.4s,v29.4s + add w5,w5,w9 + eor v4.16b,v19.16b,v16.16b + add w6,w6,w10 + eor v5.16b,v23.16b,v20.16b + add w7,w7,w11 + eor v6.16b,v27.16b,v24.16b + add w8,w8,w12 + eor v7.16b,v31.16b,v28.16b + eor w17,w17,w5 + tbl v19.16b,{v4.16b},v9.16b + eor w19,w19,w6 + tbl v23.16b,{v5.16b},v9.16b + eor w20,w20,w7 + tbl v27.16b,{v6.16b},v9.16b + eor w21,w21,w8 + tbl v31.16b,{v7.16b},v9.16b + ror w17,w17,#24 + add v18.4s,v18.4s,v19.4s + ror w19,w19,#24 + add v22.4s,v22.4s,v23.4s + ror w20,w20,#24 + add v26.4s,v26.4s,v27.4s + ror w21,w21,#24 + add v30.4s,v30.4s,v31.4s + add w13,w13,w17 + eor v4.16b,v17.16b,v18.16b + add w14,w14,w19 + eor v5.16b,v21.16b,v22.16b + add w15,w15,w20 + eor v6.16b,v25.16b,v26.16b + add w16,w16,w21 + eor v7.16b,v29.16b,v30.16b + eor w9,w9,w13 + ushr v17.4s,v4.4s,#25 + eor w10,w10,w14 + ushr v21.4s,v5.4s,#25 + eor w11,w11,w15 + ushr v25.4s,v6.4s,#25 + eor w12,w12,w16 + ushr v29.4s,v7.4s,#25 + ror w9,w9,#25 + sli v17.4s,v4.4s,#7 + ror w10,w10,#25 + sli v21.4s,v5.4s,#7 + ror w11,w11,#25 + sli v25.4s,v6.4s,#7 + ror w12,w12,#25 + sli v29.4s,v7.4s,#7 + add v16.4s,v16.4s,v21.4s + add v20.4s,v20.4s,v25.4s + add v24.4s,v24.4s,v29.4s + add v28.4s,v28.4s,v17.4s + eor v31.16b,v31.16b,v16.16b + eor v19.16b,v19.16b,v20.16b + eor v23.16b,v23.16b,v24.16b + eor v27.16b,v27.16b,v28.16b + add w5,w5,w10 + rev32 v31.8h,v31.8h + add w6,w6,w11 + rev32 v19.8h,v19.8h + add w7,w7,w12 + rev32 v23.8h,v23.8h + add w8,w8,w9 + rev32 v27.8h,v27.8h + eor w21,w21,w5 + add v26.4s,v26.4s,v31.4s + eor w17,w17,w6 + add v30.4s,v30.4s,v19.4s + eor w19,w19,w7 + add v18.4s,v18.4s,v23.4s + eor w20,w20,w8 + add v22.4s,v22.4s,v27.4s + ror w21,w21,#16 + eor v4.16b,v21.16b,v26.16b + ror w17,w17,#16 + eor v5.16b,v25.16b,v30.16b + ror w19,w19,#16 + eor v6.16b,v29.16b,v18.16b + ror w20,w20,#16 + eor v7.16b,v17.16b,v22.16b + add w15,w15,w21 + ushr v21.4s,v4.4s,#20 + add w16,w16,w17 + ushr v25.4s,v5.4s,#20 + add w13,w13,w19 + ushr v29.4s,v6.4s,#20 + add w14,w14,w20 + ushr v17.4s,v7.4s,#20 + eor w10,w10,w15 + sli v21.4s,v4.4s,#12 + eor w11,w11,w16 + sli v25.4s,v5.4s,#12 + eor w12,w12,w13 + sli v29.4s,v6.4s,#12 + eor w9,w9,w14 + sli v17.4s,v7.4s,#12 + ror w10,w10,#20 + add v16.4s,v16.4s,v21.4s + ror w11,w11,#20 + add v20.4s,v20.4s,v25.4s + ror w12,w12,#20 + add v24.4s,v24.4s,v29.4s + ror w9,w9,#20 + add v28.4s,v28.4s,v17.4s + add w5,w5,w10 + eor v4.16b,v31.16b,v16.16b + add w6,w6,w11 + eor v5.16b,v19.16b,v20.16b + add w7,w7,w12 + eor v6.16b,v23.16b,v24.16b + add w8,w8,w9 + eor v7.16b,v27.16b,v28.16b + eor w21,w21,w5 + tbl v31.16b,{v4.16b},v9.16b + eor w17,w17,w6 + tbl v19.16b,{v5.16b},v9.16b + eor w19,w19,w7 + tbl v23.16b,{v6.16b},v9.16b + eor w20,w20,w8 + tbl v27.16b,{v7.16b},v9.16b + ror w21,w21,#24 + add v26.4s,v26.4s,v31.4s + ror w17,w17,#24 + add v30.4s,v30.4s,v19.4s + ror w19,w19,#24 + add v18.4s,v18.4s,v23.4s + ror w20,w20,#24 + add v22.4s,v22.4s,v27.4s + add w15,w15,w21 + eor v4.16b,v21.16b,v26.16b + add w16,w16,w17 + eor v5.16b,v25.16b,v30.16b + add w13,w13,w19 + eor v6.16b,v29.16b,v18.16b + add w14,w14,w20 + eor v7.16b,v17.16b,v22.16b + eor w10,w10,w15 + ushr v21.4s,v4.4s,#25 + eor w11,w11,w16 + ushr v25.4s,v5.4s,#25 + eor w12,w12,w13 + ushr v29.4s,v6.4s,#25 + eor w9,w9,w14 + ushr v17.4s,v7.4s,#25 + ror w10,w10,#25 + sli v21.4s,v4.4s,#7 + ror w11,w11,#25 + sli v25.4s,v5.4s,#7 + ror w12,w12,#25 + sli v29.4s,v6.4s,#7 + ror w9,w9,#25 + sli v17.4s,v7.4s,#7 + cbnz x4,Loop_neon + + add v19.4s,v19.4s,v8.4s + + zip1 v4.4s,v16.4s,v20.4s // transpose data + zip1 v5.4s,v24.4s,v28.4s + zip2 v6.4s,v16.4s,v20.4s + zip2 v7.4s,v24.4s,v28.4s + zip1 v16.2d,v4.2d,v5.2d + zip2 v20.2d,v4.2d,v5.2d + zip1 v24.2d,v6.2d,v7.2d + zip2 v28.2d,v6.2d,v7.2d + + zip1 v4.4s,v17.4s,v21.4s + zip1 v5.4s,v25.4s,v29.4s + zip2 v6.4s,v17.4s,v21.4s + zip2 v7.4s,v25.4s,v29.4s + zip1 v17.2d,v4.2d,v5.2d + zip2 v21.2d,v4.2d,v5.2d + zip1 v25.2d,v6.2d,v7.2d + zip2 v29.2d,v6.2d,v7.2d + + zip1 v4.4s,v18.4s,v22.4s + add w5,w5,w22 // accumulate key block + zip1 v5.4s,v26.4s,v30.4s + add x6,x6,x22,lsr#32 + zip2 v6.4s,v18.4s,v22.4s + add w7,w7,w23 + zip2 v7.4s,v26.4s,v30.4s + add x8,x8,x23,lsr#32 + zip1 v18.2d,v4.2d,v5.2d + add w9,w9,w24 + zip2 v22.2d,v4.2d,v5.2d + add x10,x10,x24,lsr#32 + zip1 v26.2d,v6.2d,v7.2d + add w11,w11,w25 + zip2 v30.2d,v6.2d,v7.2d + add x12,x12,x25,lsr#32 + + zip1 v4.4s,v19.4s,v23.4s + add w13,w13,w26 + zip1 v5.4s,v27.4s,v31.4s + add x14,x14,x26,lsr#32 + zip2 v6.4s,v19.4s,v23.4s + add w15,w15,w27 + zip2 v7.4s,v27.4s,v31.4s + add x16,x16,x27,lsr#32 + zip1 v19.2d,v4.2d,v5.2d + add w17,w17,w28 + zip2 v23.2d,v4.2d,v5.2d + add x19,x19,x28,lsr#32 + zip1 v27.2d,v6.2d,v7.2d + add w20,w20,w30 + zip2 v31.2d,v6.2d,v7.2d + add x21,x21,x30,lsr#32 + + b.lo Ltail_neon + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add v16.4s,v16.4s,v0.4s // accumulate key block + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add v17.4s,v17.4s,v1.4s + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add v18.4s,v18.4s,v2.4s + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add v19.4s,v19.4s,v3.4s + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor x5,x5,x6 + add v20.4s,v20.4s,v0.4s + eor x7,x7,x8 + add v21.4s,v21.4s,v1.4s + eor x9,x9,x10 + add v22.4s,v22.4s,v2.4s + eor x11,x11,x12 + add v23.4s,v23.4s,v3.4s + eor x13,x13,x14 + eor v16.16b,v16.16b,v4.16b + movi v4.4s,#5 + eor x15,x15,x16 + eor v17.16b,v17.16b,v5.16b + eor x17,x17,x19 + eor v18.16b,v18.16b,v6.16b + eor x20,x20,x21 + eor v19.16b,v19.16b,v7.16b + add v8.4s,v8.4s,v4.4s // += 5 + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#5 // increment counter + stp x9,x11,[x0,#16] + stp x13,x15,[x0,#32] + stp x17,x20,[x0,#48] + add x0,x0,#64 + + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64 + add v24.4s,v24.4s,v0.4s + add v25.4s,v25.4s,v1.4s + add v26.4s,v26.4s,v2.4s + add v27.4s,v27.4s,v3.4s + ld1 {v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64 + + eor v20.16b,v20.16b,v4.16b + eor v21.16b,v21.16b,v5.16b + eor v22.16b,v22.16b,v6.16b + eor v23.16b,v23.16b,v7.16b + st1 {v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64 + add v28.4s,v28.4s,v0.4s + add v29.4s,v29.4s,v1.4s + add v30.4s,v30.4s,v2.4s + add v31.4s,v31.4s,v3.4s + ld1 {v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64 + + eor v24.16b,v24.16b,v16.16b + eor v25.16b,v25.16b,v17.16b + eor v26.16b,v26.16b,v18.16b + eor v27.16b,v27.16b,v19.16b + st1 {v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64 + + eor v28.16b,v28.16b,v20.16b + eor v29.16b,v29.16b,v21.16b + eor v30.16b,v30.16b,v22.16b + eor v31.16b,v31.16b,v23.16b + st1 {v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64 + + b.hi Loop_outer_neon + + ldp d8,d9,[sp] // meet ABI requirements + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + +.align 4 +Ltail_neon: + add x2,x2,#320 + ldp d8,d9,[sp] // meet ABI requirements + cmp x2,#64 + b.lo Less_than_64_neon + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor x15,x15,x16 + eor x17,x17,x19 + eor x20,x20,x21 + + stp x5,x7,[x0,#0] // store output + add v16.4s,v16.4s,v0.4s // accumulate key block + stp x9,x11,[x0,#16] + add v17.4s,v17.4s,v1.4s + stp x13,x15,[x0,#32] + add v18.4s,v18.4s,v2.4s + stp x17,x20,[x0,#48] + add v19.4s,v19.4s,v3.4s + add x0,x0,#64 + b.eq Ldone_neon + sub x2,x2,#64 + cmp x2,#64 + b.lo Last_neon + + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor v16.16b,v16.16b,v4.16b + eor v17.16b,v17.16b,v5.16b + eor v18.16b,v18.16b,v6.16b + eor v19.16b,v19.16b,v7.16b + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64 + b.eq Ldone_neon + + add v16.4s,v20.4s,v0.4s + add v17.4s,v21.4s,v1.4s + sub x2,x2,#64 + add v18.4s,v22.4s,v2.4s + cmp x2,#64 + add v19.4s,v23.4s,v3.4s + b.lo Last_neon + + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor v20.16b,v16.16b,v4.16b + eor v21.16b,v17.16b,v5.16b + eor v22.16b,v18.16b,v6.16b + eor v23.16b,v19.16b,v7.16b + st1 {v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64 + b.eq Ldone_neon + + add v16.4s,v24.4s,v0.4s + add v17.4s,v25.4s,v1.4s + sub x2,x2,#64 + add v18.4s,v26.4s,v2.4s + cmp x2,#64 + add v19.4s,v27.4s,v3.4s + b.lo Last_neon + + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor v24.16b,v16.16b,v4.16b + eor v25.16b,v17.16b,v5.16b + eor v26.16b,v18.16b,v6.16b + eor v27.16b,v19.16b,v7.16b + st1 {v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64 + b.eq Ldone_neon + + add v16.4s,v28.4s,v0.4s + add v17.4s,v29.4s,v1.4s + add v18.4s,v30.4s,v2.4s + add v19.4s,v31.4s,v3.4s + sub x2,x2,#64 + +Last_neon: + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[sp] // off-load complete block + + sub x0,x0,#1 + add x1,x1,x2 + add x0,x0,x2 + add x4,sp,x2 + neg x2,x2 + +Loop_tail_neon: + ldrb w10,[x1,x2] + ldrb w11,[x4,x2] + add x2,x2,#1 + eor w10,w10,w11 + strb w10,[x0,x2] + cbnz x2,Loop_tail_neon + + stp q0,q0,[sp,#0] // wipe off-load area + stp q0,q0,[sp,#32] // [with known constant] + +Ldone_neon: + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + +.align 4 +Less_than_64_neon: + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + b Less_than_64 + + +.align 5 +crypton_chacha20_asm_512_neon: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + + adr x5,Lsigma + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + +L512_or_more_neon: + sub sp,sp,#128+64 + + eor v7.16b,v7.16b,v7.16b + ldp x22,x23,[x5] // load sigma + ld1 {v0.4s},[x5],#16 + ldp x24,x25,[x3] // load key + ldp x26,x27,[x3,#16] + ld1 {v1.4s,v2.4s},[x3] + ldp x28,x30,[x4] // load counter + ld1 {v3.4s},[x4] + ld1 {v7.s}[0],[x5] + add x3,x5,#16 +#ifdef __AARCH64EB__ + rev64 v0.4s,v0.4s + ror x24,x24,#32 + ror x25,x25,#32 + ror x26,x26,#32 + ror x27,x27,#32 + ror x28,x28,#32 + ror x30,x30,#32 +#endif + add v3.4s,v3.4s,v7.4s // += 1 + stp q0,q1,[sp,#0] // off-load key block, invariant part + add v3.4s,v3.4s,v7.4s // not typo + str q2,[sp,#32] + add v4.4s,v3.4s,v7.4s + add v5.4s,v4.4s,v7.4s + add v6.4s,v5.4s,v7.4s + shl v7.4s,v7.4s,#2 // 1 -> 4 + + stp d8,d9,[sp,#128+0] // meet ABI requirements + stp d10,d11,[sp,#128+16] + stp d12,d13,[sp,#128+32] + stp d14,d15,[sp,#128+48] + + sub x2,x2,#512 // not typo + +Loop_outer_512_neon: + mov v8.16b,v0.16b + mov v12.16b,v0.16b + mov v16.16b,v0.16b + mov v20.16b,v0.16b + mov v24.16b,v0.16b + mov v28.16b,v0.16b + mov v9.16b,v1.16b + mov w5,w22 // unpack key block + mov v13.16b,v1.16b + lsr x6,x22,#32 + mov v17.16b,v1.16b + mov w7,w23 + mov v21.16b,v1.16b + lsr x8,x23,#32 + mov v25.16b,v1.16b + mov w9,w24 + mov v29.16b,v1.16b + lsr x10,x24,#32 + mov v11.16b,v3.16b + mov w11,w25 + mov v15.16b,v4.16b + lsr x12,x25,#32 + mov v19.16b,v5.16b + mov w13,w26 + mov v23.16b,v6.16b + lsr x14,x26,#32 + mov v10.16b,v2.16b + mov w15,w27 + mov v14.16b,v2.16b + lsr x16,x27,#32 + add v27.4s,v11.4s,v7.4s // +4 + mov w17,w28 + add v31.4s,v15.4s,v7.4s // +4 + lsr x19,x28,#32 + mov v18.16b,v2.16b + mov w20,w30 + mov v22.16b,v2.16b + lsr x21,x30,#32 + mov v26.16b,v2.16b + stp q3,q4,[sp,#48] // off-load key block, variable part + mov v30.16b,v2.16b + stp q5,q6,[sp,#80] + + mov x4,#5 + ld1 {v6.4s},[x3] + subs x2,x2,#512 +Loop_upper_neon: + sub x4,x4,#1 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#12 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#12 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#12 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#12 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#12 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#12 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#4 + ext v13.16b,v13.16b,v13.16b,#4 + ext v17.16b,v17.16b,v17.16b,#4 + ext v21.16b,v21.16b,v21.16b,#4 + ext v25.16b,v25.16b,v25.16b,#4 + ext v29.16b,v29.16b,v29.16b,#4 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#4 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#4 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#4 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#4 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#4 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#4 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#12 + ext v13.16b,v13.16b,v13.16b,#12 + ext v17.16b,v17.16b,v17.16b,#12 + ext v21.16b,v21.16b,v21.16b,#12 + ext v25.16b,v25.16b,v25.16b,#12 + ext v29.16b,v29.16b,v29.16b,#12 + cbnz x4,Loop_upper_neon + + add w5,w5,w22 // accumulate key block + add x6,x6,x22,lsr#32 + add w7,w7,w23 + add x8,x8,x23,lsr#32 + add w9,w9,w24 + add x10,x10,x24,lsr#32 + add w11,w11,w25 + add x12,x12,x25,lsr#32 + add w13,w13,w26 + add x14,x14,x26,lsr#32 + add w15,w15,w27 + add x16,x16,x27,lsr#32 + add w17,w17,w28 + add x19,x19,x28,lsr#32 + add w20,w20,w30 + add x21,x21,x30,lsr#32 + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor x15,x15,x16 + eor x17,x17,x19 + eor x20,x20,x21 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#1 // increment counter + mov w5,w22 // unpack key block + lsr x6,x22,#32 + stp x9,x11,[x0,#16] + mov w7,w23 + lsr x8,x23,#32 + stp x13,x15,[x0,#32] + mov w9,w24 + lsr x10,x24,#32 + stp x17,x20,[x0,#48] + add x0,x0,#64 + mov w11,w25 + lsr x12,x25,#32 + mov w13,w26 + lsr x14,x26,#32 + mov w15,w27 + lsr x16,x27,#32 + mov w17,w28 + lsr x19,x28,#32 + mov w20,w30 + lsr x21,x30,#32 + + mov x4,#5 +Loop_lower_neon: + sub x4,x4,#1 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#12 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#12 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#12 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#12 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#12 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#12 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#4 + ext v13.16b,v13.16b,v13.16b,#4 + ext v17.16b,v17.16b,v17.16b,#4 + ext v21.16b,v21.16b,v21.16b,#4 + ext v25.16b,v25.16b,v25.16b,#4 + ext v29.16b,v29.16b,v29.16b,#4 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#4 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#4 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#4 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#4 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#4 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#4 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#12 + ext v13.16b,v13.16b,v13.16b,#12 + ext v17.16b,v17.16b,v17.16b,#12 + ext v21.16b,v21.16b,v21.16b,#12 + ext v25.16b,v25.16b,v25.16b,#12 + ext v29.16b,v29.16b,v29.16b,#12 + cbnz x4,Loop_lower_neon + + add w5,w5,w22 // accumulate key block + ldp q0,q1,[sp,#0] + add x6,x6,x22,lsr#32 + ldp q2,q3,[sp,#32] + add w7,w7,w23 + ldp q4,q5,[sp,#64] + add x8,x8,x23,lsr#32 + ldr q6,[sp,#96] + add v8.4s,v8.4s,v0.4s + add w9,w9,w24 + add v12.4s,v12.4s,v0.4s + add x10,x10,x24,lsr#32 + add v16.4s,v16.4s,v0.4s + add w11,w11,w25 + add v20.4s,v20.4s,v0.4s + add x12,x12,x25,lsr#32 + add v24.4s,v24.4s,v0.4s + add w13,w13,w26 + add v28.4s,v28.4s,v0.4s + add x14,x14,x26,lsr#32 + add v10.4s,v10.4s,v2.4s + add w15,w15,w27 + add v14.4s,v14.4s,v2.4s + add x16,x16,x27,lsr#32 + add v18.4s,v18.4s,v2.4s + add w17,w17,w28 + add v22.4s,v22.4s,v2.4s + add x19,x19,x28,lsr#32 + add v26.4s,v26.4s,v2.4s + add w20,w20,w30 + add v30.4s,v30.4s,v2.4s + add x21,x21,x30,lsr#32 + add v27.4s,v27.4s,v7.4s // +4 + add x5,x5,x6,lsl#32 // pack + add v31.4s,v31.4s,v7.4s // +4 + add x7,x7,x8,lsl#32 + add v11.4s,v11.4s,v3.4s + ldp x6,x8,[x1,#0] // load input + add v15.4s,v15.4s,v4.4s + add x9,x9,x10,lsl#32 + add v19.4s,v19.4s,v5.4s + add x11,x11,x12,lsl#32 + add v23.4s,v23.4s,v6.4s + ldp x10,x12,[x1,#16] + add v27.4s,v27.4s,v3.4s + add x13,x13,x14,lsl#32 + add v31.4s,v31.4s,v4.4s + add x15,x15,x16,lsl#32 + add v9.4s,v9.4s,v1.4s + ldp x14,x16,[x1,#32] + add v13.4s,v13.4s,v1.4s + add x17,x17,x19,lsl#32 + add v17.4s,v17.4s,v1.4s + add x20,x20,x21,lsl#32 + add v21.4s,v21.4s,v1.4s + ldp x19,x21,[x1,#48] + add v25.4s,v25.4s,v1.4s + add x1,x1,#64 + add v29.4s,v29.4s,v1.4s + +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + ld1 {v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64 + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor v8.16b,v8.16b,v0.16b + eor x15,x15,x16 + eor v9.16b,v9.16b,v1.16b + eor x17,x17,x19 + eor v10.16b,v10.16b,v2.16b + eor x20,x20,x21 + eor v11.16b,v11.16b,v3.16b + ld1 {v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#7 // increment counter + stp x9,x11,[x0,#16] + stp x13,x15,[x0,#32] + stp x17,x20,[x0,#48] + add x0,x0,#64 + st1 {v8.16b,v9.16b,v10.16b,v11.16b},[x0],#64 + + ld1 {v8.16b,v9.16b,v10.16b,v11.16b},[x1],#64 + eor v12.16b,v12.16b,v0.16b + eor v13.16b,v13.16b,v1.16b + eor v14.16b,v14.16b,v2.16b + eor v15.16b,v15.16b,v3.16b + st1 {v12.16b,v13.16b,v14.16b,v15.16b},[x0],#64 + + ld1 {v12.16b,v13.16b,v14.16b,v15.16b},[x1],#64 + eor v16.16b,v16.16b,v8.16b + ldp q0,q1,[sp,#0] + eor v17.16b,v17.16b,v9.16b + ldp q2,q3,[sp,#32] + eor v18.16b,v18.16b,v10.16b + eor v19.16b,v19.16b,v11.16b + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64 + + ld1 {v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64 + eor v20.16b,v20.16b,v12.16b + eor v21.16b,v21.16b,v13.16b + eor v22.16b,v22.16b,v14.16b + eor v23.16b,v23.16b,v15.16b + st1 {v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64 + + ld1 {v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64 + eor v24.16b,v24.16b,v16.16b + eor v25.16b,v25.16b,v17.16b + eor v26.16b,v26.16b,v18.16b + eor v27.16b,v27.16b,v19.16b + st1 {v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64 + + shl v8.4s,v7.4s,#1 // 4 -> 8 + eor v28.16b,v28.16b,v20.16b + eor v29.16b,v29.16b,v21.16b + eor v30.16b,v30.16b,v22.16b + eor v31.16b,v31.16b,v23.16b + st1 {v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64 + + add v3.4s,v3.4s,v8.4s // += 8 + add v4.4s,v4.4s,v8.4s + add v5.4s,v5.4s,v8.4s + add v6.4s,v6.4s,v8.4s + + b.hs Loop_outer_512_neon + + adds x2,x2,#512 + ushr v7.4s,v7.4s,#1 // 4 -> 2 + + ldp d10,d11,[sp,#128+16] // meet ABI requirements + ldp d12,d13,[sp,#128+32] + ldp d14,d15,[sp,#128+48] + + stp q0,q0,[sp,#16] // wipe key off-load area + stp q0,q0,[sp,#48] // [with known constant] + stp q0,q0,[sp,#80] + + b.eq Ldone_512_neon + + // we have <512 bytes tail, harmonize state with other contexts + sub x3,x3,#16 + cmp x2,#192 + add sp,sp,#128 + sub v3.4s,v3.4s,v7.4s // -= 2 + ld1 {v8.4s,v9.4s},[x3] + b.hs Loop_outer_neon + + ldp d8,d9,[sp,#0] // meet ABI requirements + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + eor v4.16b,v4.16b,v4.16b + eor v5.16b,v5.16b,v5.16b + eor v6.16b,v6.16b,v6.16b + b Loop_outer + +Ldone_512_neon: + ldp d8,d9,[sp,#128+0] // meet ABI requirements + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + eor v4.16b,v4.16b,v4.16b + eor v5.16b,v5.16b,v5.16b + eor v6.16b,v6.16b,v6.16b + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#128+64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + diff --git a/cbits/asm/chacha-armv8-linux64.S b/cbits/asm/chacha-armv8-linux64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-armv8-linux64.S @@ -0,0 +1,2055 @@ +#ifndef __KERNEL__ +# include "arm_arch.h" + +#endif + +.text + +.align 5 +.Lsigma: +.quad 0x3320646e61707865,0x6b20657479622d32 // endian-neutral +.Lone: +.long 1,2,3,4 +.Lrot24: +.long 0x02010003,0x06050407,0x0a09080b,0x0e0d0c0f +.byte 67,104,97,67,104,97,50,48,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 + +.globl crypton_chacha20_asm_ctr32 +.type crypton_chacha20_asm_ctr32,%function +.align 5 +crypton_chacha20_asm_ctr32: + cbz x2,.Labort + cmp x2,#192 + b.lo .Lshort + +#ifndef __KERNEL__ + adrp x17,crypton_armcap_P + ldr w17,[x17,#:lo12:crypton_armcap_P] + tst w17,#ARMV7_NEON + b.ne .Lcrypton_chacha20_asm_neon +#endif + +.Lshort: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + + adr x5,.Lsigma + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#64 + + ldp x22,x23,[x5] // load sigma + ldp x24,x25,[x3] // load key + ldp x26,x27,[x3,#16] + ldp x28,x30,[x4] // load counter +#ifdef __AARCH64EB__ + ror x24,x24,#32 + ror x25,x25,#32 + ror x26,x26,#32 + ror x27,x27,#32 + ror x28,x28,#32 + ror x30,x30,#32 +#endif + +.Loop_outer: + mov w5,w22 // unpack key block + lsr x6,x22,#32 + mov w7,w23 + lsr x8,x23,#32 + mov w9,w24 + lsr x10,x24,#32 + mov w11,w25 + lsr x12,x25,#32 + mov w13,w26 + lsr x14,x26,#32 + mov w15,w27 + lsr x16,x27,#32 + mov w17,w28 + lsr x19,x28,#32 + mov w20,w30 + lsr x21,x30,#32 + + mov x4,#10 + subs x2,x2,#64 +.Loop: + sub x4,x4,#1 + add w5,w5,w9 + add w6,w6,w10 + add w7,w7,w11 + add w8,w8,w12 + eor w17,w17,w5 + eor w19,w19,w6 + eor w20,w20,w7 + eor w21,w21,w8 + ror w17,w17,#16 + ror w19,w19,#16 + ror w20,w20,#16 + ror w21,w21,#16 + add w13,w13,w17 + add w14,w14,w19 + add w15,w15,w20 + add w16,w16,w21 + eor w9,w9,w13 + eor w10,w10,w14 + eor w11,w11,w15 + eor w12,w12,w16 + ror w9,w9,#20 + ror w10,w10,#20 + ror w11,w11,#20 + ror w12,w12,#20 + add w5,w5,w9 + add w6,w6,w10 + add w7,w7,w11 + add w8,w8,w12 + eor w17,w17,w5 + eor w19,w19,w6 + eor w20,w20,w7 + eor w21,w21,w8 + ror w17,w17,#24 + ror w19,w19,#24 + ror w20,w20,#24 + ror w21,w21,#24 + add w13,w13,w17 + add w14,w14,w19 + add w15,w15,w20 + add w16,w16,w21 + eor w9,w9,w13 + eor w10,w10,w14 + eor w11,w11,w15 + eor w12,w12,w16 + ror w9,w9,#25 + ror w10,w10,#25 + ror w11,w11,#25 + ror w12,w12,#25 + add w5,w5,w10 + add w6,w6,w11 + add w7,w7,w12 + add w8,w8,w9 + eor w21,w21,w5 + eor w17,w17,w6 + eor w19,w19,w7 + eor w20,w20,w8 + ror w21,w21,#16 + ror w17,w17,#16 + ror w19,w19,#16 + ror w20,w20,#16 + add w15,w15,w21 + add w16,w16,w17 + add w13,w13,w19 + add w14,w14,w20 + eor w10,w10,w15 + eor w11,w11,w16 + eor w12,w12,w13 + eor w9,w9,w14 + ror w10,w10,#20 + ror w11,w11,#20 + ror w12,w12,#20 + ror w9,w9,#20 + add w5,w5,w10 + add w6,w6,w11 + add w7,w7,w12 + add w8,w8,w9 + eor w21,w21,w5 + eor w17,w17,w6 + eor w19,w19,w7 + eor w20,w20,w8 + ror w21,w21,#24 + ror w17,w17,#24 + ror w19,w19,#24 + ror w20,w20,#24 + add w15,w15,w21 + add w16,w16,w17 + add w13,w13,w19 + add w14,w14,w20 + eor w10,w10,w15 + eor w11,w11,w16 + eor w12,w12,w13 + eor w9,w9,w14 + ror w10,w10,#25 + ror w11,w11,#25 + ror w12,w12,#25 + ror w9,w9,#25 + cbnz x4,.Loop + + add w5,w5,w22 // accumulate key block + add x6,x6,x22,lsr#32 + add w7,w7,w23 + add x8,x8,x23,lsr#32 + add w9,w9,w24 + add x10,x10,x24,lsr#32 + add w11,w11,w25 + add x12,x12,x25,lsr#32 + add w13,w13,w26 + add x14,x14,x26,lsr#32 + add w15,w15,w27 + add x16,x16,x27,lsr#32 + add w17,w17,w28 + add x19,x19,x28,lsr#32 + add w20,w20,w30 + add x21,x21,x30,lsr#32 + + b.lo .Ltail + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor x15,x15,x16 + eor x17,x17,x19 + eor x20,x20,x21 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#1 // increment counter + stp x9,x11,[x0,#16] + stp x13,x15,[x0,#32] + stp x17,x20,[x0,#48] + add x0,x0,#64 + + b.hi .Loop_outer + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp +.Labort: + ret + +.align 4 +.Ltail: + add x2,x2,#64 +.Less_than_64: + sub x0,x0,#1 + add x1,x1,x2 + add x0,x0,x2 + add x4,sp,x2 + neg x2,x2 + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + stp x5,x7,[sp,#0] // off-load complete block + stp x9,x11,[sp,#16] + stp x13,x15,[sp,#32] + stp x17,x20,[sp,#48] + +.Loop_tail: + ldrb w10,[x1,x2] + ldrb w11,[x4,x2] + add x2,x2,#1 + eor w10,w10,w11 + strb w10,[x0,x2] + cbnz x2,.Loop_tail + + stp xzr,xzr,[sp,#0] // wipe off-load area + stp xzr,xzr,[sp,#16] + stp xzr,xzr,[sp,#32] + stp xzr,xzr,[sp,#48] + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size crypton_chacha20_asm_ctr32,.-crypton_chacha20_asm_ctr32 + +#ifdef __KERNEL__ +.globl crypton_chacha20_asm_neon +#endif +.type crypton_chacha20_asm_neon,%function +.align 5 +crypton_chacha20_asm_neon: +.Lcrypton_chacha20_asm_neon: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + + adr x5,.Lsigma + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + cmp x2,#512 + b.hs .L512_or_more_neon + + sub sp,sp,#64 + + ldp x22,x23,[x5] // load sigma + ld1 {v0.4s},[x5],#16 + ldp x24,x25,[x3] // load key + ldp x26,x27,[x3,#16] + ld1 {v1.4s,v2.4s},[x3] + ldp x28,x30,[x4] // load counter + ld1 {v3.4s},[x4] + stp d8,d9,[sp] // meet ABI requirements + ld1 {v8.4s,v9.4s},[x5] +#ifdef __AARCH64EB__ + rev64 v0.4s,v0.4s + ror x24,x24,#32 + ror x25,x25,#32 + ror x26,x26,#32 + ror x27,x27,#32 + ror x28,x28,#32 + ror x30,x30,#32 +#endif + +.Loop_outer_neon: + dup v16.4s,v0.s[0] // unpack key block + mov w5,w22 + dup v20.4s,v0.s[1] + lsr x6,x22,#32 + dup v24.4s,v0.s[2] + mov w7,w23 + dup v28.4s,v0.s[3] + lsr x8,x23,#32 + dup v17.4s,v1.s[0] + mov w9,w24 + dup v21.4s,v1.s[1] + lsr x10,x24,#32 + dup v25.4s,v1.s[2] + mov w11,w25 + dup v29.4s,v1.s[3] + lsr x12,x25,#32 + dup v19.4s,v3.s[0] + mov w13,w26 + dup v23.4s,v3.s[1] + lsr x14,x26,#32 + dup v27.4s,v3.s[2] + mov w15,w27 + dup v31.4s,v3.s[3] + lsr x16,x27,#32 + add v19.4s,v19.4s,v8.4s + mov w17,w28 + dup v18.4s,v2.s[0] + lsr x19,x28,#32 + dup v22.4s,v2.s[1] + mov w20,w30 + dup v26.4s,v2.s[2] + lsr x21,x30,#32 + dup v30.4s,v2.s[3] + + mov x4,#10 + subs x2,x2,#320 +.Loop_neon: + sub x4,x4,#1 + add v16.4s,v16.4s,v17.4s + add v20.4s,v20.4s,v21.4s + add v24.4s,v24.4s,v25.4s + add v28.4s,v28.4s,v29.4s + eor v19.16b,v19.16b,v16.16b + eor v23.16b,v23.16b,v20.16b + eor v27.16b,v27.16b,v24.16b + eor v31.16b,v31.16b,v28.16b + add w5,w5,w9 + rev32 v19.8h,v19.8h + add w6,w6,w10 + rev32 v23.8h,v23.8h + add w7,w7,w11 + rev32 v27.8h,v27.8h + add w8,w8,w12 + rev32 v31.8h,v31.8h + eor w17,w17,w5 + add v18.4s,v18.4s,v19.4s + eor w19,w19,w6 + add v22.4s,v22.4s,v23.4s + eor w20,w20,w7 + add v26.4s,v26.4s,v27.4s + eor w21,w21,w8 + add v30.4s,v30.4s,v31.4s + ror w17,w17,#16 + eor v4.16b,v17.16b,v18.16b + ror w19,w19,#16 + eor v5.16b,v21.16b,v22.16b + ror w20,w20,#16 + eor v6.16b,v25.16b,v26.16b + ror w21,w21,#16 + eor v7.16b,v29.16b,v30.16b + add w13,w13,w17 + ushr v17.4s,v4.4s,#20 + add w14,w14,w19 + ushr v21.4s,v5.4s,#20 + add w15,w15,w20 + ushr v25.4s,v6.4s,#20 + add w16,w16,w21 + ushr v29.4s,v7.4s,#20 + eor w9,w9,w13 + sli v17.4s,v4.4s,#12 + eor w10,w10,w14 + sli v21.4s,v5.4s,#12 + eor w11,w11,w15 + sli v25.4s,v6.4s,#12 + eor w12,w12,w16 + sli v29.4s,v7.4s,#12 + ror w9,w9,#20 + add v16.4s,v16.4s,v17.4s + ror w10,w10,#20 + add v20.4s,v20.4s,v21.4s + ror w11,w11,#20 + add v24.4s,v24.4s,v25.4s + ror w12,w12,#20 + add v28.4s,v28.4s,v29.4s + add w5,w5,w9 + eor v4.16b,v19.16b,v16.16b + add w6,w6,w10 + eor v5.16b,v23.16b,v20.16b + add w7,w7,w11 + eor v6.16b,v27.16b,v24.16b + add w8,w8,w12 + eor v7.16b,v31.16b,v28.16b + eor w17,w17,w5 + tbl v19.16b,{v4.16b},v9.16b + eor w19,w19,w6 + tbl v23.16b,{v5.16b},v9.16b + eor w20,w20,w7 + tbl v27.16b,{v6.16b},v9.16b + eor w21,w21,w8 + tbl v31.16b,{v7.16b},v9.16b + ror w17,w17,#24 + add v18.4s,v18.4s,v19.4s + ror w19,w19,#24 + add v22.4s,v22.4s,v23.4s + ror w20,w20,#24 + add v26.4s,v26.4s,v27.4s + ror w21,w21,#24 + add v30.4s,v30.4s,v31.4s + add w13,w13,w17 + eor v4.16b,v17.16b,v18.16b + add w14,w14,w19 + eor v5.16b,v21.16b,v22.16b + add w15,w15,w20 + eor v6.16b,v25.16b,v26.16b + add w16,w16,w21 + eor v7.16b,v29.16b,v30.16b + eor w9,w9,w13 + ushr v17.4s,v4.4s,#25 + eor w10,w10,w14 + ushr v21.4s,v5.4s,#25 + eor w11,w11,w15 + ushr v25.4s,v6.4s,#25 + eor w12,w12,w16 + ushr v29.4s,v7.4s,#25 + ror w9,w9,#25 + sli v17.4s,v4.4s,#7 + ror w10,w10,#25 + sli v21.4s,v5.4s,#7 + ror w11,w11,#25 + sli v25.4s,v6.4s,#7 + ror w12,w12,#25 + sli v29.4s,v7.4s,#7 + add v16.4s,v16.4s,v21.4s + add v20.4s,v20.4s,v25.4s + add v24.4s,v24.4s,v29.4s + add v28.4s,v28.4s,v17.4s + eor v31.16b,v31.16b,v16.16b + eor v19.16b,v19.16b,v20.16b + eor v23.16b,v23.16b,v24.16b + eor v27.16b,v27.16b,v28.16b + add w5,w5,w10 + rev32 v31.8h,v31.8h + add w6,w6,w11 + rev32 v19.8h,v19.8h + add w7,w7,w12 + rev32 v23.8h,v23.8h + add w8,w8,w9 + rev32 v27.8h,v27.8h + eor w21,w21,w5 + add v26.4s,v26.4s,v31.4s + eor w17,w17,w6 + add v30.4s,v30.4s,v19.4s + eor w19,w19,w7 + add v18.4s,v18.4s,v23.4s + eor w20,w20,w8 + add v22.4s,v22.4s,v27.4s + ror w21,w21,#16 + eor v4.16b,v21.16b,v26.16b + ror w17,w17,#16 + eor v5.16b,v25.16b,v30.16b + ror w19,w19,#16 + eor v6.16b,v29.16b,v18.16b + ror w20,w20,#16 + eor v7.16b,v17.16b,v22.16b + add w15,w15,w21 + ushr v21.4s,v4.4s,#20 + add w16,w16,w17 + ushr v25.4s,v5.4s,#20 + add w13,w13,w19 + ushr v29.4s,v6.4s,#20 + add w14,w14,w20 + ushr v17.4s,v7.4s,#20 + eor w10,w10,w15 + sli v21.4s,v4.4s,#12 + eor w11,w11,w16 + sli v25.4s,v5.4s,#12 + eor w12,w12,w13 + sli v29.4s,v6.4s,#12 + eor w9,w9,w14 + sli v17.4s,v7.4s,#12 + ror w10,w10,#20 + add v16.4s,v16.4s,v21.4s + ror w11,w11,#20 + add v20.4s,v20.4s,v25.4s + ror w12,w12,#20 + add v24.4s,v24.4s,v29.4s + ror w9,w9,#20 + add v28.4s,v28.4s,v17.4s + add w5,w5,w10 + eor v4.16b,v31.16b,v16.16b + add w6,w6,w11 + eor v5.16b,v19.16b,v20.16b + add w7,w7,w12 + eor v6.16b,v23.16b,v24.16b + add w8,w8,w9 + eor v7.16b,v27.16b,v28.16b + eor w21,w21,w5 + tbl v31.16b,{v4.16b},v9.16b + eor w17,w17,w6 + tbl v19.16b,{v5.16b},v9.16b + eor w19,w19,w7 + tbl v23.16b,{v6.16b},v9.16b + eor w20,w20,w8 + tbl v27.16b,{v7.16b},v9.16b + ror w21,w21,#24 + add v26.4s,v26.4s,v31.4s + ror w17,w17,#24 + add v30.4s,v30.4s,v19.4s + ror w19,w19,#24 + add v18.4s,v18.4s,v23.4s + ror w20,w20,#24 + add v22.4s,v22.4s,v27.4s + add w15,w15,w21 + eor v4.16b,v21.16b,v26.16b + add w16,w16,w17 + eor v5.16b,v25.16b,v30.16b + add w13,w13,w19 + eor v6.16b,v29.16b,v18.16b + add w14,w14,w20 + eor v7.16b,v17.16b,v22.16b + eor w10,w10,w15 + ushr v21.4s,v4.4s,#25 + eor w11,w11,w16 + ushr v25.4s,v5.4s,#25 + eor w12,w12,w13 + ushr v29.4s,v6.4s,#25 + eor w9,w9,w14 + ushr v17.4s,v7.4s,#25 + ror w10,w10,#25 + sli v21.4s,v4.4s,#7 + ror w11,w11,#25 + sli v25.4s,v5.4s,#7 + ror w12,w12,#25 + sli v29.4s,v6.4s,#7 + ror w9,w9,#25 + sli v17.4s,v7.4s,#7 + cbnz x4,.Loop_neon + + add v19.4s,v19.4s,v8.4s + + zip1 v4.4s,v16.4s,v20.4s // transpose data + zip1 v5.4s,v24.4s,v28.4s + zip2 v6.4s,v16.4s,v20.4s + zip2 v7.4s,v24.4s,v28.4s + zip1 v16.2d,v4.2d,v5.2d + zip2 v20.2d,v4.2d,v5.2d + zip1 v24.2d,v6.2d,v7.2d + zip2 v28.2d,v6.2d,v7.2d + + zip1 v4.4s,v17.4s,v21.4s + zip1 v5.4s,v25.4s,v29.4s + zip2 v6.4s,v17.4s,v21.4s + zip2 v7.4s,v25.4s,v29.4s + zip1 v17.2d,v4.2d,v5.2d + zip2 v21.2d,v4.2d,v5.2d + zip1 v25.2d,v6.2d,v7.2d + zip2 v29.2d,v6.2d,v7.2d + + zip1 v4.4s,v18.4s,v22.4s + add w5,w5,w22 // accumulate key block + zip1 v5.4s,v26.4s,v30.4s + add x6,x6,x22,lsr#32 + zip2 v6.4s,v18.4s,v22.4s + add w7,w7,w23 + zip2 v7.4s,v26.4s,v30.4s + add x8,x8,x23,lsr#32 + zip1 v18.2d,v4.2d,v5.2d + add w9,w9,w24 + zip2 v22.2d,v4.2d,v5.2d + add x10,x10,x24,lsr#32 + zip1 v26.2d,v6.2d,v7.2d + add w11,w11,w25 + zip2 v30.2d,v6.2d,v7.2d + add x12,x12,x25,lsr#32 + + zip1 v4.4s,v19.4s,v23.4s + add w13,w13,w26 + zip1 v5.4s,v27.4s,v31.4s + add x14,x14,x26,lsr#32 + zip2 v6.4s,v19.4s,v23.4s + add w15,w15,w27 + zip2 v7.4s,v27.4s,v31.4s + add x16,x16,x27,lsr#32 + zip1 v19.2d,v4.2d,v5.2d + add w17,w17,w28 + zip2 v23.2d,v4.2d,v5.2d + add x19,x19,x28,lsr#32 + zip1 v27.2d,v6.2d,v7.2d + add w20,w20,w30 + zip2 v31.2d,v6.2d,v7.2d + add x21,x21,x30,lsr#32 + + b.lo .Ltail_neon + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add v16.4s,v16.4s,v0.4s // accumulate key block + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add v17.4s,v17.4s,v1.4s + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add v18.4s,v18.4s,v2.4s + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add v19.4s,v19.4s,v3.4s + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor x5,x5,x6 + add v20.4s,v20.4s,v0.4s + eor x7,x7,x8 + add v21.4s,v21.4s,v1.4s + eor x9,x9,x10 + add v22.4s,v22.4s,v2.4s + eor x11,x11,x12 + add v23.4s,v23.4s,v3.4s + eor x13,x13,x14 + eor v16.16b,v16.16b,v4.16b + movi v4.4s,#5 + eor x15,x15,x16 + eor v17.16b,v17.16b,v5.16b + eor x17,x17,x19 + eor v18.16b,v18.16b,v6.16b + eor x20,x20,x21 + eor v19.16b,v19.16b,v7.16b + add v8.4s,v8.4s,v4.4s // += 5 + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#5 // increment counter + stp x9,x11,[x0,#16] + stp x13,x15,[x0,#32] + stp x17,x20,[x0,#48] + add x0,x0,#64 + + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64 + add v24.4s,v24.4s,v0.4s + add v25.4s,v25.4s,v1.4s + add v26.4s,v26.4s,v2.4s + add v27.4s,v27.4s,v3.4s + ld1 {v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64 + + eor v20.16b,v20.16b,v4.16b + eor v21.16b,v21.16b,v5.16b + eor v22.16b,v22.16b,v6.16b + eor v23.16b,v23.16b,v7.16b + st1 {v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64 + add v28.4s,v28.4s,v0.4s + add v29.4s,v29.4s,v1.4s + add v30.4s,v30.4s,v2.4s + add v31.4s,v31.4s,v3.4s + ld1 {v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64 + + eor v24.16b,v24.16b,v16.16b + eor v25.16b,v25.16b,v17.16b + eor v26.16b,v26.16b,v18.16b + eor v27.16b,v27.16b,v19.16b + st1 {v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64 + + eor v28.16b,v28.16b,v20.16b + eor v29.16b,v29.16b,v21.16b + eor v30.16b,v30.16b,v22.16b + eor v31.16b,v31.16b,v23.16b + st1 {v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64 + + b.hi .Loop_outer_neon + + ldp d8,d9,[sp] // meet ABI requirements + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret + +.align 4 +.Ltail_neon: + add x2,x2,#320 + ldp d8,d9,[sp] // meet ABI requirements + cmp x2,#64 + b.lo .Less_than_64_neon + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor x15,x15,x16 + eor x17,x17,x19 + eor x20,x20,x21 + + stp x5,x7,[x0,#0] // store output + add v16.4s,v16.4s,v0.4s // accumulate key block + stp x9,x11,[x0,#16] + add v17.4s,v17.4s,v1.4s + stp x13,x15,[x0,#32] + add v18.4s,v18.4s,v2.4s + stp x17,x20,[x0,#48] + add v19.4s,v19.4s,v3.4s + add x0,x0,#64 + b.eq .Ldone_neon + sub x2,x2,#64 + cmp x2,#64 + b.lo .Last_neon + + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor v16.16b,v16.16b,v4.16b + eor v17.16b,v17.16b,v5.16b + eor v18.16b,v18.16b,v6.16b + eor v19.16b,v19.16b,v7.16b + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64 + b.eq .Ldone_neon + + add v16.4s,v20.4s,v0.4s + add v17.4s,v21.4s,v1.4s + sub x2,x2,#64 + add v18.4s,v22.4s,v2.4s + cmp x2,#64 + add v19.4s,v23.4s,v3.4s + b.lo .Last_neon + + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor v20.16b,v16.16b,v4.16b + eor v21.16b,v17.16b,v5.16b + eor v22.16b,v18.16b,v6.16b + eor v23.16b,v19.16b,v7.16b + st1 {v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64 + b.eq .Ldone_neon + + add v16.4s,v24.4s,v0.4s + add v17.4s,v25.4s,v1.4s + sub x2,x2,#64 + add v18.4s,v26.4s,v2.4s + cmp x2,#64 + add v19.4s,v27.4s,v3.4s + b.lo .Last_neon + + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + eor v24.16b,v16.16b,v4.16b + eor v25.16b,v17.16b,v5.16b + eor v26.16b,v18.16b,v6.16b + eor v27.16b,v19.16b,v7.16b + st1 {v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64 + b.eq .Ldone_neon + + add v16.4s,v28.4s,v0.4s + add v17.4s,v29.4s,v1.4s + add v18.4s,v30.4s,v2.4s + add v19.4s,v31.4s,v3.4s + sub x2,x2,#64 + +.Last_neon: + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[sp] // off-load complete block + + sub x0,x0,#1 + add x1,x1,x2 + add x0,x0,x2 + add x4,sp,x2 + neg x2,x2 + +.Loop_tail_neon: + ldrb w10,[x1,x2] + ldrb w11,[x4,x2] + add x2,x2,#1 + eor w10,w10,w11 + strb w10,[x0,x2] + cbnz x2,.Loop_tail_neon + + stp q0,q0,[sp,#0] // wipe off-load area + stp q0,q0,[sp,#32] // [with known constant] + +.Ldone_neon: + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret + +.align 4 +.Less_than_64_neon: + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + b .Less_than_64 +.size crypton_chacha20_asm_neon,.-crypton_chacha20_asm_neon +.type crypton_chacha20_asm_512_neon,%function +.align 5 +crypton_chacha20_asm_512_neon: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + + adr x5,.Lsigma + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + +.L512_or_more_neon: + sub sp,sp,#128+64 + + eor v7.16b,v7.16b,v7.16b + ldp x22,x23,[x5] // load sigma + ld1 {v0.4s},[x5],#16 + ldp x24,x25,[x3] // load key + ldp x26,x27,[x3,#16] + ld1 {v1.4s,v2.4s},[x3] + ldp x28,x30,[x4] // load counter + ld1 {v3.4s},[x4] + ld1 {v7.s}[0],[x5] + add x3,x5,#16 +#ifdef __AARCH64EB__ + rev64 v0.4s,v0.4s + ror x24,x24,#32 + ror x25,x25,#32 + ror x26,x26,#32 + ror x27,x27,#32 + ror x28,x28,#32 + ror x30,x30,#32 +#endif + add v3.4s,v3.4s,v7.4s // += 1 + stp q0,q1,[sp,#0] // off-load key block, invariant part + add v3.4s,v3.4s,v7.4s // not typo + str q2,[sp,#32] + add v4.4s,v3.4s,v7.4s + add v5.4s,v4.4s,v7.4s + add v6.4s,v5.4s,v7.4s + shl v7.4s,v7.4s,#2 // 1 -> 4 + + stp d8,d9,[sp,#128+0] // meet ABI requirements + stp d10,d11,[sp,#128+16] + stp d12,d13,[sp,#128+32] + stp d14,d15,[sp,#128+48] + + sub x2,x2,#512 // not typo + +.Loop_outer_512_neon: + mov v8.16b,v0.16b + mov v12.16b,v0.16b + mov v16.16b,v0.16b + mov v20.16b,v0.16b + mov v24.16b,v0.16b + mov v28.16b,v0.16b + mov v9.16b,v1.16b + mov w5,w22 // unpack key block + mov v13.16b,v1.16b + lsr x6,x22,#32 + mov v17.16b,v1.16b + mov w7,w23 + mov v21.16b,v1.16b + lsr x8,x23,#32 + mov v25.16b,v1.16b + mov w9,w24 + mov v29.16b,v1.16b + lsr x10,x24,#32 + mov v11.16b,v3.16b + mov w11,w25 + mov v15.16b,v4.16b + lsr x12,x25,#32 + mov v19.16b,v5.16b + mov w13,w26 + mov v23.16b,v6.16b + lsr x14,x26,#32 + mov v10.16b,v2.16b + mov w15,w27 + mov v14.16b,v2.16b + lsr x16,x27,#32 + add v27.4s,v11.4s,v7.4s // +4 + mov w17,w28 + add v31.4s,v15.4s,v7.4s // +4 + lsr x19,x28,#32 + mov v18.16b,v2.16b + mov w20,w30 + mov v22.16b,v2.16b + lsr x21,x30,#32 + mov v26.16b,v2.16b + stp q3,q4,[sp,#48] // off-load key block, variable part + mov v30.16b,v2.16b + stp q5,q6,[sp,#80] + + mov x4,#5 + ld1 {v6.4s},[x3] + subs x2,x2,#512 +.Loop_upper_neon: + sub x4,x4,#1 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#12 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#12 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#12 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#12 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#12 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#12 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#4 + ext v13.16b,v13.16b,v13.16b,#4 + ext v17.16b,v17.16b,v17.16b,#4 + ext v21.16b,v21.16b,v21.16b,#4 + ext v25.16b,v25.16b,v25.16b,#4 + ext v29.16b,v29.16b,v29.16b,#4 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#4 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#4 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#4 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#4 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#4 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#4 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#12 + ext v13.16b,v13.16b,v13.16b,#12 + ext v17.16b,v17.16b,v17.16b,#12 + ext v21.16b,v21.16b,v21.16b,#12 + ext v25.16b,v25.16b,v25.16b,#12 + ext v29.16b,v29.16b,v29.16b,#12 + cbnz x4,.Loop_upper_neon + + add w5,w5,w22 // accumulate key block + add x6,x6,x22,lsr#32 + add w7,w7,w23 + add x8,x8,x23,lsr#32 + add w9,w9,w24 + add x10,x10,x24,lsr#32 + add w11,w11,w25 + add x12,x12,x25,lsr#32 + add w13,w13,w26 + add x14,x14,x26,lsr#32 + add w15,w15,w27 + add x16,x16,x27,lsr#32 + add w17,w17,w28 + add x19,x19,x28,lsr#32 + add w20,w20,w30 + add x21,x21,x30,lsr#32 + + add x5,x5,x6,lsl#32 // pack + add x7,x7,x8,lsl#32 + ldp x6,x8,[x1,#0] // load input + add x9,x9,x10,lsl#32 + add x11,x11,x12,lsl#32 + ldp x10,x12,[x1,#16] + add x13,x13,x14,lsl#32 + add x15,x15,x16,lsl#32 + ldp x14,x16,[x1,#32] + add x17,x17,x19,lsl#32 + add x20,x20,x21,lsl#32 + ldp x19,x21,[x1,#48] + add x1,x1,#64 +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor x15,x15,x16 + eor x17,x17,x19 + eor x20,x20,x21 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#1 // increment counter + mov w5,w22 // unpack key block + lsr x6,x22,#32 + stp x9,x11,[x0,#16] + mov w7,w23 + lsr x8,x23,#32 + stp x13,x15,[x0,#32] + mov w9,w24 + lsr x10,x24,#32 + stp x17,x20,[x0,#48] + add x0,x0,#64 + mov w11,w25 + lsr x12,x25,#32 + mov w13,w26 + lsr x14,x26,#32 + mov w15,w27 + lsr x16,x27,#32 + mov w17,w28 + lsr x19,x28,#32 + mov w20,w30 + lsr x21,x30,#32 + + mov x4,#5 +.Loop_lower_neon: + sub x4,x4,#1 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#12 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#12 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#12 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#12 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#12 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#12 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#4 + ext v13.16b,v13.16b,v13.16b,#4 + ext v17.16b,v17.16b,v17.16b,#4 + ext v21.16b,v21.16b,v21.16b,#4 + ext v25.16b,v25.16b,v25.16b,#4 + ext v29.16b,v29.16b,v29.16b,#4 + add v8.4s,v8.4s,v9.4s + add w5,w5,w9 + add v12.4s,v12.4s,v13.4s + add w6,w6,w10 + add v16.4s,v16.4s,v17.4s + add w7,w7,w11 + add v20.4s,v20.4s,v21.4s + add w8,w8,w12 + add v24.4s,v24.4s,v25.4s + eor w17,w17,w5 + add v28.4s,v28.4s,v29.4s + eor w19,w19,w6 + eor v11.16b,v11.16b,v8.16b + eor w20,w20,w7 + eor v15.16b,v15.16b,v12.16b + eor w21,w21,w8 + eor v19.16b,v19.16b,v16.16b + ror w17,w17,#16 + eor v23.16b,v23.16b,v20.16b + ror w19,w19,#16 + eor v27.16b,v27.16b,v24.16b + ror w20,w20,#16 + eor v31.16b,v31.16b,v28.16b + ror w21,w21,#16 + rev32 v11.8h,v11.8h + add w13,w13,w17 + rev32 v15.8h,v15.8h + add w14,w14,w19 + rev32 v19.8h,v19.8h + add w15,w15,w20 + rev32 v23.8h,v23.8h + add w16,w16,w21 + rev32 v27.8h,v27.8h + eor w9,w9,w13 + rev32 v31.8h,v31.8h + eor w10,w10,w14 + add v10.4s,v10.4s,v11.4s + eor w11,w11,w15 + add v14.4s,v14.4s,v15.4s + eor w12,w12,w16 + add v18.4s,v18.4s,v19.4s + ror w9,w9,#20 + add v22.4s,v22.4s,v23.4s + ror w10,w10,#20 + add v26.4s,v26.4s,v27.4s + ror w11,w11,#20 + add v30.4s,v30.4s,v31.4s + ror w12,w12,#20 + eor v0.16b,v9.16b,v10.16b + add w5,w5,w9 + eor v1.16b,v13.16b,v14.16b + add w6,w6,w10 + eor v2.16b,v17.16b,v18.16b + add w7,w7,w11 + eor v3.16b,v21.16b,v22.16b + add w8,w8,w12 + eor v4.16b,v25.16b,v26.16b + eor w17,w17,w5 + eor v5.16b,v29.16b,v30.16b + eor w19,w19,w6 + ushr v9.4s,v0.4s,#20 + eor w20,w20,w7 + ushr v13.4s,v1.4s,#20 + eor w21,w21,w8 + ushr v17.4s,v2.4s,#20 + ror w17,w17,#24 + ushr v21.4s,v3.4s,#20 + ror w19,w19,#24 + ushr v25.4s,v4.4s,#20 + ror w20,w20,#24 + ushr v29.4s,v5.4s,#20 + ror w21,w21,#24 + sli v9.4s,v0.4s,#12 + add w13,w13,w17 + sli v13.4s,v1.4s,#12 + add w14,w14,w19 + sli v17.4s,v2.4s,#12 + add w15,w15,w20 + sli v21.4s,v3.4s,#12 + add w16,w16,w21 + sli v25.4s,v4.4s,#12 + eor w9,w9,w13 + sli v29.4s,v5.4s,#12 + eor w10,w10,w14 + add v8.4s,v8.4s,v9.4s + eor w11,w11,w15 + add v12.4s,v12.4s,v13.4s + eor w12,w12,w16 + add v16.4s,v16.4s,v17.4s + ror w9,w9,#25 + add v20.4s,v20.4s,v21.4s + ror w10,w10,#25 + add v24.4s,v24.4s,v25.4s + ror w11,w11,#25 + add v28.4s,v28.4s,v29.4s + ror w12,w12,#25 + eor v11.16b,v11.16b,v8.16b + add w5,w5,w10 + eor v15.16b,v15.16b,v12.16b + add w6,w6,w11 + eor v19.16b,v19.16b,v16.16b + add w7,w7,w12 + eor v23.16b,v23.16b,v20.16b + add w8,w8,w9 + eor v27.16b,v27.16b,v24.16b + eor w21,w21,w5 + eor v31.16b,v31.16b,v28.16b + eor w17,w17,w6 + tbl v11.16b,{v11.16b},v6.16b + eor w19,w19,w7 + tbl v15.16b,{v15.16b},v6.16b + eor w20,w20,w8 + tbl v19.16b,{v19.16b},v6.16b + ror w21,w21,#16 + tbl v23.16b,{v23.16b},v6.16b + ror w17,w17,#16 + tbl v27.16b,{v27.16b},v6.16b + ror w19,w19,#16 + tbl v31.16b,{v31.16b},v6.16b + ror w20,w20,#16 + add v10.4s,v10.4s,v11.4s + add w15,w15,w21 + add v14.4s,v14.4s,v15.4s + add w16,w16,w17 + add v18.4s,v18.4s,v19.4s + add w13,w13,w19 + add v22.4s,v22.4s,v23.4s + add w14,w14,w20 + add v26.4s,v26.4s,v27.4s + eor w10,w10,w15 + add v30.4s,v30.4s,v31.4s + eor w11,w11,w16 + eor v0.16b,v9.16b,v10.16b + eor w12,w12,w13 + eor v1.16b,v13.16b,v14.16b + eor w9,w9,w14 + eor v2.16b,v17.16b,v18.16b + ror w10,w10,#20 + eor v3.16b,v21.16b,v22.16b + ror w11,w11,#20 + eor v4.16b,v25.16b,v26.16b + ror w12,w12,#20 + eor v5.16b,v29.16b,v30.16b + ror w9,w9,#20 + ushr v9.4s,v0.4s,#25 + add w5,w5,w10 + ushr v13.4s,v1.4s,#25 + add w6,w6,w11 + ushr v17.4s,v2.4s,#25 + add w7,w7,w12 + ushr v21.4s,v3.4s,#25 + add w8,w8,w9 + ushr v25.4s,v4.4s,#25 + eor w21,w21,w5 + ushr v29.4s,v5.4s,#25 + eor w17,w17,w6 + sli v9.4s,v0.4s,#7 + eor w19,w19,w7 + sli v13.4s,v1.4s,#7 + eor w20,w20,w8 + sli v17.4s,v2.4s,#7 + ror w21,w21,#24 + sli v21.4s,v3.4s,#7 + ror w17,w17,#24 + sli v25.4s,v4.4s,#7 + ror w19,w19,#24 + sli v29.4s,v5.4s,#7 + ror w20,w20,#24 + ext v10.16b,v10.16b,v10.16b,#8 + add w15,w15,w21 + ext v14.16b,v14.16b,v14.16b,#8 + add w16,w16,w17 + ext v18.16b,v18.16b,v18.16b,#8 + add w13,w13,w19 + ext v22.16b,v22.16b,v22.16b,#8 + add w14,w14,w20 + ext v26.16b,v26.16b,v26.16b,#8 + eor w10,w10,w15 + ext v30.16b,v30.16b,v30.16b,#8 + eor w11,w11,w16 + ext v11.16b,v11.16b,v11.16b,#4 + eor w12,w12,w13 + ext v15.16b,v15.16b,v15.16b,#4 + eor w9,w9,w14 + ext v19.16b,v19.16b,v19.16b,#4 + ror w10,w10,#25 + ext v23.16b,v23.16b,v23.16b,#4 + ror w11,w11,#25 + ext v27.16b,v27.16b,v27.16b,#4 + ror w12,w12,#25 + ext v31.16b,v31.16b,v31.16b,#4 + ror w9,w9,#25 + ext v9.16b,v9.16b,v9.16b,#12 + ext v13.16b,v13.16b,v13.16b,#12 + ext v17.16b,v17.16b,v17.16b,#12 + ext v21.16b,v21.16b,v21.16b,#12 + ext v25.16b,v25.16b,v25.16b,#12 + ext v29.16b,v29.16b,v29.16b,#12 + cbnz x4,.Loop_lower_neon + + add w5,w5,w22 // accumulate key block + ldp q0,q1,[sp,#0] + add x6,x6,x22,lsr#32 + ldp q2,q3,[sp,#32] + add w7,w7,w23 + ldp q4,q5,[sp,#64] + add x8,x8,x23,lsr#32 + ldr q6,[sp,#96] + add v8.4s,v8.4s,v0.4s + add w9,w9,w24 + add v12.4s,v12.4s,v0.4s + add x10,x10,x24,lsr#32 + add v16.4s,v16.4s,v0.4s + add w11,w11,w25 + add v20.4s,v20.4s,v0.4s + add x12,x12,x25,lsr#32 + add v24.4s,v24.4s,v0.4s + add w13,w13,w26 + add v28.4s,v28.4s,v0.4s + add x14,x14,x26,lsr#32 + add v10.4s,v10.4s,v2.4s + add w15,w15,w27 + add v14.4s,v14.4s,v2.4s + add x16,x16,x27,lsr#32 + add v18.4s,v18.4s,v2.4s + add w17,w17,w28 + add v22.4s,v22.4s,v2.4s + add x19,x19,x28,lsr#32 + add v26.4s,v26.4s,v2.4s + add w20,w20,w30 + add v30.4s,v30.4s,v2.4s + add x21,x21,x30,lsr#32 + add v27.4s,v27.4s,v7.4s // +4 + add x5,x5,x6,lsl#32 // pack + add v31.4s,v31.4s,v7.4s // +4 + add x7,x7,x8,lsl#32 + add v11.4s,v11.4s,v3.4s + ldp x6,x8,[x1,#0] // load input + add v15.4s,v15.4s,v4.4s + add x9,x9,x10,lsl#32 + add v19.4s,v19.4s,v5.4s + add x11,x11,x12,lsl#32 + add v23.4s,v23.4s,v6.4s + ldp x10,x12,[x1,#16] + add v27.4s,v27.4s,v3.4s + add x13,x13,x14,lsl#32 + add v31.4s,v31.4s,v4.4s + add x15,x15,x16,lsl#32 + add v9.4s,v9.4s,v1.4s + ldp x14,x16,[x1,#32] + add v13.4s,v13.4s,v1.4s + add x17,x17,x19,lsl#32 + add v17.4s,v17.4s,v1.4s + add x20,x20,x21,lsl#32 + add v21.4s,v21.4s,v1.4s + ldp x19,x21,[x1,#48] + add v25.4s,v25.4s,v1.4s + add x1,x1,#64 + add v29.4s,v29.4s,v1.4s + +#ifdef __AARCH64EB__ + rev x5,x5 + rev x7,x7 + rev x9,x9 + rev x11,x11 + rev x13,x13 + rev x15,x15 + rev x17,x17 + rev x20,x20 +#endif + ld1 {v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64 + eor x5,x5,x6 + eor x7,x7,x8 + eor x9,x9,x10 + eor x11,x11,x12 + eor x13,x13,x14 + eor v8.16b,v8.16b,v0.16b + eor x15,x15,x16 + eor v9.16b,v9.16b,v1.16b + eor x17,x17,x19 + eor v10.16b,v10.16b,v2.16b + eor x20,x20,x21 + eor v11.16b,v11.16b,v3.16b + ld1 {v0.16b,v1.16b,v2.16b,v3.16b},[x1],#64 + + stp x5,x7,[x0,#0] // store output + add x28,x28,#7 // increment counter + stp x9,x11,[x0,#16] + stp x13,x15,[x0,#32] + stp x17,x20,[x0,#48] + add x0,x0,#64 + st1 {v8.16b,v9.16b,v10.16b,v11.16b},[x0],#64 + + ld1 {v8.16b,v9.16b,v10.16b,v11.16b},[x1],#64 + eor v12.16b,v12.16b,v0.16b + eor v13.16b,v13.16b,v1.16b + eor v14.16b,v14.16b,v2.16b + eor v15.16b,v15.16b,v3.16b + st1 {v12.16b,v13.16b,v14.16b,v15.16b},[x0],#64 + + ld1 {v12.16b,v13.16b,v14.16b,v15.16b},[x1],#64 + eor v16.16b,v16.16b,v8.16b + ldp q0,q1,[sp,#0] + eor v17.16b,v17.16b,v9.16b + ldp q2,q3,[sp,#32] + eor v18.16b,v18.16b,v10.16b + eor v19.16b,v19.16b,v11.16b + st1 {v16.16b,v17.16b,v18.16b,v19.16b},[x0],#64 + + ld1 {v16.16b,v17.16b,v18.16b,v19.16b},[x1],#64 + eor v20.16b,v20.16b,v12.16b + eor v21.16b,v21.16b,v13.16b + eor v22.16b,v22.16b,v14.16b + eor v23.16b,v23.16b,v15.16b + st1 {v20.16b,v21.16b,v22.16b,v23.16b},[x0],#64 + + ld1 {v20.16b,v21.16b,v22.16b,v23.16b},[x1],#64 + eor v24.16b,v24.16b,v16.16b + eor v25.16b,v25.16b,v17.16b + eor v26.16b,v26.16b,v18.16b + eor v27.16b,v27.16b,v19.16b + st1 {v24.16b,v25.16b,v26.16b,v27.16b},[x0],#64 + + shl v8.4s,v7.4s,#1 // 4 -> 8 + eor v28.16b,v28.16b,v20.16b + eor v29.16b,v29.16b,v21.16b + eor v30.16b,v30.16b,v22.16b + eor v31.16b,v31.16b,v23.16b + st1 {v28.16b,v29.16b,v30.16b,v31.16b},[x0],#64 + + add v3.4s,v3.4s,v8.4s // += 8 + add v4.4s,v4.4s,v8.4s + add v5.4s,v5.4s,v8.4s + add v6.4s,v6.4s,v8.4s + + b.hs .Loop_outer_512_neon + + adds x2,x2,#512 + ushr v7.4s,v7.4s,#1 // 4 -> 2 + + ldp d10,d11,[sp,#128+16] // meet ABI requirements + ldp d12,d13,[sp,#128+32] + ldp d14,d15,[sp,#128+48] + + stp q0,q0,[sp,#16] // wipe key off-load area + stp q0,q0,[sp,#48] // [with known constant] + stp q0,q0,[sp,#80] + + b.eq .Ldone_512_neon + + // we have <512 bytes tail, harmonize state with other contexts + sub x3,x3,#16 + cmp x2,#192 + add sp,sp,#128 + sub v3.4s,v3.4s,v7.4s // -= 2 + ld1 {v8.4s,v9.4s},[x3] + b.hs .Loop_outer_neon + + ldp d8,d9,[sp,#0] // meet ABI requirements + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + eor v4.16b,v4.16b,v4.16b + eor v5.16b,v5.16b,v5.16b + eor v6.16b,v6.16b,v6.16b + b .Loop_outer + +.Ldone_512_neon: + ldp d8,d9,[sp,#128+0] // meet ABI requirements + eor v1.16b,v1.16b,v1.16b // cleanse key and nonce + eor v2.16b,v2.16b,v2.16b + eor v3.16b,v3.16b,v3.16b + eor v4.16b,v4.16b,v4.16b + eor v5.16b,v5.16b,v5.16b + eor v6.16b,v6.16b,v6.16b + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#128+64 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#12*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size crypton_chacha20_asm_512_neon,.-crypton_chacha20_asm_512_neon + +.section .note.GNU-stack,"",%progbits diff --git a/cbits/asm/chacha-armv8.pl b/cbits/asm/chacha-armv8.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-armv8.pl @@ -0,0 +1,1328 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# June 2015 +# +# ChaCha20 for ARMv8. +# +# April 2019 +# +# Replace 3xNEON+1xIALU code path with 4+1. 4+1 is actually fastest +# option on most(*), but not all, processors, yet 6+2 is retained. +# This is because penalties are considered tolerable in comparison to +# improvement on processors where 6+2 helps. Most notably +37% on +# ThunderX2. It's server-oriented processor which will have to serve +# as many requests as possible. While others are mostly clients, when +# performance doesn't have to be absolute top-notch, just fast enough, +# as majority of time is spent "entertaining" relatively slow human. +# +# Performance in cycles per byte out of large buffer. +# +# IALU/gcc-4.9 4xNEON+1xIALU 6xNEON+2xIALU +# +# Apple A7 5.50/+49% 2.72 1.60 +# Apple A14/M1 4.50/+27% 1.84 1.27 +# Cortex-A53 8.40/+80% 4.06 4.45(*) +# Cortex-A57 8.06/+43% 4.08 4.40(*) +# Cortex-A76 5.52 2.90 2.40 +# Cortex-X2 4.35 2.53 1.62 +# Cortex-X925 3.94 1.79 1.30 +# Denver 4.50/+82% 2.30 2.70(*) +# X-Gene 9.50/+46% 8.20 8.90(*) +# Mongoose 8.00/+44% 2.74 3.12(*) +# Kryo 8.17/+50% 4.47 4.65(*) +# ThunderX2 7.22/+48% 5.64 4.10 +# Snapdragon X 3.90 1.79 1.25 +# +# (*) slower than 4+1:-( + +$flavour=shift; +$output=shift; + +if ($flavour && $flavour ne "void") { + $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; + ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or + ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or + die "can't locate arm-xlate.pl"; + + open STDOUT,"| \"$^X\" $xlate $flavour $output"; +} else { + open STDOUT,">$output"; +} + +sub AUTOLOAD() # thunk [simplified] x86-style perlasm +{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://; $opcode =~ s/_/\./; + my $arg = pop; + $arg = "#$arg" if ($arg*1 eq $arg); + $code .= "\t$opcode\t".join(',',@_,$arg)."\n"; +} + +my ($out,$inp,$len,$key,$ctr) = map("x$_",(0..4)); + +my @x=map("x$_",(5..17,19..21)); +my @d=map("x$_",(22..28,30)); + +sub ROUND { +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); + + ( + "&add_32 (@x[$a0],@x[$a0],@x[$b0])", + "&add_32 (@x[$a1],@x[$a1],@x[$b1])", + "&add_32 (@x[$a2],@x[$a2],@x[$b2])", + "&add_32 (@x[$a3],@x[$a3],@x[$b3])", + "&eor_32 (@x[$d0],@x[$d0],@x[$a0])", + "&eor_32 (@x[$d1],@x[$d1],@x[$a1])", + "&eor_32 (@x[$d2],@x[$d2],@x[$a2])", + "&eor_32 (@x[$d3],@x[$d3],@x[$a3])", + "&ror_32 (@x[$d0],@x[$d0],16)", + "&ror_32 (@x[$d1],@x[$d1],16)", + "&ror_32 (@x[$d2],@x[$d2],16)", + "&ror_32 (@x[$d3],@x[$d3],16)", + + "&add_32 (@x[$c0],@x[$c0],@x[$d0])", + "&add_32 (@x[$c1],@x[$c1],@x[$d1])", + "&add_32 (@x[$c2],@x[$c2],@x[$d2])", + "&add_32 (@x[$c3],@x[$c3],@x[$d3])", + "&eor_32 (@x[$b0],@x[$b0],@x[$c0])", + "&eor_32 (@x[$b1],@x[$b1],@x[$c1])", + "&eor_32 (@x[$b2],@x[$b2],@x[$c2])", + "&eor_32 (@x[$b3],@x[$b3],@x[$c3])", + "&ror_32 (@x[$b0],@x[$b0],20)", + "&ror_32 (@x[$b1],@x[$b1],20)", + "&ror_32 (@x[$b2],@x[$b2],20)", + "&ror_32 (@x[$b3],@x[$b3],20)", + + "&add_32 (@x[$a0],@x[$a0],@x[$b0])", + "&add_32 (@x[$a1],@x[$a1],@x[$b1])", + "&add_32 (@x[$a2],@x[$a2],@x[$b2])", + "&add_32 (@x[$a3],@x[$a3],@x[$b3])", + "&eor_32 (@x[$d0],@x[$d0],@x[$a0])", + "&eor_32 (@x[$d1],@x[$d1],@x[$a1])", + "&eor_32 (@x[$d2],@x[$d2],@x[$a2])", + "&eor_32 (@x[$d3],@x[$d3],@x[$a3])", + "&ror_32 (@x[$d0],@x[$d0],24)", + "&ror_32 (@x[$d1],@x[$d1],24)", + "&ror_32 (@x[$d2],@x[$d2],24)", + "&ror_32 (@x[$d3],@x[$d3],24)", + + "&add_32 (@x[$c0],@x[$c0],@x[$d0])", + "&add_32 (@x[$c1],@x[$c1],@x[$d1])", + "&add_32 (@x[$c2],@x[$c2],@x[$d2])", + "&add_32 (@x[$c3],@x[$c3],@x[$d3])", + "&eor_32 (@x[$b0],@x[$b0],@x[$c0])", + "&eor_32 (@x[$b1],@x[$b1],@x[$c1])", + "&eor_32 (@x[$b2],@x[$b2],@x[$c2])", + "&eor_32 (@x[$b3],@x[$b3],@x[$c3])", + "&ror_32 (@x[$b0],@x[$b0],25)", + "&ror_32 (@x[$b1],@x[$b1],25)", + "&ror_32 (@x[$b2],@x[$b2],25)", + "&ror_32 (@x[$b3],@x[$b3],25)" + ); +} + +$code.=<<___; +#ifndef __KERNEL__ +# include "arm_arch.h" +.extern OPENSSL_armcap_P +#endif + +.text + +.align 5 +.Lsigma: +.quad 0x3320646e61707865,0x6b20657479622d32 // endian-neutral +.Lone: +.long 1,2,3,4 +.Lrot24: +.long 0x02010003,0x06050407,0x0a09080b,0x0e0d0c0f +.asciz "ChaCha20 for ARMv8, CRYPTOGAMS by \@dot-asm" + +.globl ChaCha20_ctr32 +.type ChaCha20_ctr32,%function +.align 5 +ChaCha20_ctr32: + cbz $len,.Labort + cmp $len,#192 + b.lo .Lshort + +#ifndef __KERNEL__ + adrp c17,OPENSSL_armcap_P + ldr w17,[c17,#:lo12:OPENSSL_armcap_P] + tst w17,#ARMV7_NEON + b.ne .LChaCha20_neon +#endif + +.Lshort: + .inst 0xd503233f // paciasp + stp c29,c30,[sp,#-12*__SIZEOF_POINTER__]! + add c29,csp,#0 + + adr @x[0],.Lsigma + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + sub csp,csp,#64 + + ldp @d[0],@d[1],[@x[0]] // load sigma + ldp @d[2],@d[3],[$key] // load key + ldp @d[4],@d[5],[$key,#16] + ldp @d[6],@d[7],[$ctr] // load counter +#ifdef __AARCH64EB__ + ror @d[2],@d[2],#32 + ror @d[3],@d[3],#32 + ror @d[4],@d[4],#32 + ror @d[5],@d[5],#32 + ror @d[6],@d[6],#32 + ror @d[7],@d[7],#32 +#endif + +.Loop_outer: + mov.32 @x[0],@d[0] // unpack key block + lsr @x[1],@d[0],#32 + mov.32 @x[2],@d[1] + lsr @x[3],@d[1],#32 + mov.32 @x[4],@d[2] + lsr @x[5],@d[2],#32 + mov.32 @x[6],@d[3] + lsr @x[7],@d[3],#32 + mov.32 @x[8],@d[4] + lsr @x[9],@d[4],#32 + mov.32 @x[10],@d[5] + lsr @x[11],@d[5],#32 + mov.32 @x[12],@d[6] + lsr @x[13],@d[6],#32 + mov.32 @x[14],@d[7] + lsr @x[15],@d[7],#32 + + mov $ctr,#10 + subs $len,$len,#64 +.Loop: + sub $ctr,$ctr,#1 +___ + foreach (&ROUND(0, 4, 8,12)) { eval; } + foreach (&ROUND(0, 5,10,15)) { eval; } +$code.=<<___; + cbnz $ctr,.Loop + + add.32 @x[0],@x[0],@d[0] // accumulate key block + add @x[1],@x[1],@d[0],lsr#32 + add.32 @x[2],@x[2],@d[1] + add @x[3],@x[3],@d[1],lsr#32 + add.32 @x[4],@x[4],@d[2] + add @x[5],@x[5],@d[2],lsr#32 + add.32 @x[6],@x[6],@d[3] + add @x[7],@x[7],@d[3],lsr#32 + add.32 @x[8],@x[8],@d[4] + add @x[9],@x[9],@d[4],lsr#32 + add.32 @x[10],@x[10],@d[5] + add @x[11],@x[11],@d[5],lsr#32 + add.32 @x[12],@x[12],@d[6] + add @x[13],@x[13],@d[6],lsr#32 + add.32 @x[14],@x[14],@d[7] + add @x[15],@x[15],@d[7],lsr#32 + + b.lo .Ltail + + add @x[0],@x[0],@x[1],lsl#32 // pack + add @x[2],@x[2],@x[3],lsl#32 + ldp @x[1],@x[3],[$inp,#0] // load input + add @x[4],@x[4],@x[5],lsl#32 + add @x[6],@x[6],@x[7],lsl#32 + ldp @x[5],@x[7],[$inp,#16] + add @x[8],@x[8],@x[9],lsl#32 + add @x[10],@x[10],@x[11],lsl#32 + ldp @x[9],@x[11],[$inp,#32] + add @x[12],@x[12],@x[13],lsl#32 + add @x[14],@x[14],@x[15],lsl#32 + ldp @x[13],@x[15],[$inp,#48] + cadd $inp,$inp,#64 +#ifdef __AARCH64EB__ + rev @x[0],@x[0] + rev @x[2],@x[2] + rev @x[4],@x[4] + rev @x[6],@x[6] + rev @x[8],@x[8] + rev @x[10],@x[10] + rev @x[12],@x[12] + rev @x[14],@x[14] +#endif + eor @x[0],@x[0],@x[1] + eor @x[2],@x[2],@x[3] + eor @x[4],@x[4],@x[5] + eor @x[6],@x[6],@x[7] + eor @x[8],@x[8],@x[9] + eor @x[10],@x[10],@x[11] + eor @x[12],@x[12],@x[13] + eor @x[14],@x[14],@x[15] + + stp @x[0],@x[2],[$out,#0] // store output + add @d[6],@d[6],#1 // increment counter + stp @x[4],@x[6],[$out,#16] + stp @x[8],@x[10],[$out,#32] + stp @x[12],@x[14],[$out,#48] + cadd $out,$out,#64 + + b.hi .Loop_outer + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#64 + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#12*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp +.Labort: + ret + +.align 4 +.Ltail: + add $len,$len,#64 +.Less_than_64: + csub $out,$out,#1 + cadd $inp,$inp,$len + cadd $out,$out,$len + cadd $ctr,sp,$len + neg $len,$len + + add @x[0],@x[0],@x[1],lsl#32 // pack + add @x[2],@x[2],@x[3],lsl#32 + add @x[4],@x[4],@x[5],lsl#32 + add @x[6],@x[6],@x[7],lsl#32 + add @x[8],@x[8],@x[9],lsl#32 + add @x[10],@x[10],@x[11],lsl#32 + add @x[12],@x[12],@x[13],lsl#32 + add @x[14],@x[14],@x[15],lsl#32 +#ifdef __AARCH64EB__ + rev @x[0],@x[0] + rev @x[2],@x[2] + rev @x[4],@x[4] + rev @x[6],@x[6] + rev @x[8],@x[8] + rev @x[10],@x[10] + rev @x[12],@x[12] + rev @x[14],@x[14] +#endif + stp @x[0],@x[2],[sp,#0] // off-load complete block + stp @x[4],@x[6],[sp,#16] + stp @x[8],@x[10],[sp,#32] + stp @x[12],@x[14],[sp,#48] + +.Loop_tail: + ldrb w10,[$inp,$len] + ldrb w11,[$ctr,$len] + add $len,$len,#1 + eor w10,w10,w11 + strb w10,[$out,$len] + cbnz $len,.Loop_tail + + stp xzr,xzr,[sp,#0] // wipe off-load area + stp xzr,xzr,[sp,#16] + stp xzr,xzr,[sp,#32] + stp xzr,xzr,[sp,#48] + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#64 + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#12*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size ChaCha20_ctr32,.-ChaCha20_ctr32 +___ + +{{{ +######################################################################## +# 4x"vertical" layout reduces *total* amount of instructions by trading +# 60 "horizontal" permutations in inner loop for 32-instruction diagonal +# transposition at the loop exit. And since NEON instruction issue rate +# is customarily limited, it's possible to process one additional block +# with scalar instructions at no additional cost. Hence the "4+1" +# description... + +my @K = map("v$_.4s",(0..3)); +my ($xt0,$xt1,$xt2,$xt3, $CTR,$ROT24) = map("v$_.4s",(4..9)); +my @X = map("v$_.4s",(16,20,24,28, 17,21,25,29, 18,22,26,30, 19,23,27,31)); +my ($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3) = @X; + +sub NEON_lane_ROUND { +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); +my @x=map("'$_'",@X); + + ( + "&add (@x[$a0],@x[$a0],@x[$b0])", # Q1 + "&add (@x[$a1],@x[$a1],@x[$b1])", # Q2 + "&add (@x[$a2],@x[$a2],@x[$b2])", # Q3 + "&add (@x[$a3],@x[$a3],@x[$b3])", # Q4 + "&eor (@x[$d0],@x[$d0],@x[$a0])", + "&eor (@x[$d1],@x[$d1],@x[$a1])", + "&eor (@x[$d2],@x[$d2],@x[$a2])", + "&eor (@x[$d3],@x[$d3],@x[$a3])", + "&rev32_16 (@x[$d0],@x[$d0])", + "&rev32_16 (@x[$d1],@x[$d1])", + "&rev32_16 (@x[$d2],@x[$d2])", + "&rev32_16 (@x[$d3],@x[$d3])", + + "&add (@x[$c0],@x[$c0],@x[$d0])", + "&add (@x[$c1],@x[$c1],@x[$d1])", + "&add (@x[$c2],@x[$c2],@x[$d2])", + "&add (@x[$c3],@x[$c3],@x[$d3])", + "&eor ('$xt0',@x[$b0],@x[$c0])", + "&eor ('$xt1',@x[$b1],@x[$c1])", + "&eor ('$xt2',@x[$b2],@x[$c2])", + "&eor ('$xt3',@x[$b3],@x[$c3])", + "&ushr (@x[$b0],'$xt0',20)", + "&ushr (@x[$b1],'$xt1',20)", + "&ushr (@x[$b2],'$xt2',20)", + "&ushr (@x[$b3],'$xt3',20)", + "&sli (@x[$b0],'$xt0',12)", + "&sli (@x[$b1],'$xt1',12)", + "&sli (@x[$b2],'$xt2',12)", + "&sli (@x[$b3],'$xt3',12)", + + "&add (@x[$a0],@x[$a0],@x[$b0])", + "&add (@x[$a1],@x[$a1],@x[$b1])", + "&add (@x[$a2],@x[$a2],@x[$b2])", + "&add (@x[$a3],@x[$a3],@x[$b3])", + "&eor ('$xt0',@x[$d0],@x[$a0])", + "&eor ('$xt1',@x[$d1],@x[$a1])", + "&eor ('$xt2',@x[$d2],@x[$a2])", + "&eor ('$xt3',@x[$d3],@x[$a3])", + "&tbl (@x[$d0],'{$xt0}','$ROT24')", + "&tbl (@x[$d1],'{$xt1}','$ROT24')", + "&tbl (@x[$d2],'{$xt2}','$ROT24')", + "&tbl (@x[$d3],'{$xt3}','$ROT24')", + + "&add (@x[$c0],@x[$c0],@x[$d0])", + "&add (@x[$c1],@x[$c1],@x[$d1])", + "&add (@x[$c2],@x[$c2],@x[$d2])", + "&add (@x[$c3],@x[$c3],@x[$d3])", + "&eor ('$xt0',@x[$b0],@x[$c0])", + "&eor ('$xt1',@x[$b1],@x[$c1])", + "&eor ('$xt2',@x[$b2],@x[$c2])", + "&eor ('$xt3',@x[$b3],@x[$c3])", + "&ushr (@x[$b0],'$xt0',25)", + "&ushr (@x[$b1],'$xt1',25)", + "&ushr (@x[$b2],'$xt2',25)", + "&ushr (@x[$b3],'$xt3',25)", + "&sli (@x[$b0],'$xt0',7)", + "&sli (@x[$b1],'$xt1',7)", + "&sli (@x[$b2],'$xt2',7)", + "&sli (@x[$b3],'$xt3',7)" + ); +} + +$code.=<<___; + +#ifdef __KERNEL__ +.globl ChaCha20_neon +#endif +.type ChaCha20_neon,%function +.align 5 +ChaCha20_neon: +.LChaCha20_neon: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-12*__SIZEOF_POINTER__]! + add c29,csp,#0 + + adr @x[0],.Lsigma + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + cmp $len,#512 + b.hs .L512_or_more_neon + + sub csp,csp,#64 + + ldp @d[0],@d[1],[@x[0]] // load sigma + ld1 {@K[0]},[@x[0]],#16 + ldp @d[2],@d[3],[$key] // load key + ldp @d[4],@d[5],[$key,#16] + ld1 {@K[1],@K[2]},[$key] + ldp @d[6],@d[7],[$ctr] // load counter + ld1 {@K[3]},[$ctr] + stp d8,d9,[sp] // meet ABI requirements + ld1 {$CTR,$ROT24},[@x[0]] +#ifdef __AARCH64EB__ + rev64 @K[0],@K[0] + ror @d[2],@d[2],#32 + ror @d[3],@d[3],#32 + ror @d[4],@d[4],#32 + ror @d[5],@d[5],#32 + ror @d[6],@d[6],#32 + ror @d[7],@d[7],#32 +#endif + +.Loop_outer_neon: + dup $xa0,@{K[0]}[0] // unpack key block + mov.32 @x[0],@d[0] + dup $xa1,@{K[0]}[1] + lsr @x[1],@d[0],#32 + dup $xa2,@{K[0]}[2] + mov.32 @x[2],@d[1] + dup $xa3,@{K[0]}[3] + lsr @x[3],@d[1],#32 + dup $xb0,@{K[1]}[0] + mov.32 @x[4],@d[2] + dup $xb1,@{K[1]}[1] + lsr @x[5],@d[2],#32 + dup $xb2,@{K[1]}[2] + mov.32 @x[6],@d[3] + dup $xb3,@{K[1]}[3] + lsr @x[7],@d[3],#32 + dup $xd0,@{K[3]}[0] + mov.32 @x[8],@d[4] + dup $xd1,@{K[3]}[1] + lsr @x[9],@d[4],#32 + dup $xd2,@{K[3]}[2] + mov.32 @x[10],@d[5] + dup $xd3,@{K[3]}[3] + lsr @x[11],@d[5],#32 + add $xd0,$xd0,$CTR + mov.32 @x[12],@d[6] + dup $xc0,@{K[2]}[0] + lsr @x[13],@d[6],#32 + dup $xc1,@{K[2]}[1] + mov.32 @x[14],@d[7] + dup $xc2,@{K[2]}[2] + lsr @x[15],@d[7],#32 + dup $xc3,@{K[2]}[3] + + mov $ctr,#10 + subs $len,$len,#320 +.Loop_neon: + sub $ctr,$ctr,#1 +___ + my @plus_one=&ROUND(0,4,8,12); my $i=0; + foreach (&NEON_lane_ROUND(0,4,8,12)) { eval; eval(shift(@plus_one)) if ($i++ > 6); } + foreach (@plus_one) { eval; } + + @plus_one=&ROUND(0,5,10,15); $i=0; + foreach (&NEON_lane_ROUND(0,5,10,15)) { eval; eval(shift(@plus_one)) if ($i++ > 6); } + foreach (@plus_one) { eval; } +$code.=<<___; + cbnz $ctr,.Loop_neon + + add $xd0,$xd0,$CTR + + zip1 $xt0,$xa0,$xa1 // transpose data + zip1 $xt1,$xa2,$xa3 + zip2 $xt2,$xa0,$xa1 + zip2 $xt3,$xa2,$xa3 + zip1.64 $xa0,$xt0,$xt1 + zip2.64 $xa1,$xt0,$xt1 + zip1.64 $xa2,$xt2,$xt3 + zip2.64 $xa3,$xt2,$xt3 + + zip1 $xt0,$xb0,$xb1 + zip1 $xt1,$xb2,$xb3 + zip2 $xt2,$xb0,$xb1 + zip2 $xt3,$xb2,$xb3 + zip1.64 $xb0,$xt0,$xt1 + zip2.64 $xb1,$xt0,$xt1 + zip1.64 $xb2,$xt2,$xt3 + zip2.64 $xb3,$xt2,$xt3 + + zip1 $xt0,$xc0,$xc1 + add.32 @x[0],@x[0],@d[0] // accumulate key block + zip1 $xt1,$xc2,$xc3 + add @x[1],@x[1],@d[0],lsr#32 + zip2 $xt2,$xc0,$xc1 + add.32 @x[2],@x[2],@d[1] + zip2 $xt3,$xc2,$xc3 + add @x[3],@x[3],@d[1],lsr#32 + zip1.64 $xc0,$xt0,$xt1 + add.32 @x[4],@x[4],@d[2] + zip2.64 $xc1,$xt0,$xt1 + add @x[5],@x[5],@d[2],lsr#32 + zip1.64 $xc2,$xt2,$xt3 + add.32 @x[6],@x[6],@d[3] + zip2.64 $xc3,$xt2,$xt3 + add @x[7],@x[7],@d[3],lsr#32 + + zip1 $xt0,$xd0,$xd1 + add.32 @x[8],@x[8],@d[4] + zip1 $xt1,$xd2,$xd3 + add @x[9],@x[9],@d[4],lsr#32 + zip2 $xt2,$xd0,$xd1 + add.32 @x[10],@x[10],@d[5] + zip2 $xt3,$xd2,$xd3 + add @x[11],@x[11],@d[5],lsr#32 + zip1.64 $xd0,$xt0,$xt1 + add.32 @x[12],@x[12],@d[6] + zip2.64 $xd1,$xt0,$xt1 + add @x[13],@x[13],@d[6],lsr#32 + zip1.64 $xd2,$xt2,$xt3 + add.32 @x[14],@x[14],@d[7] + zip2.64 $xd3,$xt2,$xt3 + add @x[15],@x[15],@d[7],lsr#32 + + b.lo .Ltail_neon + + add @x[0],@x[0],@x[1],lsl#32 // pack + add @x[2],@x[2],@x[3],lsl#32 + ldp @x[1],@x[3],[$inp,#0] // load input + add $xa0,$xa0,@K[0] // accumulate key block + add @x[4],@x[4],@x[5],lsl#32 + add @x[6],@x[6],@x[7],lsl#32 + ldp @x[5],@x[7],[$inp,#16] + add $xb0,$xb0,@K[1] + add @x[8],@x[8],@x[9],lsl#32 + add @x[10],@x[10],@x[11],lsl#32 + ldp @x[9],@x[11],[$inp,#32] + add $xc0,$xc0,@K[2] + add @x[12],@x[12],@x[13],lsl#32 + add @x[14],@x[14],@x[15],lsl#32 + ldp @x[13],@x[15],[$inp,#48] + add $xd0,$xd0,@K[3] + cadd $inp,$inp,#64 +#ifdef __AARCH64EB__ + rev @x[0],@x[0] + rev @x[2],@x[2] + rev @x[4],@x[4] + rev @x[6],@x[6] + rev @x[8],@x[8] + rev @x[10],@x[10] + rev @x[12],@x[12] + rev @x[14],@x[14] +#endif + ld1.8 {$xt0-$xt3},[$inp],#64 + eor @x[0],@x[0],@x[1] + add $xa1,$xa1,@K[0] + eor @x[2],@x[2],@x[3] + add $xb1,$xb1,@K[1] + eor @x[4],@x[4],@x[5] + add $xc1,$xc1,@K[2] + eor @x[6],@x[6],@x[7] + add $xd1,$xd1,@K[3] + eor @x[8],@x[8],@x[9] + eor $xa0,$xa0,$xt0 + movi $xt0,#5 + eor @x[10],@x[10],@x[11] + eor $xb0,$xb0,$xt1 + eor @x[12],@x[12],@x[13] + eor $xc0,$xc0,$xt2 + eor @x[14],@x[14],@x[15] + eor $xd0,$xd0,$xt3 + add $CTR,$CTR,$xt0 // += 5 + ld1.8 {$xt0-$xt3},[$inp],#64 + + stp @x[0],@x[2],[$out,#0] // store output + add @d[6],@d[6],#5 // increment counter + stp @x[4],@x[6],[$out,#16] + stp @x[8],@x[10],[$out,#32] + stp @x[12],@x[14],[$out,#48] + cadd $out,$out,#64 + + st1.8 {$xa0-$xd0},[$out],#64 + add $xa2,$xa2,@K[0] + add $xb2,$xb2,@K[1] + add $xc2,$xc2,@K[2] + add $xd2,$xd2,@K[3] + ld1.8 {$xa0-$xd0},[$inp],#64 + + eor $xa1,$xa1,$xt0 + eor $xb1,$xb1,$xt1 + eor $xc1,$xc1,$xt2 + eor $xd1,$xd1,$xt3 + st1.8 {$xa1-$xd1},[$out],#64 + add $xa3,$xa3,@K[0] + add $xb3,$xb3,@K[1] + add $xc3,$xc3,@K[2] + add $xd3,$xd3,@K[3] + ld1.8 {$xa1-$xd1},[$inp],#64 + + eor $xa2,$xa2,$xa0 + eor $xb2,$xb2,$xb0 + eor $xc2,$xc2,$xc0 + eor $xd2,$xd2,$xd0 + st1.8 {$xa2-$xd2},[$out],#64 + + eor $xa3,$xa3,$xa1 + eor $xb3,$xb3,$xb1 + eor $xc3,$xc3,$xc1 + eor $xd3,$xd3,$xd1 + st1.8 {$xa3-$xd3},[$out],#64 + + b.hi .Loop_outer_neon + + ldp d8,d9,[sp] // meet ABI requirements + eor @K[1],@K[1],@K[1] // cleanse key and nonce + eor @K[2],@K[2],@K[2] + eor @K[3],@K[3],@K[3] + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#64 + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#12*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret + +.align 4 +.Ltail_neon: + add $len,$len,#320 + ldp d8,d9,[sp] // meet ABI requirements + cmp $len,#64 + b.lo .Less_than_64_neon + + add @x[0],@x[0],@x[1],lsl#32 // pack + add @x[2],@x[2],@x[3],lsl#32 + ldp @x[1],@x[3],[$inp,#0] // load input + add @x[4],@x[4],@x[5],lsl#32 + add @x[6],@x[6],@x[7],lsl#32 + ldp @x[5],@x[7],[$inp,#16] + add @x[8],@x[8],@x[9],lsl#32 + add @x[10],@x[10],@x[11],lsl#32 + ldp @x[9],@x[11],[$inp,#32] + add @x[12],@x[12],@x[13],lsl#32 + add @x[14],@x[14],@x[15],lsl#32 + ldp @x[13],@x[15],[$inp,#48] + cadd $inp,$inp,#64 +#ifdef __AARCH64EB__ + rev @x[0],@x[0] + rev @x[2],@x[2] + rev @x[4],@x[4] + rev @x[6],@x[6] + rev @x[8],@x[8] + rev @x[10],@x[10] + rev @x[12],@x[12] + rev @x[14],@x[14] +#endif + eor @x[0],@x[0],@x[1] + eor @x[2],@x[2],@x[3] + eor @x[4],@x[4],@x[5] + eor @x[6],@x[6],@x[7] + eor @x[8],@x[8],@x[9] + eor @x[10],@x[10],@x[11] + eor @x[12],@x[12],@x[13] + eor @x[14],@x[14],@x[15] + + stp @x[0],@x[2],[$out,#0] // store output + add $xa0,$xa0,@K[0] // accumulate key block + stp @x[4],@x[6],[$out,#16] + add $xb0,$xb0,@K[1] + stp @x[8],@x[10],[$out,#32] + add $xc0,$xc0,@K[2] + stp @x[12],@x[14],[$out,#48] + add $xd0,$xd0,@K[3] + cadd $out,$out,#64 + b.eq .Ldone_neon + sub $len,$len,#64 + cmp $len,#64 + b.lo .Last_neon + + ld1.8 {$xt0-$xt3},[$inp],#64 + eor $xa0,$xa0,$xt0 + eor $xb0,$xb0,$xt1 + eor $xc0,$xc0,$xt2 + eor $xd0,$xd0,$xt3 + st1.8 {$xa0-$xd0},[$out],#64 + b.eq .Ldone_neon + + add $xa0,$xa1,@K[0] + add $xb0,$xb1,@K[1] + sub $len,$len,#64 + add $xc0,$xc1,@K[2] + cmp $len,#64 + add $xd0,$xd1,@K[3] + b.lo .Last_neon + + ld1.8 {$xt0-$xt3},[$inp],#64 + eor $xa1,$xa0,$xt0 + eor $xb1,$xb0,$xt1 + eor $xc1,$xc0,$xt2 + eor $xd1,$xd0,$xt3 + st1.8 {$xa1-$xd1},[$out],#64 + b.eq .Ldone_neon + + add $xa0,$xa2,@K[0] + add $xb0,$xb2,@K[1] + sub $len,$len,#64 + add $xc0,$xc2,@K[2] + cmp $len,#64 + add $xd0,$xd2,@K[3] + b.lo .Last_neon + + ld1.8 {$xt0-$xt3},[$inp],#64 + eor $xa2,$xa0,$xt0 + eor $xb2,$xb0,$xt1 + eor $xc2,$xc0,$xt2 + eor $xd2,$xd0,$xt3 + st1.8 {$xa2-$xd2},[$out],#64 + b.eq .Ldone_neon + + add $xa0,$xa3,@K[0] + add $xb0,$xb3,@K[1] + add $xc0,$xc3,@K[2] + add $xd0,$xd3,@K[3] + sub $len,$len,#64 + +.Last_neon: + st1.8 {$xa0-$xd0},[sp] // off-load complete block + + csub $out,$out,#1 + cadd $inp,$inp,$len + cadd $out,$out,$len + cadd $ctr,sp,$len + neg $len,$len + +.Loop_tail_neon: + ldrb w10,[$inp,$len] + ldrb w11,[$ctr,$len] + add $len,$len,#1 + eor w10,w10,w11 + strb w10,[$out,$len] + cbnz $len,.Loop_tail_neon + + stp @K[0],@K[0],[sp,#0] // wipe off-load area + stp @K[0],@K[0],[sp,#32] // [with known constant] + +.Ldone_neon: + eor @K[1],@K[1],@K[1] // cleanse key and nonce + eor @K[2],@K[2],@K[2] + eor @K[3],@K[3],@K[3] + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#64 + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#12*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret + +.align 4 +.Less_than_64_neon: + eor @K[1],@K[1],@K[1] // cleanse key and nonce + eor @K[2],@K[2],@K[2] + eor @K[3],@K[3],@K[3] + b .Less_than_64 +.size ChaCha20_neon,.-ChaCha20_neon +___ +{ +######################################################################## +# While "vertical" layout minimizes total amount of instructions, number +# of blocks processed in parallel is limited to 4x. And trouble is that +# if NEON instructions are high-latency enough, algorithmic dependencies +# will manifest themselves as idle/wasted cycles. 6x"horizontal" avoids +# these gaps and achieves better performance. Since NEON instruction +# sequence is >2x longer, it's possible to slip in two additional blocks +# processed with scalar code path at no additional cost. Hence the "6+2" +# description... + +my @K = map("v$_.4s",(0..6)); +my ($T0,$T1,$T2,$T3,$T4,$T5)=@K; +my ($A0,$B0,$C0,$D0,$A1,$B1,$C1,$D1,$A2,$B2,$C2,$D2, + $A3,$B3,$C3,$D3,$A4,$B4,$C4,$D4,$A5,$B5,$C5,$D5) = map("v$_.4s",(8..31)); +my $rot24 = @K[6]; +my $ONE = "v7.4s"; + +sub NEONROUND { +my $odd = pop; +my ($a,$b,$c,$d,$t)=@_; + + ( + "&add ('$a','$a','$b')", + "&eor ('$d','$d','$a')", + "&rev32_16 ('$d','$d')", # vrot ($d,16) + + "&add ('$c','$c','$d')", + "&eor ('$t','$b','$c')", + "&ushr ('$b','$t',20)", + "&sli ('$b','$t',12)", + + "&add ('$a','$a','$b')", + "&eor ('$d','$d','$a')", + "&tbl ('$d','{$d}','$rot24')", + + "&add ('$c','$c','$d')", + "&eor ('$t','$b','$c')", + "&ushr ('$b','$t',25)", + "&sli ('$b','$t',7)", + + "&ext ('$c','$c','$c',8)", + "&ext ('$d','$d','$d',$odd?4:12)", + "&ext ('$b','$b','$b',$odd?12:4)" + ); +} + +$code.=<<___; +.type ChaCha20_512_neon,%function +.align 5 +ChaCha20_512_neon: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-12*__SIZEOF_POINTER__]! + add c29,csp,#0 + + adr @x[0],.Lsigma + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + +.L512_or_more_neon: + sub csp,csp,#128+64 + + eor $ONE,$ONE,$ONE + ldp @d[0],@d[1],[@x[0]] // load sigma + ld1 {@K[0]},[@x[0]],#16 + ldp @d[2],@d[3],[$key] // load key + ldp @d[4],@d[5],[$key,#16] + ld1 {@K[1],@K[2]},[$key] + ldp @d[6],@d[7],[$ctr] // load counter + ld1 {@K[3]},[$ctr] + ld1 {$ONE}[0],[@x[0]] + cadd $key,@x[0],#16 // .Lrot24 +#ifdef __AARCH64EB__ + rev64 @K[0],@K[0] + ror @d[2],@d[2],#32 + ror @d[3],@d[3],#32 + ror @d[4],@d[4],#32 + ror @d[5],@d[5],#32 + ror @d[6],@d[6],#32 + ror @d[7],@d[7],#32 +#endif + add @K[3],@K[3],$ONE // += 1 + stp @K[0],@K[1],[sp,#0] // off-load key block, invariant part + add @K[3],@K[3],$ONE // not typo + str @K[2],[sp,#32] + add @K[4],@K[3],$ONE + add @K[5],@K[4],$ONE + add @K[6],@K[5],$ONE + shl $ONE,$ONE,#2 // 1 -> 4 + + stp d8,d9,[sp,#128+0] // meet ABI requirements + stp d10,d11,[sp,#128+16] + stp d12,d13,[sp,#128+32] + stp d14,d15,[sp,#128+48] + + sub $len,$len,#512 // not typo + +.Loop_outer_512_neon: + mov $A0,@K[0] + mov $A1,@K[0] + mov $A2,@K[0] + mov $A3,@K[0] + mov $A4,@K[0] + mov $A5,@K[0] + mov $B0,@K[1] + mov.32 @x[0],@d[0] // unpack key block + mov $B1,@K[1] + lsr @x[1],@d[0],#32 + mov $B2,@K[1] + mov.32 @x[2],@d[1] + mov $B3,@K[1] + lsr @x[3],@d[1],#32 + mov $B4,@K[1] + mov.32 @x[4],@d[2] + mov $B5,@K[1] + lsr @x[5],@d[2],#32 + mov $D0,@K[3] + mov.32 @x[6],@d[3] + mov $D1,@K[4] + lsr @x[7],@d[3],#32 + mov $D2,@K[5] + mov.32 @x[8],@d[4] + mov $D3,@K[6] + lsr @x[9],@d[4],#32 + mov $C0,@K[2] + mov.32 @x[10],@d[5] + mov $C1,@K[2] + lsr @x[11],@d[5],#32 + add $D4,$D0,$ONE // +4 + mov.32 @x[12],@d[6] + add $D5,$D1,$ONE // +4 + lsr @x[13],@d[6],#32 + mov $C2,@K[2] + mov.32 @x[14],@d[7] + mov $C3,@K[2] + lsr @x[15],@d[7],#32 + mov $C4,@K[2] + stp @K[3],@K[4],[sp,#48] // off-load key block, variable part + mov $C5,@K[2] + stp @K[5],@K[6],[sp,#80] + + mov $ctr,#5 + ld1 {$rot24},[$key] + subs $len,$len,#512 +.Loop_upper_neon: + sub $ctr,$ctr,#1 +___ + my @thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,0); + my @thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,0); + my @thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,0); + my @thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,0); + my @thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,0); + my @thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,0); + my @thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15)); + my $diff = ($#thread0+1)*6 - $#thread67 - 1; + my $i = 0; + + foreach (@thread0) { + eval; eval(shift(@thread67)); + eval(shift(@thread1)); eval(shift(@thread67)); + eval(shift(@thread2)); eval(shift(@thread67)); + eval(shift(@thread3)); eval(shift(@thread67)); + eval(shift(@thread4)); eval(shift(@thread67)); + eval(shift(@thread5)); eval(shift(@thread67)); + } + + @thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,1); + @thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,1); + @thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,1); + @thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,1); + @thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,1); + @thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,1); + @thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15)); + + foreach (@thread0) { + eval; eval(shift(@thread67)); + eval(shift(@thread1)); eval(shift(@thread67)); + eval(shift(@thread2)); eval(shift(@thread67)); + eval(shift(@thread3)); eval(shift(@thread67)); + eval(shift(@thread4)); eval(shift(@thread67)); + eval(shift(@thread5)); eval(shift(@thread67)); + } +$code.=<<___; + cbnz $ctr,.Loop_upper_neon + + add.32 @x[0],@x[0],@d[0] // accumulate key block + add @x[1],@x[1],@d[0],lsr#32 + add.32 @x[2],@x[2],@d[1] + add @x[3],@x[3],@d[1],lsr#32 + add.32 @x[4],@x[4],@d[2] + add @x[5],@x[5],@d[2],lsr#32 + add.32 @x[6],@x[6],@d[3] + add @x[7],@x[7],@d[3],lsr#32 + add.32 @x[8],@x[8],@d[4] + add @x[9],@x[9],@d[4],lsr#32 + add.32 @x[10],@x[10],@d[5] + add @x[11],@x[11],@d[5],lsr#32 + add.32 @x[12],@x[12],@d[6] + add @x[13],@x[13],@d[6],lsr#32 + add.32 @x[14],@x[14],@d[7] + add @x[15],@x[15],@d[7],lsr#32 + + add @x[0],@x[0],@x[1],lsl#32 // pack + add @x[2],@x[2],@x[3],lsl#32 + ldp @x[1],@x[3],[$inp,#0] // load input + add @x[4],@x[4],@x[5],lsl#32 + add @x[6],@x[6],@x[7],lsl#32 + ldp @x[5],@x[7],[$inp,#16] + add @x[8],@x[8],@x[9],lsl#32 + add @x[10],@x[10],@x[11],lsl#32 + ldp @x[9],@x[11],[$inp,#32] + add @x[12],@x[12],@x[13],lsl#32 + add @x[14],@x[14],@x[15],lsl#32 + ldp @x[13],@x[15],[$inp,#48] + cadd $inp,$inp,#64 +#ifdef __AARCH64EB__ + rev @x[0],@x[0] + rev @x[2],@x[2] + rev @x[4],@x[4] + rev @x[6],@x[6] + rev @x[8],@x[8] + rev @x[10],@x[10] + rev @x[12],@x[12] + rev @x[14],@x[14] +#endif + eor @x[0],@x[0],@x[1] + eor @x[2],@x[2],@x[3] + eor @x[4],@x[4],@x[5] + eor @x[6],@x[6],@x[7] + eor @x[8],@x[8],@x[9] + eor @x[10],@x[10],@x[11] + eor @x[12],@x[12],@x[13] + eor @x[14],@x[14],@x[15] + + stp @x[0],@x[2],[$out,#0] // store output + add @d[6],@d[6],#1 // increment counter + mov.32 @x[0],@d[0] // unpack key block + lsr @x[1],@d[0],#32 + stp @x[4],@x[6],[$out,#16] + mov.32 @x[2],@d[1] + lsr @x[3],@d[1],#32 + stp @x[8],@x[10],[$out,#32] + mov.32 @x[4],@d[2] + lsr @x[5],@d[2],#32 + stp @x[12],@x[14],[$out,#48] + cadd $out,$out,#64 + mov.32 @x[6],@d[3] + lsr @x[7],@d[3],#32 + mov.32 @x[8],@d[4] + lsr @x[9],@d[4],#32 + mov.32 @x[10],@d[5] + lsr @x[11],@d[5],#32 + mov.32 @x[12],@d[6] + lsr @x[13],@d[6],#32 + mov.32 @x[14],@d[7] + lsr @x[15],@d[7],#32 + + mov $ctr,#5 +.Loop_lower_neon: + sub $ctr,$ctr,#1 +___ + @thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,0); + @thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,0); + @thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,0); + @thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,0); + @thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,0); + @thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,0); + @thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15)); + + foreach (@thread0) { + eval; eval(shift(@thread67)); + eval(shift(@thread1)); eval(shift(@thread67)); + eval(shift(@thread2)); eval(shift(@thread67)); + eval(shift(@thread3)); eval(shift(@thread67)); + eval(shift(@thread4)); eval(shift(@thread67)); + eval(shift(@thread5)); eval(shift(@thread67)); + } + + @thread0=&NEONROUND($A0,$B0,$C0,$D0,$T0,1); + @thread1=&NEONROUND($A1,$B1,$C1,$D1,$T1,1); + @thread2=&NEONROUND($A2,$B2,$C2,$D2,$T2,1); + @thread3=&NEONROUND($A3,$B3,$C3,$D3,$T3,1); + @thread4=&NEONROUND($A4,$B4,$C4,$D4,$T4,1); + @thread5=&NEONROUND($A5,$B5,$C5,$D5,$T5,1); + @thread67=(&ROUND(0,4,8,12),&ROUND(0,5,10,15)); + + foreach (@thread0) { + eval; eval(shift(@thread67)); + eval(shift(@thread1)); eval(shift(@thread67)); + eval(shift(@thread2)); eval(shift(@thread67)); + eval(shift(@thread3)); eval(shift(@thread67)); + eval(shift(@thread4)); eval(shift(@thread67)); + eval(shift(@thread5)); eval(shift(@thread67)); + } +$code.=<<___; + cbnz $ctr,.Loop_lower_neon + + add.32 @x[0],@x[0],@d[0] // accumulate key block + ldp @K[0],@K[1],[sp,#0] + add @x[1],@x[1],@d[0],lsr#32 + ldp @K[2],@K[3],[sp,#32] + add.32 @x[2],@x[2],@d[1] + ldp @K[4],@K[5],[sp,#64] + add @x[3],@x[3],@d[1],lsr#32 + ldr @K[6],[sp,#96] + add $A0,$A0,@K[0] + add.32 @x[4],@x[4],@d[2] + add $A1,$A1,@K[0] + add @x[5],@x[5],@d[2],lsr#32 + add $A2,$A2,@K[0] + add.32 @x[6],@x[6],@d[3] + add $A3,$A3,@K[0] + add @x[7],@x[7],@d[3],lsr#32 + add $A4,$A4,@K[0] + add.32 @x[8],@x[8],@d[4] + add $A5,$A5,@K[0] + add @x[9],@x[9],@d[4],lsr#32 + add $C0,$C0,@K[2] + add.32 @x[10],@x[10],@d[5] + add $C1,$C1,@K[2] + add @x[11],@x[11],@d[5],lsr#32 + add $C2,$C2,@K[2] + add.32 @x[12],@x[12],@d[6] + add $C3,$C3,@K[2] + add @x[13],@x[13],@d[6],lsr#32 + add $C4,$C4,@K[2] + add.32 @x[14],@x[14],@d[7] + add $C5,$C5,@K[2] + add @x[15],@x[15],@d[7],lsr#32 + add $D4,$D4,$ONE // +4 + add @x[0],@x[0],@x[1],lsl#32 // pack + add $D5,$D5,$ONE // +4 + add @x[2],@x[2],@x[3],lsl#32 + add $D0,$D0,@K[3] + ldp @x[1],@x[3],[$inp,#0] // load input + add $D1,$D1,@K[4] + add @x[4],@x[4],@x[5],lsl#32 + add $D2,$D2,@K[5] + add @x[6],@x[6],@x[7],lsl#32 + add $D3,$D3,@K[6] + ldp @x[5],@x[7],[$inp,#16] + add $D4,$D4,@K[3] + add @x[8],@x[8],@x[9],lsl#32 + add $D5,$D5,@K[4] + add @x[10],@x[10],@x[11],lsl#32 + add $B0,$B0,@K[1] + ldp @x[9],@x[11],[$inp,#32] + add $B1,$B1,@K[1] + add @x[12],@x[12],@x[13],lsl#32 + add $B2,$B2,@K[1] + add @x[14],@x[14],@x[15],lsl#32 + add $B3,$B3,@K[1] + ldp @x[13],@x[15],[$inp,#48] + add $B4,$B4,@K[1] + cadd $inp,$inp,#64 + add $B5,$B5,@K[1] + +#ifdef __AARCH64EB__ + rev @x[0],@x[0] + rev @x[2],@x[2] + rev @x[4],@x[4] + rev @x[6],@x[6] + rev @x[8],@x[8] + rev @x[10],@x[10] + rev @x[12],@x[12] + rev @x[14],@x[14] +#endif + ld1.8 {$T0-$T3},[$inp],#64 + eor @x[0],@x[0],@x[1] + eor @x[2],@x[2],@x[3] + eor @x[4],@x[4],@x[5] + eor @x[6],@x[6],@x[7] + eor @x[8],@x[8],@x[9] + eor $A0,$A0,$T0 + eor @x[10],@x[10],@x[11] + eor $B0,$B0,$T1 + eor @x[12],@x[12],@x[13] + eor $C0,$C0,$T2 + eor @x[14],@x[14],@x[15] + eor $D0,$D0,$T3 + ld1.8 {$T0-$T3},[$inp],#64 + + stp @x[0],@x[2],[$out,#0] // store output + add @d[6],@d[6],#7 // increment counter + stp @x[4],@x[6],[$out,#16] + stp @x[8],@x[10],[$out,#32] + stp @x[12],@x[14],[$out,#48] + cadd $out,$out,#64 + st1.8 {$A0-$D0},[$out],#64 + + ld1.8 {$A0-$D0},[$inp],#64 + eor $A1,$A1,$T0 + eor $B1,$B1,$T1 + eor $C1,$C1,$T2 + eor $D1,$D1,$T3 + st1.8 {$A1-$D1},[$out],#64 + + ld1.8 {$A1-$D1},[$inp],#64 + eor $A2,$A2,$A0 + ldp @K[0],@K[1],[sp,#0] + eor $B2,$B2,$B0 + ldp @K[2],@K[3],[sp,#32] + eor $C2,$C2,$C0 + eor $D2,$D2,$D0 + st1.8 {$A2-$D2},[$out],#64 + + ld1.8 {$A2-$D2},[$inp],#64 + eor $A3,$A3,$A1 + eor $B3,$B3,$B1 + eor $C3,$C3,$C1 + eor $D3,$D3,$D1 + st1.8 {$A3-$D3},[$out],#64 + + ld1.8 {$A3-$D3},[$inp],#64 + eor $A4,$A4,$A2 + eor $B4,$B4,$B2 + eor $C4,$C4,$C2 + eor $D4,$D4,$D2 + st1.8 {$A4-$D4},[$out],#64 + + shl $A0,$ONE,#1 // 4 -> 8 + eor $A5,$A5,$A3 + eor $B5,$B5,$B3 + eor $C5,$C5,$C3 + eor $D5,$D5,$D3 + st1.8 {$A5-$D5},[$out],#64 + + add @K[3],@K[3],$A0 // += 8 + add @K[4],@K[4],$A0 + add @K[5],@K[5],$A0 + add @K[6],@K[6],$A0 + + b.hs .Loop_outer_512_neon + + adds $len,$len,#512 + ushr $ONE,$ONE,#1 // 4 -> 2 + + ldp d10,d11,[sp,#128+16] // meet ABI requirements + ldp d12,d13,[sp,#128+32] + ldp d14,d15,[sp,#128+48] + + stp @K[0],@K[0],[sp,#16] // wipe key off-load area + stp @K[0],@K[0],[sp,#48] // [with known constant] + stp @K[0],@K[0],[sp,#80] + + b.eq .Ldone_512_neon + + // we have <512 bytes tail, harmonize state with other contexts + csub $key,$key,#16 // .Lone + cmp $len,#192 + cadd sp,sp,#128 + sub @K[3],@K[3],$ONE // -= 2 + ld1 {$CTR,$ROT24},[$key] + b.hs .Loop_outer_neon + + ldp d8,d9,[sp,#0] // meet ABI requirements + eor @K[1],@K[1],@K[1] // cleanse key and nonce + eor @K[2],@K[2],@K[2] + eor @K[3],@K[3],@K[3] + eor @K[4],@K[4],@K[4] + eor @K[5],@K[5],@K[5] + eor @K[6],@K[6],@K[6] + b .Loop_outer + +.Ldone_512_neon: + ldp d8,d9,[sp,#128+0] // meet ABI requirements + eor @K[1],@K[1],@K[1] // cleanse key and nonce + eor @K[2],@K[2],@K[2] + eor @K[3],@K[3],@K[3] + eor @K[4],@K[4],@K[4] + eor @K[5],@K[5],@K[5] + eor @K[6],@K[6],@K[6] + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#128+64 + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#12*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size ChaCha20_512_neon,.-ChaCha20_512_neon +___ +} +}}} + +foreach (split("\n",$code)) { + s/\`([^\`]*)\`/eval $1/geo; + + (s/\b([a-z]+)\.32\b/$1/ and (s/x([0-9]+)/w$1/g or 1)) or + (m/\b(eor|ext|mov|tbl)\b/ and (s/\.4s/\.16b/g or 1)) or + (s/\b((?:ld|st)1)\.8\b/$1/ and (s/\.4s/\.16b/g or 1)) or + (m/\b(ld|st)[rp]\b/ and (s/v([0-9]+)\.4s/q$1/g or 1)) or + (m/\b(dup|ld1)\b/ and (s/\.4(s}?\[[0-3]\])/.$1/g or 1)) or + (s/\b(zip[12])\.64\b/$1/ and (s/\.4s/\.2d/g or 1)) or + (s/\brev32\.16\b/rev32/ and (s/\.4s/\.8h/g or 1)); + + #s/\bq([0-9]+)#(lo|hi)/sprintf "d%d",2*$1+($2 eq "hi")/geo; + + print $_,"\n"; +} +close STDOUT; # flush diff --git a/cbits/asm/chacha-x86_64-elf.S b/cbits/asm/chacha-x86_64-elf.S new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-x86_64-elf.S @@ -0,0 +1,2241 @@ +.text + + + +.align 64 +.Lzero: +.long 0,0,0,0 +.Lone: +.long 1,0,0,0 +.Linc: +.long 0,1,2,3 +.Lfour: +.long 4,4,4,4 +.Lincy: +.long 0,2,4,6,1,3,5,7 +.Leight: +.long 8,8,8,8,8,8,8,8 +.Lrot16: +.byte 0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd +.Lrot24: +.byte 0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe +.Ltwoy: +.long 2,0,0,0, 2,0,0,0 +.align 64 +.Lzeroz: +.long 0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0 +.Lfourz: +.long 4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0 +.Lincz: +.long 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 +.Lsixteen: +.long 16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16 +.Lsigma: +.byte 101,120,112,97,110,100,32,51,50,45,98,121,116,101,32,107,0 +.byte 67,104,97,67,104,97,50,48,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.globl crypton_chacha20_asm_ctr32 +.type crypton_chacha20_asm_ctr32,@function +.align 64 +crypton_chacha20_asm_ctr32: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + cmpq $0,%rdx + je .Lno_data + movq crypton_ia32cap_P+4(%rip),%r9 + testl $512,%r9d + jnz .Lcrypton_chacha20_asm_ssse3 + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + subq $64+24,%rsp +.cfi_adjust_cfa_offset 88 +.Lctr32_body: + + movq %rdx,%rbp + + movq 0(%rcx),%r12 + movq 8(%rcx),%r13 + movq 16(%rcx),%r14 + movq 24(%rcx),%r15 + movq 0(%r8),%rax + movq 8(%r8),%rdx + movq %r12,16(%rsp) + movq %r13,24(%rsp) + movq %r14,0(%rsp) + movq %r15,8(%rsp) + movq %rax,48(%rsp) + movq %rdx,56(%rsp) + jmp .Loop_outer + +.align 32 +.Loop_outer: + movl $0x61707865,%eax + movl $0x3320646e,%ebx + movl $0x79622d32,%ecx + movl $0x6b206574,%edx + movl 16(%rsp),%r8d + movl 20(%rsp),%r9d + movl 24(%rsp),%r10d + movl 28(%rsp),%r11d + movl 48(%rsp),%r12d + movl 52(%rsp),%r13d + movl 56(%rsp),%r14d + movq %r15,40(%rsp) + movl 60(%rsp),%r15d + + movq %rbp,64+0(%rsp) + movq %rsi,64+8(%rsp) + movl 0(%rsp),%esi + movq %rdi,64+16(%rsp) + movl 4(%rsp),%edi + movl $10,%ebp + jmp .Loop + +.align 32 +.Loop: + addl %r8d,%eax + xorl %eax,%r12d + roll $16,%r12d + addl %r9d,%ebx + xorl %ebx,%r13d + roll $16,%r13d + addl %r12d,%esi + xorl %esi,%r8d + roll $12,%r8d + addl %r13d,%edi + xorl %edi,%r9d + roll $12,%r9d + addl %r8d,%eax + xorl %eax,%r12d + roll $8,%r12d + addl %r9d,%ebx + xorl %ebx,%r13d + roll $8,%r13d + addl %r12d,%esi + xorl %esi,%r8d + roll $7,%r8d + addl %r13d,%edi + xorl %edi,%r9d + roll $7,%r9d + movl %esi,32(%rsp) + movl %edi,36(%rsp) + movl 40(%rsp),%esi + movl 44(%rsp),%edi + addl %r10d,%ecx + xorl %ecx,%r14d + roll $16,%r14d + addl %r11d,%edx + xorl %edx,%r15d + roll $16,%r15d + addl %r14d,%esi + xorl %esi,%r10d + roll $12,%r10d + addl %r15d,%edi + xorl %edi,%r11d + roll $12,%r11d + addl %r10d,%ecx + xorl %ecx,%r14d + roll $8,%r14d + addl %r11d,%edx + xorl %edx,%r15d + roll $8,%r15d + addl %r14d,%esi + xorl %esi,%r10d + roll $7,%r10d + addl %r15d,%edi + xorl %edi,%r11d + roll $7,%r11d + addl %r9d,%eax + xorl %eax,%r15d + roll $16,%r15d + addl %r10d,%ebx + xorl %ebx,%r12d + roll $16,%r12d + addl %r15d,%esi + xorl %esi,%r9d + roll $12,%r9d + addl %r12d,%edi + xorl %edi,%r10d + roll $12,%r10d + addl %r9d,%eax + xorl %eax,%r15d + roll $8,%r15d + addl %r10d,%ebx + xorl %ebx,%r12d + roll $8,%r12d + addl %r15d,%esi + xorl %esi,%r9d + roll $7,%r9d + addl %r12d,%edi + xorl %edi,%r10d + roll $7,%r10d + movl %esi,40(%rsp) + movl %edi,44(%rsp) + movl 32(%rsp),%esi + movl 36(%rsp),%edi + addl %r11d,%ecx + xorl %ecx,%r13d + roll $16,%r13d + addl %r8d,%edx + xorl %edx,%r14d + roll $16,%r14d + addl %r13d,%esi + xorl %esi,%r11d + roll $12,%r11d + addl %r14d,%edi + xorl %edi,%r8d + roll $12,%r8d + addl %r11d,%ecx + xorl %ecx,%r13d + roll $8,%r13d + addl %r8d,%edx + xorl %edx,%r14d + roll $8,%r14d + addl %r13d,%esi + xorl %esi,%r11d + roll $7,%r11d + addl %r14d,%edi + xorl %edi,%r8d + roll $7,%r8d + decl %ebp + jnz .Loop + addl 0(%rsp),%esi + addl 4(%rsp),%edi + movq 64(%rsp),%rbp + movl %esi,32(%rsp) + movq 64+8(%rsp),%rsi + movl %edi,36(%rsp) + movq 64+16(%rsp),%rdi + + addl $0x61707865,%eax + addl $0x3320646e,%ebx + addl $0x79622d32,%ecx + addl $0x6b206574,%edx + addl 16(%rsp),%r8d + addl 20(%rsp),%r9d + addl 24(%rsp),%r10d + addl 28(%rsp),%r11d + addl 48(%rsp),%r12d + addl 52(%rsp),%r13d + addl 56(%rsp),%r14d + addl 60(%rsp),%r15d + + cmpq $64,%rbp + jb .Ltail + + xorl 0(%rsi),%eax + xorl 4(%rsi),%ebx + xorl 8(%rsi),%ecx + xorl 12(%rsi),%edx + movl %eax,0(%rdi) + movl 32(%rsp),%eax + movl %ebx,4(%rdi) + movl 36(%rsp),%ebx + movl %ecx,8(%rdi) + movl 40(%rsp),%ecx + movl %edx,12(%rdi) + movl 44(%rsp),%edx + xorl 16(%rsi),%r8d + addl 8(%rsp),%ecx + xorl 20(%rsi),%r9d + addl 12(%rsp),%edx + xorl 24(%rsi),%r10d + xorl 28(%rsi),%r11d + xorl 32(%rsi),%eax + xorl 36(%rsi),%ebx + xorl 40(%rsi),%ecx + xorl 44(%rsi),%edx + xorl 48(%rsi),%r12d + xorl 52(%rsi),%r13d + xorl 56(%rsi),%r14d + xorl 60(%rsi),%r15d + leaq 64(%rsi),%rsi + + addl $1,48(%rsp) + + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + movl %eax,32(%rdi) + movl %ebx,36(%rdi) + movl %ecx,40(%rdi) + movl %edx,44(%rdi) + movl %r12d,48(%rdi) + movl %r13d,52(%rdi) + movl %r14d,56(%rdi) + movl %r15d,60(%rdi) + leaq 64(%rdi),%rdi + movq 8(%rsp),%r15 + + subq $64,%rbp + jnz .Loop_outer + + jmp .Ldone + +.align 16 +.Ltail: + movl %eax,0(%rsp) + movl 8(%rsp),%eax + movl %ebx,4(%rsp) + movl 12(%rsp),%ebx + movl %ecx,8(%rsp) + addl 40(%rsp),%eax + movl %edx,12(%rsp) + addl 44(%rsp),%ebx + movl %r8d,16(%rsp) + movl %r9d,20(%rsp) + movl %r10d,24(%rsp) + movl %r11d,28(%rsp) + movl %eax,40(%rsp) + movl %ebx,44(%rsp) + xorq %rbx,%rbx + movl %r12d,48(%rsp) + movl %r13d,52(%rsp) + movl %r14d,56(%rsp) + movl %r15d,60(%rsp) + +.Loop_tail: + movzbl (%rsi,%rbx,1),%eax + movzbl (%rsp,%rbx,1),%edx + leaq 1(%rbx),%rbx + xorl %edx,%eax + movb %al,-1(%rdi,%rbx,1) + decq %rbp + jnz .Loop_tail + +.Ldone: + leaq 64+24+48(%rsp),%rsi +.cfi_def_cfa %rsi,8 + movq -48(%rsi),%r15 +.cfi_restore %r15 + movq -40(%rsi),%r14 +.cfi_restore %r14 + movq -32(%rsi),%r13 +.cfi_restore %r13 + movq -24(%rsi),%r12 +.cfi_restore %r12 + movq -16(%rsi),%rbp +.cfi_restore %rbp + movq -8(%rsi),%rbx +.cfi_restore %rbx + leaq (%rsi),%rsp +.cfi_def_cfa_register %rsp +.Lno_data: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_chacha20_asm_ctr32,.-crypton_chacha20_asm_ctr32 +.type crypton_chacha20_asm_ssse3,@function +.align 32 +crypton_chacha20_asm_ssse3: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +.Lcrypton_chacha20_asm_ssse3: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + testl $2048,%r9d + jnz .Lcrypton_chacha20_asm_4xop + cmpq $128,%rdx + je .Lcrypton_chacha20_asm_128 + ja .Lcrypton_chacha20_asm_4x + +.Ldo_sse3_after_all: + subq $64+8,%rsp + andq $-16,%rsp + movdqa .Lsigma(%rip),%xmm0 + movdqu (%rcx),%xmm1 + movdqu 16(%rcx),%xmm2 + movdqu (%r8),%xmm3 + movdqa .Lrot16(%rip),%xmm6 + movdqa .Lrot24(%rip),%xmm7 + + movdqa %xmm0,0(%rsp) + movdqa %xmm1,16(%rsp) + movdqa %xmm2,32(%rsp) + movdqa %xmm3,48(%rsp) + movq $10,%r8 + jmp .Loop_ssse3 + +.align 32 +.Loop_outer_ssse3: + movdqa .Lone(%rip),%xmm3 + movdqa 0(%rsp),%xmm0 + movdqa 16(%rsp),%xmm1 + movdqa 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + movq $10,%r8 + movdqa %xmm3,48(%rsp) + jmp .Loop_ssse3 + +.align 32 +.Loop_ssse3: + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,222 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $20,%xmm1 + pslld $12,%xmm4 + por %xmm4,%xmm1 + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,223 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $25,%xmm1 + pslld $7,%xmm4 + por %xmm4,%xmm1 + pshufd $78,%xmm2,%xmm2 + pshufd $57,%xmm1,%xmm1 + pshufd $147,%xmm3,%xmm3 + nop + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,222 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $20,%xmm1 + pslld $12,%xmm4 + por %xmm4,%xmm1 + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,223 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $25,%xmm1 + pslld $7,%xmm4 + por %xmm4,%xmm1 + pshufd $78,%xmm2,%xmm2 + pshufd $147,%xmm1,%xmm1 + pshufd $57,%xmm3,%xmm3 + decq %r8 + jnz .Loop_ssse3 + paddd 0(%rsp),%xmm0 + paddd 16(%rsp),%xmm1 + paddd 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + + cmpq $64,%rdx + jb .Ltail_ssse3 + + movdqu 0(%rsi),%xmm4 + movdqu 16(%rsi),%xmm5 + pxor %xmm4,%xmm0 + movdqu 32(%rsi),%xmm4 + pxor %xmm5,%xmm1 + movdqu 48(%rsi),%xmm5 + leaq 64(%rsi),%rsi + pxor %xmm4,%xmm2 + pxor %xmm5,%xmm3 + + movdqu %xmm0,0(%rdi) + movdqu %xmm1,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm3,48(%rdi) + leaq 64(%rdi),%rdi + + subq $64,%rdx + jnz .Loop_outer_ssse3 + + jmp .Ldone_ssse3 + +.align 16 +.Ltail_ssse3: + movdqa %xmm0,0(%rsp) + movdqa %xmm1,16(%rsp) + movdqa %xmm2,32(%rsp) + movdqa %xmm3,48(%rsp) + xorq %r8,%r8 + +.Loop_tail_ssse3: + movzbl (%rsi,%r8,1),%eax + movzbl (%rsp,%r8,1),%ecx + leaq 1(%r8),%r8 + xorl %ecx,%eax + movb %al,-1(%rdi,%r8,1) + decq %rdx + jnz .Loop_tail_ssse3 + +.Ldone_ssse3: + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +.Lssse3_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_chacha20_asm_ssse3,.-crypton_chacha20_asm_ssse3 +.type crypton_chacha20_asm_128,@function +.align 32 +crypton_chacha20_asm_128: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +.Lcrypton_chacha20_asm_128: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + subq $64+8,%rsp + andq $-16,%rsp + movdqa .Lsigma(%rip),%xmm8 + movdqu (%rcx),%xmm9 + movdqu 16(%rcx),%xmm2 + movdqu (%r8),%xmm3 + movdqa .Lone(%rip),%xmm1 + movdqa .Lrot16(%rip),%xmm6 + movdqa .Lrot24(%rip),%xmm7 + + movdqa %xmm8,%xmm10 + movdqa %xmm8,0(%rsp) + movdqa %xmm9,%xmm11 + movdqa %xmm9,16(%rsp) + movdqa %xmm2,%xmm0 + movdqa %xmm2,32(%rsp) + paddd %xmm3,%xmm1 + movdqa %xmm3,48(%rsp) + movq $10,%r8 + jmp .Loop_128 + +.align 32 +.Loop_128: + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,222 +.byte 102,15,56,0,206 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $20,%xmm9 + movdqa %xmm11,%xmm5 + pslld $12,%xmm4 + psrld $20,%xmm11 + por %xmm4,%xmm9 + pslld $12,%xmm5 + por %xmm5,%xmm11 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,223 +.byte 102,15,56,0,207 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $25,%xmm9 + movdqa %xmm11,%xmm5 + pslld $7,%xmm4 + psrld $25,%xmm11 + por %xmm4,%xmm9 + pslld $7,%xmm5 + por %xmm5,%xmm11 + pshufd $78,%xmm2,%xmm2 + pshufd $57,%xmm9,%xmm9 + pshufd $147,%xmm3,%xmm3 + pshufd $78,%xmm0,%xmm0 + pshufd $57,%xmm11,%xmm11 + pshufd $147,%xmm1,%xmm1 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,222 +.byte 102,15,56,0,206 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $20,%xmm9 + movdqa %xmm11,%xmm5 + pslld $12,%xmm4 + psrld $20,%xmm11 + por %xmm4,%xmm9 + pslld $12,%xmm5 + por %xmm5,%xmm11 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,223 +.byte 102,15,56,0,207 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $25,%xmm9 + movdqa %xmm11,%xmm5 + pslld $7,%xmm4 + psrld $25,%xmm11 + por %xmm4,%xmm9 + pslld $7,%xmm5 + por %xmm5,%xmm11 + pshufd $78,%xmm2,%xmm2 + pshufd $147,%xmm9,%xmm9 + pshufd $57,%xmm3,%xmm3 + pshufd $78,%xmm0,%xmm0 + pshufd $147,%xmm11,%xmm11 + pshufd $57,%xmm1,%xmm1 + decq %r8 + jnz .Loop_128 + paddd 0(%rsp),%xmm8 + paddd 16(%rsp),%xmm9 + paddd 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + paddd .Lone(%rip),%xmm1 + paddd 0(%rsp),%xmm10 + paddd 16(%rsp),%xmm11 + paddd 32(%rsp),%xmm0 + paddd 48(%rsp),%xmm1 + + movdqu 0(%rsi),%xmm4 + movdqu 16(%rsi),%xmm5 + pxor %xmm4,%xmm8 + movdqu 32(%rsi),%xmm4 + pxor %xmm5,%xmm9 + movdqu 48(%rsi),%xmm5 + pxor %xmm4,%xmm2 + movdqu 64(%rsi),%xmm4 + pxor %xmm5,%xmm3 + movdqu 80(%rsi),%xmm5 + pxor %xmm4,%xmm10 + movdqu 96(%rsi),%xmm4 + pxor %xmm5,%xmm11 + movdqu 112(%rsi),%xmm5 + pxor %xmm4,%xmm0 + pxor %xmm5,%xmm1 + + movdqu %xmm8,0(%rdi) + movdqu %xmm9,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm3,48(%rdi) + movdqu %xmm10,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm0,96(%rdi) + movdqu %xmm1,112(%rdi) + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +.L128_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_chacha20_asm_128,.-crypton_chacha20_asm_128 +.type crypton_chacha20_asm_4x,@function +.align 32 +crypton_chacha20_asm_4x: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +.Lcrypton_chacha20_asm_4x: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + movq %r9,%r11 + shrq $32,%r9 + testq $32,%r9 + jnz .Lcrypton_chacha20_asm_8x + cmpq $192,%rdx + ja .Lproceed4x + + andq $71303168,%r11 + cmpq $4194304,%r11 + je .Ldo_sse3_after_all + +.Lproceed4x: + subq $0x140+8,%rsp + andq $-16,%rsp + movdqa .Lsigma(%rip),%xmm11 + movdqu (%rcx),%xmm15 + movdqu 16(%rcx),%xmm7 + movdqu (%r8),%xmm3 + leaq 256(%rsp),%rcx + leaq .Lrot16(%rip),%r9 + leaq .Lrot24(%rip),%r11 + + pshufd $0x00,%xmm11,%xmm8 + pshufd $0x55,%xmm11,%xmm9 + movdqa %xmm8,64(%rsp) + pshufd $0xaa,%xmm11,%xmm10 + movdqa %xmm9,80(%rsp) + pshufd $0xff,%xmm11,%xmm11 + movdqa %xmm10,96(%rsp) + movdqa %xmm11,112(%rsp) + + pshufd $0x00,%xmm15,%xmm12 + pshufd $0x55,%xmm15,%xmm13 + movdqa %xmm12,128-256(%rcx) + pshufd $0xaa,%xmm15,%xmm14 + movdqa %xmm13,144-256(%rcx) + pshufd $0xff,%xmm15,%xmm15 + movdqa %xmm14,160-256(%rcx) + movdqa %xmm15,176-256(%rcx) + + pshufd $0x00,%xmm7,%xmm4 + pshufd $0x55,%xmm7,%xmm5 + movdqa %xmm4,192-256(%rcx) + pshufd $0xaa,%xmm7,%xmm6 + movdqa %xmm5,208-256(%rcx) + pshufd $0xff,%xmm7,%xmm7 + movdqa %xmm6,224-256(%rcx) + movdqa %xmm7,240-256(%rcx) + + pshufd $0x00,%xmm3,%xmm0 + pshufd $0x55,%xmm3,%xmm1 + paddd .Linc(%rip),%xmm0 + pshufd $0xaa,%xmm3,%xmm2 + movdqa %xmm1,272-256(%rcx) + pshufd $0xff,%xmm3,%xmm3 + movdqa %xmm2,288-256(%rcx) + movdqa %xmm3,304-256(%rcx) + + jmp .Loop_enter4x + +.align 32 +.Loop_outer4x: + movdqa 64(%rsp),%xmm8 + movdqa 80(%rsp),%xmm9 + movdqa 96(%rsp),%xmm10 + movdqa 112(%rsp),%xmm11 + movdqa 128-256(%rcx),%xmm12 + movdqa 144-256(%rcx),%xmm13 + movdqa 160-256(%rcx),%xmm14 + movdqa 176-256(%rcx),%xmm15 + movdqa 192-256(%rcx),%xmm4 + movdqa 208-256(%rcx),%xmm5 + movdqa 224-256(%rcx),%xmm6 + movdqa 240-256(%rcx),%xmm7 + movdqa 256-256(%rcx),%xmm0 + movdqa 272-256(%rcx),%xmm1 + movdqa 288-256(%rcx),%xmm2 + movdqa 304-256(%rcx),%xmm3 + paddd .Lfour(%rip),%xmm0 + +.Loop_enter4x: + movdqa %xmm6,32(%rsp) + movdqa %xmm7,48(%rsp) + movdqa (%r9),%xmm7 + movl $10,%eax + movdqa %xmm0,256-256(%rcx) + jmp .Loop4x + +.align 32 +.Loop4x: + paddd %xmm12,%xmm8 + paddd %xmm13,%xmm9 + pxor %xmm8,%xmm0 + pxor %xmm9,%xmm1 +.byte 102,15,56,0,199 +.byte 102,15,56,0,207 + paddd %xmm0,%xmm4 + paddd %xmm1,%xmm5 + pxor %xmm4,%xmm12 + pxor %xmm5,%xmm13 + movdqa %xmm12,%xmm6 + pslld $12,%xmm12 + psrld $20,%xmm6 + movdqa %xmm13,%xmm7 + pslld $12,%xmm13 + por %xmm6,%xmm12 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm13 + paddd %xmm12,%xmm8 + paddd %xmm13,%xmm9 + pxor %xmm8,%xmm0 + pxor %xmm9,%xmm1 +.byte 102,15,56,0,198 +.byte 102,15,56,0,206 + paddd %xmm0,%xmm4 + paddd %xmm1,%xmm5 + pxor %xmm4,%xmm12 + pxor %xmm5,%xmm13 + movdqa %xmm12,%xmm7 + pslld $7,%xmm12 + psrld $25,%xmm7 + movdqa %xmm13,%xmm6 + pslld $7,%xmm13 + por %xmm7,%xmm12 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm13 + movdqa %xmm4,0(%rsp) + movdqa %xmm5,16(%rsp) + movdqa 32(%rsp),%xmm4 + movdqa 48(%rsp),%xmm5 + paddd %xmm14,%xmm10 + paddd %xmm15,%xmm11 + pxor %xmm10,%xmm2 + pxor %xmm11,%xmm3 +.byte 102,15,56,0,215 +.byte 102,15,56,0,223 + paddd %xmm2,%xmm4 + paddd %xmm3,%xmm5 + pxor %xmm4,%xmm14 + pxor %xmm5,%xmm15 + movdqa %xmm14,%xmm6 + pslld $12,%xmm14 + psrld $20,%xmm6 + movdqa %xmm15,%xmm7 + pslld $12,%xmm15 + por %xmm6,%xmm14 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm15 + paddd %xmm14,%xmm10 + paddd %xmm15,%xmm11 + pxor %xmm10,%xmm2 + pxor %xmm11,%xmm3 +.byte 102,15,56,0,214 +.byte 102,15,56,0,222 + paddd %xmm2,%xmm4 + paddd %xmm3,%xmm5 + pxor %xmm4,%xmm14 + pxor %xmm5,%xmm15 + movdqa %xmm14,%xmm7 + pslld $7,%xmm14 + psrld $25,%xmm7 + movdqa %xmm15,%xmm6 + pslld $7,%xmm15 + por %xmm7,%xmm14 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm15 + paddd %xmm13,%xmm8 + paddd %xmm14,%xmm9 + pxor %xmm8,%xmm3 + pxor %xmm9,%xmm0 +.byte 102,15,56,0,223 +.byte 102,15,56,0,199 + paddd %xmm3,%xmm4 + paddd %xmm0,%xmm5 + pxor %xmm4,%xmm13 + pxor %xmm5,%xmm14 + movdqa %xmm13,%xmm6 + pslld $12,%xmm13 + psrld $20,%xmm6 + movdqa %xmm14,%xmm7 + pslld $12,%xmm14 + por %xmm6,%xmm13 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm14 + paddd %xmm13,%xmm8 + paddd %xmm14,%xmm9 + pxor %xmm8,%xmm3 + pxor %xmm9,%xmm0 +.byte 102,15,56,0,222 +.byte 102,15,56,0,198 + paddd %xmm3,%xmm4 + paddd %xmm0,%xmm5 + pxor %xmm4,%xmm13 + pxor %xmm5,%xmm14 + movdqa %xmm13,%xmm7 + pslld $7,%xmm13 + psrld $25,%xmm7 + movdqa %xmm14,%xmm6 + pslld $7,%xmm14 + por %xmm7,%xmm13 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm14 + movdqa %xmm4,32(%rsp) + movdqa %xmm5,48(%rsp) + movdqa 0(%rsp),%xmm4 + movdqa 16(%rsp),%xmm5 + paddd %xmm15,%xmm10 + paddd %xmm12,%xmm11 + pxor %xmm10,%xmm1 + pxor %xmm11,%xmm2 +.byte 102,15,56,0,207 +.byte 102,15,56,0,215 + paddd %xmm1,%xmm4 + paddd %xmm2,%xmm5 + pxor %xmm4,%xmm15 + pxor %xmm5,%xmm12 + movdqa %xmm15,%xmm6 + pslld $12,%xmm15 + psrld $20,%xmm6 + movdqa %xmm12,%xmm7 + pslld $12,%xmm12 + por %xmm6,%xmm15 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm12 + paddd %xmm15,%xmm10 + paddd %xmm12,%xmm11 + pxor %xmm10,%xmm1 + pxor %xmm11,%xmm2 +.byte 102,15,56,0,206 +.byte 102,15,56,0,214 + paddd %xmm1,%xmm4 + paddd %xmm2,%xmm5 + pxor %xmm4,%xmm15 + pxor %xmm5,%xmm12 + movdqa %xmm15,%xmm7 + pslld $7,%xmm15 + psrld $25,%xmm7 + movdqa %xmm12,%xmm6 + pslld $7,%xmm12 + por %xmm7,%xmm15 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm12 + decl %eax + jnz .Loop4x + + paddd 64(%rsp),%xmm8 + paddd 80(%rsp),%xmm9 + paddd 96(%rsp),%xmm10 + paddd 112(%rsp),%xmm11 + + movdqa %xmm8,%xmm6 + punpckldq %xmm9,%xmm8 + movdqa %xmm10,%xmm7 + punpckldq %xmm11,%xmm10 + punpckhdq %xmm9,%xmm6 + punpckhdq %xmm11,%xmm7 + movdqa %xmm8,%xmm9 + punpcklqdq %xmm10,%xmm8 + movdqa %xmm6,%xmm11 + punpcklqdq %xmm7,%xmm6 + punpckhqdq %xmm10,%xmm9 + punpckhqdq %xmm7,%xmm11 + paddd 128-256(%rcx),%xmm12 + paddd 144-256(%rcx),%xmm13 + paddd 160-256(%rcx),%xmm14 + paddd 176-256(%rcx),%xmm15 + + movdqa %xmm8,0(%rsp) + movdqa %xmm9,16(%rsp) + movdqa 32(%rsp),%xmm8 + movdqa 48(%rsp),%xmm9 + + movdqa %xmm12,%xmm10 + punpckldq %xmm13,%xmm12 + movdqa %xmm14,%xmm7 + punpckldq %xmm15,%xmm14 + punpckhdq %xmm13,%xmm10 + punpckhdq %xmm15,%xmm7 + movdqa %xmm12,%xmm13 + punpcklqdq %xmm14,%xmm12 + movdqa %xmm10,%xmm15 + punpcklqdq %xmm7,%xmm10 + punpckhqdq %xmm14,%xmm13 + punpckhqdq %xmm7,%xmm15 + paddd 192-256(%rcx),%xmm4 + paddd 208-256(%rcx),%xmm5 + paddd 224-256(%rcx),%xmm8 + paddd 240-256(%rcx),%xmm9 + + movdqa %xmm6,32(%rsp) + movdqa %xmm11,48(%rsp) + + movdqa %xmm4,%xmm14 + punpckldq %xmm5,%xmm4 + movdqa %xmm8,%xmm7 + punpckldq %xmm9,%xmm8 + punpckhdq %xmm5,%xmm14 + punpckhdq %xmm9,%xmm7 + movdqa %xmm4,%xmm5 + punpcklqdq %xmm8,%xmm4 + movdqa %xmm14,%xmm9 + punpcklqdq %xmm7,%xmm14 + punpckhqdq %xmm8,%xmm5 + punpckhqdq %xmm7,%xmm9 + paddd 256-256(%rcx),%xmm0 + paddd 272-256(%rcx),%xmm1 + paddd 288-256(%rcx),%xmm2 + paddd 304-256(%rcx),%xmm3 + + movdqa %xmm0,%xmm8 + punpckldq %xmm1,%xmm0 + movdqa %xmm2,%xmm7 + punpckldq %xmm3,%xmm2 + punpckhdq %xmm1,%xmm8 + punpckhdq %xmm3,%xmm7 + movdqa %xmm0,%xmm1 + punpcklqdq %xmm2,%xmm0 + movdqa %xmm8,%xmm3 + punpcklqdq %xmm7,%xmm8 + punpckhqdq %xmm2,%xmm1 + punpckhqdq %xmm7,%xmm3 + cmpq $256,%rdx + jb .Ltail4x + + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + + movdqu %xmm6,64(%rdi) + movdqu 0(%rsi),%xmm6 + movdqu %xmm11,80(%rdi) + movdqu 16(%rsi),%xmm11 + movdqu %xmm2,96(%rdi) + movdqu 32(%rsi),%xmm2 + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + movdqu 48(%rsi),%xmm7 + pxor 32(%rsp),%xmm6 + pxor %xmm10,%xmm11 + pxor %xmm14,%xmm2 + pxor %xmm8,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 48(%rsp),%xmm6 + pxor %xmm15,%xmm11 + pxor %xmm9,%xmm2 + pxor %xmm3,%xmm7 + movdqu %xmm6,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm2,96(%rdi) + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + + subq $256,%rdx + jnz .Loop_outer4x + + jmp .Ldone4x + +.Ltail4x: + cmpq $192,%rdx + jae .L192_or_more4x + cmpq $128,%rdx + jae .L128_or_more4x + cmpq $64,%rdx + jae .L64_or_more4x + + + xorq %r9,%r9 + + movdqa %xmm12,16(%rsp) + movdqa %xmm4,32(%rsp) + movdqa %xmm0,48(%rsp) + jmp .Loop_tail4x + +.align 32 +.L64_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + movdqu %xmm6,0(%rdi) + movdqu %xmm11,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm7,48(%rdi) + je .Ldone4x + + movdqa 16(%rsp),%xmm6 + leaq 64(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm13,16(%rsp) + leaq 64(%rdi),%rdi + movdqa %xmm5,32(%rsp) + subq $64,%rdx + movdqa %xmm1,48(%rsp) + jmp .Loop_tail4x + +.align 32 +.L128_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + movdqu %xmm6,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm2,96(%rdi) + movdqu %xmm7,112(%rdi) + je .Ldone4x + + movdqa 32(%rsp),%xmm6 + leaq 128(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm10,16(%rsp) + leaq 128(%rdi),%rdi + movdqa %xmm14,32(%rsp) + subq $128,%rdx + movdqa %xmm8,48(%rsp) + jmp .Loop_tail4x + +.align 32 +.L192_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + + movdqu %xmm6,64(%rdi) + movdqu 0(%rsi),%xmm6 + movdqu %xmm11,80(%rdi) + movdqu 16(%rsi),%xmm11 + movdqu %xmm2,96(%rdi) + movdqu 32(%rsi),%xmm2 + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + movdqu 48(%rsi),%xmm7 + pxor 32(%rsp),%xmm6 + pxor %xmm10,%xmm11 + pxor %xmm14,%xmm2 + pxor %xmm8,%xmm7 + movdqu %xmm6,0(%rdi) + movdqu %xmm11,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm7,48(%rdi) + je .Ldone4x + + movdqa 48(%rsp),%xmm6 + leaq 64(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm15,16(%rsp) + leaq 64(%rdi),%rdi + movdqa %xmm9,32(%rsp) + subq $192,%rdx + movdqa %xmm3,48(%rsp) + +.Loop_tail4x: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz .Loop_tail4x + +.Ldone4x: + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +.L4x_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_chacha20_asm_4x,.-crypton_chacha20_asm_4x +.type crypton_chacha20_asm_4xop,@function +.align 32 +crypton_chacha20_asm_4xop: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +.Lcrypton_chacha20_asm_4xop: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + subq $0x140+8,%rsp + andq $-16,%rsp + vzeroupper + + vmovdqa .Lsigma(%rip),%xmm11 + vmovdqu (%rcx),%xmm3 + vmovdqu 16(%rcx),%xmm15 + vmovdqu (%r8),%xmm7 + leaq 256(%rsp),%rcx + + vpshufd $0x00,%xmm11,%xmm8 + vpshufd $0x55,%xmm11,%xmm9 + vmovdqa %xmm8,64(%rsp) + vpshufd $0xaa,%xmm11,%xmm10 + vmovdqa %xmm9,80(%rsp) + vpshufd $0xff,%xmm11,%xmm11 + vmovdqa %xmm10,96(%rsp) + vmovdqa %xmm11,112(%rsp) + + vpshufd $0x00,%xmm3,%xmm0 + vpshufd $0x55,%xmm3,%xmm1 + vmovdqa %xmm0,128-256(%rcx) + vpshufd $0xaa,%xmm3,%xmm2 + vmovdqa %xmm1,144-256(%rcx) + vpshufd $0xff,%xmm3,%xmm3 + vmovdqa %xmm2,160-256(%rcx) + vmovdqa %xmm3,176-256(%rcx) + + vpshufd $0x00,%xmm15,%xmm12 + vpshufd $0x55,%xmm15,%xmm13 + vmovdqa %xmm12,192-256(%rcx) + vpshufd $0xaa,%xmm15,%xmm14 + vmovdqa %xmm13,208-256(%rcx) + vpshufd $0xff,%xmm15,%xmm15 + vmovdqa %xmm14,224-256(%rcx) + vmovdqa %xmm15,240-256(%rcx) + + vpshufd $0x00,%xmm7,%xmm4 + vpshufd $0x55,%xmm7,%xmm5 + vpaddd .Linc(%rip),%xmm4,%xmm4 + vpshufd $0xaa,%xmm7,%xmm6 + vmovdqa %xmm5,272-256(%rcx) + vpshufd $0xff,%xmm7,%xmm7 + vmovdqa %xmm6,288-256(%rcx) + vmovdqa %xmm7,304-256(%rcx) + + jmp .Loop_enter4xop + +.align 32 +.Loop_outer4xop: + vmovdqa 64(%rsp),%xmm8 + vmovdqa 80(%rsp),%xmm9 + vmovdqa 96(%rsp),%xmm10 + vmovdqa 112(%rsp),%xmm11 + vmovdqa 128-256(%rcx),%xmm0 + vmovdqa 144-256(%rcx),%xmm1 + vmovdqa 160-256(%rcx),%xmm2 + vmovdqa 176-256(%rcx),%xmm3 + vmovdqa 192-256(%rcx),%xmm12 + vmovdqa 208-256(%rcx),%xmm13 + vmovdqa 224-256(%rcx),%xmm14 + vmovdqa 240-256(%rcx),%xmm15 + vmovdqa 256-256(%rcx),%xmm4 + vmovdqa 272-256(%rcx),%xmm5 + vmovdqa 288-256(%rcx),%xmm6 + vmovdqa 304-256(%rcx),%xmm7 + vpaddd .Lfour(%rip),%xmm4,%xmm4 + +.Loop_enter4xop: + movl $10,%eax + vmovdqa %xmm4,256-256(%rcx) + jmp .Loop4xop + +.align 32 +.Loop4xop: + vpaddd %xmm0,%xmm8,%xmm8 + vpaddd %xmm1,%xmm9,%xmm9 + vpaddd %xmm2,%xmm10,%xmm10 + vpaddd %xmm3,%xmm11,%xmm11 + vpxor %xmm4,%xmm8,%xmm4 + vpxor %xmm5,%xmm9,%xmm5 + vpxor %xmm6,%xmm10,%xmm6 + vpxor %xmm7,%xmm11,%xmm7 +.byte 143,232,120,194,228,16 +.byte 143,232,120,194,237,16 +.byte 143,232,120,194,246,16 +.byte 143,232,120,194,255,16 + vpaddd %xmm4,%xmm12,%xmm12 + vpaddd %xmm5,%xmm13,%xmm13 + vpaddd %xmm6,%xmm14,%xmm14 + vpaddd %xmm7,%xmm15,%xmm15 + vpxor %xmm0,%xmm12,%xmm0 + vpxor %xmm1,%xmm13,%xmm1 + vpxor %xmm14,%xmm2,%xmm2 + vpxor %xmm15,%xmm3,%xmm3 +.byte 143,232,120,194,192,12 +.byte 143,232,120,194,201,12 +.byte 143,232,120,194,210,12 +.byte 143,232,120,194,219,12 + vpaddd %xmm8,%xmm0,%xmm8 + vpaddd %xmm9,%xmm1,%xmm9 + vpaddd %xmm2,%xmm10,%xmm10 + vpaddd %xmm3,%xmm11,%xmm11 + vpxor %xmm4,%xmm8,%xmm4 + vpxor %xmm5,%xmm9,%xmm5 + vpxor %xmm6,%xmm10,%xmm6 + vpxor %xmm7,%xmm11,%xmm7 +.byte 143,232,120,194,228,8 +.byte 143,232,120,194,237,8 +.byte 143,232,120,194,246,8 +.byte 143,232,120,194,255,8 + vpaddd %xmm4,%xmm12,%xmm12 + vpaddd %xmm5,%xmm13,%xmm13 + vpaddd %xmm6,%xmm14,%xmm14 + vpaddd %xmm7,%xmm15,%xmm15 + vpxor %xmm0,%xmm12,%xmm0 + vpxor %xmm1,%xmm13,%xmm1 + vpxor %xmm14,%xmm2,%xmm2 + vpxor %xmm15,%xmm3,%xmm3 +.byte 143,232,120,194,192,7 +.byte 143,232,120,194,201,7 +.byte 143,232,120,194,210,7 +.byte 143,232,120,194,219,7 + vpaddd %xmm1,%xmm8,%xmm8 + vpaddd %xmm2,%xmm9,%xmm9 + vpaddd %xmm3,%xmm10,%xmm10 + vpaddd %xmm0,%xmm11,%xmm11 + vpxor %xmm7,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm4 + vpxor %xmm5,%xmm10,%xmm5 + vpxor %xmm6,%xmm11,%xmm6 +.byte 143,232,120,194,255,16 +.byte 143,232,120,194,228,16 +.byte 143,232,120,194,237,16 +.byte 143,232,120,194,246,16 + vpaddd %xmm7,%xmm14,%xmm14 + vpaddd %xmm4,%xmm15,%xmm15 + vpaddd %xmm5,%xmm12,%xmm12 + vpaddd %xmm6,%xmm13,%xmm13 + vpxor %xmm1,%xmm14,%xmm1 + vpxor %xmm2,%xmm15,%xmm2 + vpxor %xmm12,%xmm3,%xmm3 + vpxor %xmm13,%xmm0,%xmm0 +.byte 143,232,120,194,201,12 +.byte 143,232,120,194,210,12 +.byte 143,232,120,194,219,12 +.byte 143,232,120,194,192,12 + vpaddd %xmm8,%xmm1,%xmm8 + vpaddd %xmm9,%xmm2,%xmm9 + vpaddd %xmm3,%xmm10,%xmm10 + vpaddd %xmm0,%xmm11,%xmm11 + vpxor %xmm7,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm4 + vpxor %xmm5,%xmm10,%xmm5 + vpxor %xmm6,%xmm11,%xmm6 +.byte 143,232,120,194,255,8 +.byte 143,232,120,194,228,8 +.byte 143,232,120,194,237,8 +.byte 143,232,120,194,246,8 + vpaddd %xmm7,%xmm14,%xmm14 + vpaddd %xmm4,%xmm15,%xmm15 + vpaddd %xmm5,%xmm12,%xmm12 + vpaddd %xmm6,%xmm13,%xmm13 + vpxor %xmm1,%xmm14,%xmm1 + vpxor %xmm2,%xmm15,%xmm2 + vpxor %xmm12,%xmm3,%xmm3 + vpxor %xmm13,%xmm0,%xmm0 +.byte 143,232,120,194,201,7 +.byte 143,232,120,194,210,7 +.byte 143,232,120,194,219,7 +.byte 143,232,120,194,192,7 + decl %eax + jnz .Loop4xop + + vpaddd 64(%rsp),%xmm8,%xmm8 + vpaddd 80(%rsp),%xmm9,%xmm9 + vpaddd 96(%rsp),%xmm10,%xmm10 + vpaddd 112(%rsp),%xmm11,%xmm11 + + vmovdqa %xmm14,32(%rsp) + vmovdqa %xmm15,48(%rsp) + + vpunpckldq %xmm9,%xmm8,%xmm14 + vpunpckldq %xmm11,%xmm10,%xmm15 + vpunpckhdq %xmm9,%xmm8,%xmm8 + vpunpckhdq %xmm11,%xmm10,%xmm10 + vpunpcklqdq %xmm15,%xmm14,%xmm9 + vpunpckhqdq %xmm15,%xmm14,%xmm14 + vpunpcklqdq %xmm10,%xmm8,%xmm11 + vpunpckhqdq %xmm10,%xmm8,%xmm8 + vpaddd 128-256(%rcx),%xmm0,%xmm0 + vpaddd 144-256(%rcx),%xmm1,%xmm1 + vpaddd 160-256(%rcx),%xmm2,%xmm2 + vpaddd 176-256(%rcx),%xmm3,%xmm3 + + vmovdqa %xmm9,0(%rsp) + vmovdqa %xmm14,16(%rsp) + vmovdqa 32(%rsp),%xmm9 + vmovdqa 48(%rsp),%xmm14 + + vpunpckldq %xmm1,%xmm0,%xmm10 + vpunpckldq %xmm3,%xmm2,%xmm15 + vpunpckhdq %xmm1,%xmm0,%xmm0 + vpunpckhdq %xmm3,%xmm2,%xmm2 + vpunpcklqdq %xmm15,%xmm10,%xmm1 + vpunpckhqdq %xmm15,%xmm10,%xmm10 + vpunpcklqdq %xmm2,%xmm0,%xmm3 + vpunpckhqdq %xmm2,%xmm0,%xmm0 + vpaddd 192-256(%rcx),%xmm12,%xmm12 + vpaddd 208-256(%rcx),%xmm13,%xmm13 + vpaddd 224-256(%rcx),%xmm9,%xmm9 + vpaddd 240-256(%rcx),%xmm14,%xmm14 + + vpunpckldq %xmm13,%xmm12,%xmm2 + vpunpckldq %xmm14,%xmm9,%xmm15 + vpunpckhdq %xmm13,%xmm12,%xmm12 + vpunpckhdq %xmm14,%xmm9,%xmm9 + vpunpcklqdq %xmm15,%xmm2,%xmm13 + vpunpckhqdq %xmm15,%xmm2,%xmm2 + vpunpcklqdq %xmm9,%xmm12,%xmm14 + vpunpckhqdq %xmm9,%xmm12,%xmm12 + vpaddd 256-256(%rcx),%xmm4,%xmm4 + vpaddd 272-256(%rcx),%xmm5,%xmm5 + vpaddd 288-256(%rcx),%xmm6,%xmm6 + vpaddd 304-256(%rcx),%xmm7,%xmm7 + + vpunpckldq %xmm5,%xmm4,%xmm9 + vpunpckldq %xmm7,%xmm6,%xmm15 + vpunpckhdq %xmm5,%xmm4,%xmm4 + vpunpckhdq %xmm7,%xmm6,%xmm6 + vpunpcklqdq %xmm15,%xmm9,%xmm5 + vpunpckhqdq %xmm15,%xmm9,%xmm9 + vpunpcklqdq %xmm6,%xmm4,%xmm7 + vpunpckhqdq %xmm6,%xmm4,%xmm4 + vmovdqa 0(%rsp),%xmm6 + vmovdqa 16(%rsp),%xmm15 + + cmpq $256,%rdx + jb .Ltail4xop + + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + leaq 128(%rsi),%rsi + vpxor 0(%rsi),%xmm11,%xmm11 + vpxor 16(%rsi),%xmm3,%xmm3 + vpxor 32(%rsi),%xmm14,%xmm14 + vpxor 48(%rsi),%xmm7,%xmm7 + vpxor 64(%rsi),%xmm8,%xmm8 + vpxor 80(%rsi),%xmm0,%xmm0 + vpxor 96(%rsi),%xmm12,%xmm12 + vpxor 112(%rsi),%xmm4,%xmm4 + leaq 128(%rsi),%rsi + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + leaq 128(%rdi),%rdi + vmovdqu %xmm11,0(%rdi) + vmovdqu %xmm3,16(%rdi) + vmovdqu %xmm14,32(%rdi) + vmovdqu %xmm7,48(%rdi) + vmovdqu %xmm8,64(%rdi) + vmovdqu %xmm0,80(%rdi) + vmovdqu %xmm12,96(%rdi) + vmovdqu %xmm4,112(%rdi) + leaq 128(%rdi),%rdi + + subq $256,%rdx + jnz .Loop_outer4xop + + jmp .Ldone4xop + +.align 32 +.Ltail4xop: + cmpq $192,%rdx + jae .L192_or_more4xop + cmpq $128,%rdx + jae .L128_or_more4xop + cmpq $64,%rdx + jae .L64_or_more4xop + + xorq %r9,%r9 + vmovdqa %xmm6,0(%rsp) + vmovdqa %xmm1,16(%rsp) + vmovdqa %xmm13,32(%rsp) + vmovdqa %xmm5,48(%rsp) + jmp .Loop_tail4xop + +.align 32 +.L64_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + je .Ldone4xop + + leaq 64(%rsi),%rsi + vmovdqa %xmm15,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm10,16(%rsp) + leaq 64(%rdi),%rdi + vmovdqa %xmm2,32(%rsp) + subq $64,%rdx + vmovdqa %xmm9,48(%rsp) + jmp .Loop_tail4xop + +.align 32 +.L128_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + je .Ldone4xop + + leaq 128(%rsi),%rsi + vmovdqa %xmm11,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm3,16(%rsp) + leaq 128(%rdi),%rdi + vmovdqa %xmm14,32(%rsp) + subq $128,%rdx + vmovdqa %xmm7,48(%rsp) + jmp .Loop_tail4xop + +.align 32 +.L192_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + leaq 128(%rsi),%rsi + vpxor 0(%rsi),%xmm11,%xmm11 + vpxor 16(%rsi),%xmm3,%xmm3 + vpxor 32(%rsi),%xmm14,%xmm14 + vpxor 48(%rsi),%xmm7,%xmm7 + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + leaq 128(%rdi),%rdi + vmovdqu %xmm11,0(%rdi) + vmovdqu %xmm3,16(%rdi) + vmovdqu %xmm14,32(%rdi) + vmovdqu %xmm7,48(%rdi) + je .Ldone4xop + + leaq 64(%rsi),%rsi + vmovdqa %xmm8,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm0,16(%rsp) + leaq 64(%rdi),%rdi + vmovdqa %xmm12,32(%rsp) + subq $192,%rdx + vmovdqa %xmm4,48(%rsp) + +.Loop_tail4xop: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz .Loop_tail4xop + +.Ldone4xop: + vzeroupper + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +.L4xop_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_chacha20_asm_4xop,.-crypton_chacha20_asm_4xop +.type crypton_chacha20_asm_avx2,@function +.align 32 +crypton_chacha20_asm_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +.Lcrypton_chacha20_asm_8x: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + subq $0x280+8,%rsp + andq $-32,%rsp + vzeroupper + + + + + + + + + + + vbroadcasti128 .Lsigma(%rip),%ymm11 + vbroadcasti128 (%rcx),%ymm3 + vbroadcasti128 16(%rcx),%ymm15 + vbroadcasti128 (%r8),%ymm7 + leaq 256(%rsp),%rcx + leaq 512(%rsp),%rax + leaq .Lrot16(%rip),%r9 + leaq .Lrot24(%rip),%r11 + + vpshufd $0x00,%ymm11,%ymm8 + vpshufd $0x55,%ymm11,%ymm9 + vmovdqa %ymm8,128-256(%rcx) + vpshufd $0xaa,%ymm11,%ymm10 + vmovdqa %ymm9,160-256(%rcx) + vpshufd $0xff,%ymm11,%ymm11 + vmovdqa %ymm10,192-256(%rcx) + vmovdqa %ymm11,224-256(%rcx) + + vpshufd $0x00,%ymm3,%ymm0 + vpshufd $0x55,%ymm3,%ymm1 + vmovdqa %ymm0,256-256(%rcx) + vpshufd $0xaa,%ymm3,%ymm2 + vmovdqa %ymm1,288-256(%rcx) + vpshufd $0xff,%ymm3,%ymm3 + vmovdqa %ymm2,320-256(%rcx) + vmovdqa %ymm3,352-256(%rcx) + + vpshufd $0x00,%ymm15,%ymm12 + vpshufd $0x55,%ymm15,%ymm13 + vmovdqa %ymm12,384-512(%rax) + vpshufd $0xaa,%ymm15,%ymm14 + vmovdqa %ymm13,416-512(%rax) + vpshufd $0xff,%ymm15,%ymm15 + vmovdqa %ymm14,448-512(%rax) + vmovdqa %ymm15,480-512(%rax) + + vpshufd $0x00,%ymm7,%ymm4 + vpshufd $0x55,%ymm7,%ymm5 + vpaddd .Lincy(%rip),%ymm4,%ymm4 + vpshufd $0xaa,%ymm7,%ymm6 + vmovdqa %ymm5,544-512(%rax) + vpshufd $0xff,%ymm7,%ymm7 + vmovdqa %ymm6,576-512(%rax) + vmovdqa %ymm7,608-512(%rax) + + jmp .Loop_enter8x + +.align 32 +.Loop_outer8x: + vmovdqa 128-256(%rcx),%ymm8 + vmovdqa 160-256(%rcx),%ymm9 + vmovdqa 192-256(%rcx),%ymm10 + vmovdqa 224-256(%rcx),%ymm11 + vmovdqa 256-256(%rcx),%ymm0 + vmovdqa 288-256(%rcx),%ymm1 + vmovdqa 320-256(%rcx),%ymm2 + vmovdqa 352-256(%rcx),%ymm3 + vmovdqa 384-512(%rax),%ymm12 + vmovdqa 416-512(%rax),%ymm13 + vmovdqa 448-512(%rax),%ymm14 + vmovdqa 480-512(%rax),%ymm15 + vmovdqa 512-512(%rax),%ymm4 + vmovdqa 544-512(%rax),%ymm5 + vmovdqa 576-512(%rax),%ymm6 + vmovdqa 608-512(%rax),%ymm7 + vpaddd .Leight(%rip),%ymm4,%ymm4 + +.Loop_enter8x: + vmovdqa %ymm14,64(%rsp) + vmovdqa %ymm15,96(%rsp) + vbroadcasti128 (%r9),%ymm15 + vmovdqa %ymm4,512-512(%rax) + movl $10,%eax + jmp .Loop8x + +.align 32 +.Loop8x: + vpaddd %ymm0,%ymm8,%ymm8 + vpxor %ymm4,%ymm8,%ymm4 + vpshufb %ymm15,%ymm4,%ymm4 + vpaddd %ymm1,%ymm9,%ymm9 + vpxor %ymm5,%ymm9,%ymm5 + vpshufb %ymm15,%ymm5,%ymm5 + vpaddd %ymm4,%ymm12,%ymm12 + vpxor %ymm0,%ymm12,%ymm0 + vpslld $12,%ymm0,%ymm14 + vpsrld $20,%ymm0,%ymm0 + vpor %ymm0,%ymm14,%ymm0 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm5,%ymm13,%ymm13 + vpxor %ymm1,%ymm13,%ymm1 + vpslld $12,%ymm1,%ymm15 + vpsrld $20,%ymm1,%ymm1 + vpor %ymm1,%ymm15,%ymm1 + vpaddd %ymm0,%ymm8,%ymm8 + vpxor %ymm4,%ymm8,%ymm4 + vpshufb %ymm14,%ymm4,%ymm4 + vpaddd %ymm1,%ymm9,%ymm9 + vpxor %ymm5,%ymm9,%ymm5 + vpshufb %ymm14,%ymm5,%ymm5 + vpaddd %ymm4,%ymm12,%ymm12 + vpxor %ymm0,%ymm12,%ymm0 + vpslld $7,%ymm0,%ymm15 + vpsrld $25,%ymm0,%ymm0 + vpor %ymm0,%ymm15,%ymm0 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm5,%ymm13,%ymm13 + vpxor %ymm1,%ymm13,%ymm1 + vpslld $7,%ymm1,%ymm14 + vpsrld $25,%ymm1,%ymm1 + vpor %ymm1,%ymm14,%ymm1 + vmovdqa %ymm12,0(%rsp) + vmovdqa %ymm13,32(%rsp) + vmovdqa 64(%rsp),%ymm12 + vmovdqa 96(%rsp),%ymm13 + vpaddd %ymm2,%ymm10,%ymm10 + vpxor %ymm6,%ymm10,%ymm6 + vpshufb %ymm15,%ymm6,%ymm6 + vpaddd %ymm3,%ymm11,%ymm11 + vpxor %ymm7,%ymm11,%ymm7 + vpshufb %ymm15,%ymm7,%ymm7 + vpaddd %ymm6,%ymm12,%ymm12 + vpxor %ymm2,%ymm12,%ymm2 + vpslld $12,%ymm2,%ymm14 + vpsrld $20,%ymm2,%ymm2 + vpor %ymm2,%ymm14,%ymm2 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm7,%ymm13,%ymm13 + vpxor %ymm3,%ymm13,%ymm3 + vpslld $12,%ymm3,%ymm15 + vpsrld $20,%ymm3,%ymm3 + vpor %ymm3,%ymm15,%ymm3 + vpaddd %ymm2,%ymm10,%ymm10 + vpxor %ymm6,%ymm10,%ymm6 + vpshufb %ymm14,%ymm6,%ymm6 + vpaddd %ymm3,%ymm11,%ymm11 + vpxor %ymm7,%ymm11,%ymm7 + vpshufb %ymm14,%ymm7,%ymm7 + vpaddd %ymm6,%ymm12,%ymm12 + vpxor %ymm2,%ymm12,%ymm2 + vpslld $7,%ymm2,%ymm15 + vpsrld $25,%ymm2,%ymm2 + vpor %ymm2,%ymm15,%ymm2 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm7,%ymm13,%ymm13 + vpxor %ymm3,%ymm13,%ymm3 + vpslld $7,%ymm3,%ymm14 + vpsrld $25,%ymm3,%ymm3 + vpor %ymm3,%ymm14,%ymm3 + vpaddd %ymm1,%ymm8,%ymm8 + vpxor %ymm7,%ymm8,%ymm7 + vpshufb %ymm15,%ymm7,%ymm7 + vpaddd %ymm2,%ymm9,%ymm9 + vpxor %ymm4,%ymm9,%ymm4 + vpshufb %ymm15,%ymm4,%ymm4 + vpaddd %ymm7,%ymm12,%ymm12 + vpxor %ymm1,%ymm12,%ymm1 + vpslld $12,%ymm1,%ymm14 + vpsrld $20,%ymm1,%ymm1 + vpor %ymm1,%ymm14,%ymm1 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm4,%ymm13,%ymm13 + vpxor %ymm2,%ymm13,%ymm2 + vpslld $12,%ymm2,%ymm15 + vpsrld $20,%ymm2,%ymm2 + vpor %ymm2,%ymm15,%ymm2 + vpaddd %ymm1,%ymm8,%ymm8 + vpxor %ymm7,%ymm8,%ymm7 + vpshufb %ymm14,%ymm7,%ymm7 + vpaddd %ymm2,%ymm9,%ymm9 + vpxor %ymm4,%ymm9,%ymm4 + vpshufb %ymm14,%ymm4,%ymm4 + vpaddd %ymm7,%ymm12,%ymm12 + vpxor %ymm1,%ymm12,%ymm1 + vpslld $7,%ymm1,%ymm15 + vpsrld $25,%ymm1,%ymm1 + vpor %ymm1,%ymm15,%ymm1 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm4,%ymm13,%ymm13 + vpxor %ymm2,%ymm13,%ymm2 + vpslld $7,%ymm2,%ymm14 + vpsrld $25,%ymm2,%ymm2 + vpor %ymm2,%ymm14,%ymm2 + vmovdqa %ymm12,64(%rsp) + vmovdqa %ymm13,96(%rsp) + vmovdqa 0(%rsp),%ymm12 + vmovdqa 32(%rsp),%ymm13 + vpaddd %ymm3,%ymm10,%ymm10 + vpxor %ymm5,%ymm10,%ymm5 + vpshufb %ymm15,%ymm5,%ymm5 + vpaddd %ymm0,%ymm11,%ymm11 + vpxor %ymm6,%ymm11,%ymm6 + vpshufb %ymm15,%ymm6,%ymm6 + vpaddd %ymm5,%ymm12,%ymm12 + vpxor %ymm3,%ymm12,%ymm3 + vpslld $12,%ymm3,%ymm14 + vpsrld $20,%ymm3,%ymm3 + vpor %ymm3,%ymm14,%ymm3 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm6,%ymm13,%ymm13 + vpxor %ymm0,%ymm13,%ymm0 + vpslld $12,%ymm0,%ymm15 + vpsrld $20,%ymm0,%ymm0 + vpor %ymm0,%ymm15,%ymm0 + vpaddd %ymm3,%ymm10,%ymm10 + vpxor %ymm5,%ymm10,%ymm5 + vpshufb %ymm14,%ymm5,%ymm5 + vpaddd %ymm0,%ymm11,%ymm11 + vpxor %ymm6,%ymm11,%ymm6 + vpshufb %ymm14,%ymm6,%ymm6 + vpaddd %ymm5,%ymm12,%ymm12 + vpxor %ymm3,%ymm12,%ymm3 + vpslld $7,%ymm3,%ymm15 + vpsrld $25,%ymm3,%ymm3 + vpor %ymm3,%ymm15,%ymm3 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm6,%ymm13,%ymm13 + vpxor %ymm0,%ymm13,%ymm0 + vpslld $7,%ymm0,%ymm14 + vpsrld $25,%ymm0,%ymm0 + vpor %ymm0,%ymm14,%ymm0 + decl %eax + jnz .Loop8x + + leaq 512(%rsp),%rax + vpaddd 128-256(%rcx),%ymm8,%ymm8 + vpaddd 160-256(%rcx),%ymm9,%ymm9 + vpaddd 192-256(%rcx),%ymm10,%ymm10 + vpaddd 224-256(%rcx),%ymm11,%ymm11 + + vpunpckldq %ymm9,%ymm8,%ymm14 + vpunpckldq %ymm11,%ymm10,%ymm15 + vpunpckhdq %ymm9,%ymm8,%ymm8 + vpunpckhdq %ymm11,%ymm10,%ymm10 + vpunpcklqdq %ymm15,%ymm14,%ymm9 + vpunpckhqdq %ymm15,%ymm14,%ymm14 + vpunpcklqdq %ymm10,%ymm8,%ymm11 + vpunpckhqdq %ymm10,%ymm8,%ymm8 + vpaddd 256-256(%rcx),%ymm0,%ymm0 + vpaddd 288-256(%rcx),%ymm1,%ymm1 + vpaddd 320-256(%rcx),%ymm2,%ymm2 + vpaddd 352-256(%rcx),%ymm3,%ymm3 + + vpunpckldq %ymm1,%ymm0,%ymm10 + vpunpckldq %ymm3,%ymm2,%ymm15 + vpunpckhdq %ymm1,%ymm0,%ymm0 + vpunpckhdq %ymm3,%ymm2,%ymm2 + vpunpcklqdq %ymm15,%ymm10,%ymm1 + vpunpckhqdq %ymm15,%ymm10,%ymm10 + vpunpcklqdq %ymm2,%ymm0,%ymm3 + vpunpckhqdq %ymm2,%ymm0,%ymm0 + vperm2i128 $0x20,%ymm1,%ymm9,%ymm15 + vperm2i128 $0x31,%ymm1,%ymm9,%ymm1 + vperm2i128 $0x20,%ymm10,%ymm14,%ymm9 + vperm2i128 $0x31,%ymm10,%ymm14,%ymm10 + vperm2i128 $0x20,%ymm3,%ymm11,%ymm14 + vperm2i128 $0x31,%ymm3,%ymm11,%ymm3 + vperm2i128 $0x20,%ymm0,%ymm8,%ymm11 + vperm2i128 $0x31,%ymm0,%ymm8,%ymm0 + vmovdqa %ymm15,0(%rsp) + vmovdqa %ymm9,32(%rsp) + vmovdqa 64(%rsp),%ymm15 + vmovdqa 96(%rsp),%ymm9 + + vpaddd 384-512(%rax),%ymm12,%ymm12 + vpaddd 416-512(%rax),%ymm13,%ymm13 + vpaddd 448-512(%rax),%ymm15,%ymm15 + vpaddd 480-512(%rax),%ymm9,%ymm9 + + vpunpckldq %ymm13,%ymm12,%ymm2 + vpunpckldq %ymm9,%ymm15,%ymm8 + vpunpckhdq %ymm13,%ymm12,%ymm12 + vpunpckhdq %ymm9,%ymm15,%ymm15 + vpunpcklqdq %ymm8,%ymm2,%ymm13 + vpunpckhqdq %ymm8,%ymm2,%ymm2 + vpunpcklqdq %ymm15,%ymm12,%ymm9 + vpunpckhqdq %ymm15,%ymm12,%ymm12 + vpaddd 512-512(%rax),%ymm4,%ymm4 + vpaddd 544-512(%rax),%ymm5,%ymm5 + vpaddd 576-512(%rax),%ymm6,%ymm6 + vpaddd 608-512(%rax),%ymm7,%ymm7 + + vpunpckldq %ymm5,%ymm4,%ymm15 + vpunpckldq %ymm7,%ymm6,%ymm8 + vpunpckhdq %ymm5,%ymm4,%ymm4 + vpunpckhdq %ymm7,%ymm6,%ymm6 + vpunpcklqdq %ymm8,%ymm15,%ymm5 + vpunpckhqdq %ymm8,%ymm15,%ymm15 + vpunpcklqdq %ymm6,%ymm4,%ymm7 + vpunpckhqdq %ymm6,%ymm4,%ymm4 + vperm2i128 $0x20,%ymm5,%ymm13,%ymm8 + vperm2i128 $0x31,%ymm5,%ymm13,%ymm5 + vperm2i128 $0x20,%ymm15,%ymm2,%ymm13 + vperm2i128 $0x31,%ymm15,%ymm2,%ymm15 + vperm2i128 $0x20,%ymm7,%ymm9,%ymm2 + vperm2i128 $0x31,%ymm7,%ymm9,%ymm7 + vperm2i128 $0x20,%ymm4,%ymm12,%ymm9 + vperm2i128 $0x31,%ymm4,%ymm12,%ymm4 + vmovdqa 0(%rsp),%ymm6 + vmovdqa 32(%rsp),%ymm12 + + cmpq $512,%rdx + jb .Ltail8x + + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + leaq 128(%rsi),%rsi + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm12,%ymm12 + vpxor 32(%rsi),%ymm13,%ymm13 + vpxor 64(%rsi),%ymm10,%ymm10 + vpxor 96(%rsi),%ymm15,%ymm15 + leaq 128(%rsi),%rsi + vmovdqu %ymm12,0(%rdi) + vmovdqu %ymm13,32(%rdi) + vmovdqu %ymm10,64(%rdi) + vmovdqu %ymm15,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm14,%ymm14 + vpxor 32(%rsi),%ymm2,%ymm2 + vpxor 64(%rsi),%ymm3,%ymm3 + vpxor 96(%rsi),%ymm7,%ymm7 + leaq 128(%rsi),%rsi + vmovdqu %ymm14,0(%rdi) + vmovdqu %ymm2,32(%rdi) + vmovdqu %ymm3,64(%rdi) + vmovdqu %ymm7,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm11,%ymm11 + vpxor 32(%rsi),%ymm9,%ymm9 + vpxor 64(%rsi),%ymm0,%ymm0 + vpxor 96(%rsi),%ymm4,%ymm4 + leaq 128(%rsi),%rsi + vmovdqu %ymm11,0(%rdi) + vmovdqu %ymm9,32(%rdi) + vmovdqu %ymm0,64(%rdi) + vmovdqu %ymm4,96(%rdi) + leaq 128(%rdi),%rdi + + subq $512,%rdx + jnz .Loop_outer8x + + jmp .Ldone8x + +.Ltail8x: + cmpq $448,%rdx + jae .L448_or_more8x + cmpq $384,%rdx + jae .L384_or_more8x + cmpq $320,%rdx + jae .L320_or_more8x + cmpq $256,%rdx + jae .L256_or_more8x + cmpq $192,%rdx + jae .L192_or_more8x + cmpq $128,%rdx + jae .L128_or_more8x + cmpq $64,%rdx + jae .L64_or_more8x + + xorq %r9,%r9 + vmovdqa %ymm6,0(%rsp) + vmovdqa %ymm8,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L64_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + je .Ldone8x + + leaq 64(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm1,0(%rsp) + leaq 64(%rdi),%rdi + subq $64,%rdx + vmovdqa %ymm5,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L128_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + je .Ldone8x + + leaq 128(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm12,0(%rsp) + leaq 128(%rdi),%rdi + subq $128,%rdx + vmovdqa %ymm13,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L192_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + je .Ldone8x + + leaq 192(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm10,0(%rsp) + leaq 192(%rdi),%rdi + subq $192,%rdx + vmovdqa %ymm15,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L256_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + je .Ldone8x + + leaq 256(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm14,0(%rsp) + leaq 256(%rdi),%rdi + subq $256,%rdx + vmovdqa %ymm2,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L320_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + je .Ldone8x + + leaq 320(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm3,0(%rsp) + leaq 320(%rdi),%rdi + subq $320,%rdx + vmovdqa %ymm7,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L384_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vpxor 320(%rsi),%ymm3,%ymm3 + vpxor 352(%rsi),%ymm7,%ymm7 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + vmovdqu %ymm3,320(%rdi) + vmovdqu %ymm7,352(%rdi) + je .Ldone8x + + leaq 384(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm11,0(%rsp) + leaq 384(%rdi),%rdi + subq $384,%rdx + vmovdqa %ymm9,32(%rsp) + jmp .Loop_tail8x + +.align 32 +.L448_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vpxor 320(%rsi),%ymm3,%ymm3 + vpxor 352(%rsi),%ymm7,%ymm7 + vpxor 384(%rsi),%ymm11,%ymm11 + vpxor 416(%rsi),%ymm9,%ymm9 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + vmovdqu %ymm3,320(%rdi) + vmovdqu %ymm7,352(%rdi) + vmovdqu %ymm11,384(%rdi) + vmovdqu %ymm9,416(%rdi) + je .Ldone8x + + leaq 448(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm0,0(%rsp) + leaq 448(%rdi),%rdi + subq $448,%rdx + vmovdqa %ymm4,32(%rsp) + +.Loop_tail8x: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz .Loop_tail8x + +.Ldone8x: + vzeroall + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +.Lavx2_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_chacha20_asm_avx2,.-crypton_chacha20_asm_avx2 + +.section .note.gnu.property,"a",@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align 8 +2: + +.section .note.GNU-stack,"",@progbits diff --git a/cbits/asm/chacha-x86_64-macosx.S b/cbits/asm/chacha-x86_64-macosx.S new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-x86_64-macosx.S @@ -0,0 +1,2232 @@ +.text + + + +.p2align 6 +L$zero: +.long 0,0,0,0 +L$one: +.long 1,0,0,0 +L$inc: +.long 0,1,2,3 +L$four: +.long 4,4,4,4 +L$incy: +.long 0,2,4,6,1,3,5,7 +L$eight: +.long 8,8,8,8,8,8,8,8 +L$rot16: +.byte 0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd +L$rot24: +.byte 0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe +L$twoy: +.long 2,0,0,0, 2,0,0,0 +.p2align 6 +L$zeroz: +.long 0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0 +L$fourz: +.long 4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0 +L$incz: +.long 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 +L$sixteen: +.long 16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16 +L$sigma: +.byte 101,120,112,97,110,100,32,51,50,45,98,121,116,101,32,107,0 +.byte 67,104,97,67,104,97,50,48,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.globl _crypton_chacha20_asm_ctr32 + +.p2align 6 +_crypton_chacha20_asm_ctr32: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + cmpq $0,%rdx + je L$no_data + movq _crypton_ia32cap_P+4(%rip),%r9 + testl $512,%r9d + jnz L$crypton_chacha20_asm_ssse3 + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + subq $64+24,%rsp +.cfi_adjust_cfa_offset 88 +L$ctr32_body: + + movq %rdx,%rbp + + movq 0(%rcx),%r12 + movq 8(%rcx),%r13 + movq 16(%rcx),%r14 + movq 24(%rcx),%r15 + movq 0(%r8),%rax + movq 8(%r8),%rdx + movq %r12,16(%rsp) + movq %r13,24(%rsp) + movq %r14,0(%rsp) + movq %r15,8(%rsp) + movq %rax,48(%rsp) + movq %rdx,56(%rsp) + jmp L$oop_outer + +.p2align 5 +L$oop_outer: + movl $0x61707865,%eax + movl $0x3320646e,%ebx + movl $0x79622d32,%ecx + movl $0x6b206574,%edx + movl 16(%rsp),%r8d + movl 20(%rsp),%r9d + movl 24(%rsp),%r10d + movl 28(%rsp),%r11d + movl 48(%rsp),%r12d + movl 52(%rsp),%r13d + movl 56(%rsp),%r14d + movq %r15,40(%rsp) + movl 60(%rsp),%r15d + + movq %rbp,64+0(%rsp) + movq %rsi,64+8(%rsp) + movl 0(%rsp),%esi + movq %rdi,64+16(%rsp) + movl 4(%rsp),%edi + movl $10,%ebp + jmp L$oop + +.p2align 5 +L$oop: + addl %r8d,%eax + xorl %eax,%r12d + roll $16,%r12d + addl %r9d,%ebx + xorl %ebx,%r13d + roll $16,%r13d + addl %r12d,%esi + xorl %esi,%r8d + roll $12,%r8d + addl %r13d,%edi + xorl %edi,%r9d + roll $12,%r9d + addl %r8d,%eax + xorl %eax,%r12d + roll $8,%r12d + addl %r9d,%ebx + xorl %ebx,%r13d + roll $8,%r13d + addl %r12d,%esi + xorl %esi,%r8d + roll $7,%r8d + addl %r13d,%edi + xorl %edi,%r9d + roll $7,%r9d + movl %esi,32(%rsp) + movl %edi,36(%rsp) + movl 40(%rsp),%esi + movl 44(%rsp),%edi + addl %r10d,%ecx + xorl %ecx,%r14d + roll $16,%r14d + addl %r11d,%edx + xorl %edx,%r15d + roll $16,%r15d + addl %r14d,%esi + xorl %esi,%r10d + roll $12,%r10d + addl %r15d,%edi + xorl %edi,%r11d + roll $12,%r11d + addl %r10d,%ecx + xorl %ecx,%r14d + roll $8,%r14d + addl %r11d,%edx + xorl %edx,%r15d + roll $8,%r15d + addl %r14d,%esi + xorl %esi,%r10d + roll $7,%r10d + addl %r15d,%edi + xorl %edi,%r11d + roll $7,%r11d + addl %r9d,%eax + xorl %eax,%r15d + roll $16,%r15d + addl %r10d,%ebx + xorl %ebx,%r12d + roll $16,%r12d + addl %r15d,%esi + xorl %esi,%r9d + roll $12,%r9d + addl %r12d,%edi + xorl %edi,%r10d + roll $12,%r10d + addl %r9d,%eax + xorl %eax,%r15d + roll $8,%r15d + addl %r10d,%ebx + xorl %ebx,%r12d + roll $8,%r12d + addl %r15d,%esi + xorl %esi,%r9d + roll $7,%r9d + addl %r12d,%edi + xorl %edi,%r10d + roll $7,%r10d + movl %esi,40(%rsp) + movl %edi,44(%rsp) + movl 32(%rsp),%esi + movl 36(%rsp),%edi + addl %r11d,%ecx + xorl %ecx,%r13d + roll $16,%r13d + addl %r8d,%edx + xorl %edx,%r14d + roll $16,%r14d + addl %r13d,%esi + xorl %esi,%r11d + roll $12,%r11d + addl %r14d,%edi + xorl %edi,%r8d + roll $12,%r8d + addl %r11d,%ecx + xorl %ecx,%r13d + roll $8,%r13d + addl %r8d,%edx + xorl %edx,%r14d + roll $8,%r14d + addl %r13d,%esi + xorl %esi,%r11d + roll $7,%r11d + addl %r14d,%edi + xorl %edi,%r8d + roll $7,%r8d + decl %ebp + jnz L$oop + addl 0(%rsp),%esi + addl 4(%rsp),%edi + movq 64(%rsp),%rbp + movl %esi,32(%rsp) + movq 64+8(%rsp),%rsi + movl %edi,36(%rsp) + movq 64+16(%rsp),%rdi + + addl $0x61707865,%eax + addl $0x3320646e,%ebx + addl $0x79622d32,%ecx + addl $0x6b206574,%edx + addl 16(%rsp),%r8d + addl 20(%rsp),%r9d + addl 24(%rsp),%r10d + addl 28(%rsp),%r11d + addl 48(%rsp),%r12d + addl 52(%rsp),%r13d + addl 56(%rsp),%r14d + addl 60(%rsp),%r15d + + cmpq $64,%rbp + jb L$tail + + xorl 0(%rsi),%eax + xorl 4(%rsi),%ebx + xorl 8(%rsi),%ecx + xorl 12(%rsi),%edx + movl %eax,0(%rdi) + movl 32(%rsp),%eax + movl %ebx,4(%rdi) + movl 36(%rsp),%ebx + movl %ecx,8(%rdi) + movl 40(%rsp),%ecx + movl %edx,12(%rdi) + movl 44(%rsp),%edx + xorl 16(%rsi),%r8d + addl 8(%rsp),%ecx + xorl 20(%rsi),%r9d + addl 12(%rsp),%edx + xorl 24(%rsi),%r10d + xorl 28(%rsi),%r11d + xorl 32(%rsi),%eax + xorl 36(%rsi),%ebx + xorl 40(%rsi),%ecx + xorl 44(%rsi),%edx + xorl 48(%rsi),%r12d + xorl 52(%rsi),%r13d + xorl 56(%rsi),%r14d + xorl 60(%rsi),%r15d + leaq 64(%rsi),%rsi + + addl $1,48(%rsp) + + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + movl %eax,32(%rdi) + movl %ebx,36(%rdi) + movl %ecx,40(%rdi) + movl %edx,44(%rdi) + movl %r12d,48(%rdi) + movl %r13d,52(%rdi) + movl %r14d,56(%rdi) + movl %r15d,60(%rdi) + leaq 64(%rdi),%rdi + movq 8(%rsp),%r15 + + subq $64,%rbp + jnz L$oop_outer + + jmp L$done + +.p2align 4 +L$tail: + movl %eax,0(%rsp) + movl 8(%rsp),%eax + movl %ebx,4(%rsp) + movl 12(%rsp),%ebx + movl %ecx,8(%rsp) + addl 40(%rsp),%eax + movl %edx,12(%rsp) + addl 44(%rsp),%ebx + movl %r8d,16(%rsp) + movl %r9d,20(%rsp) + movl %r10d,24(%rsp) + movl %r11d,28(%rsp) + movl %eax,40(%rsp) + movl %ebx,44(%rsp) + xorq %rbx,%rbx + movl %r12d,48(%rsp) + movl %r13d,52(%rsp) + movl %r14d,56(%rsp) + movl %r15d,60(%rsp) + +L$oop_tail: + movzbl (%rsi,%rbx,1),%eax + movzbl (%rsp,%rbx,1),%edx + leaq 1(%rbx),%rbx + xorl %edx,%eax + movb %al,-1(%rdi,%rbx,1) + decq %rbp + jnz L$oop_tail + +L$done: + leaq 64+24+48(%rsp),%rsi +.cfi_def_cfa %rsi,8 + movq -48(%rsi),%r15 +.cfi_restore %r15 + movq -40(%rsi),%r14 +.cfi_restore %r14 + movq -32(%rsi),%r13 +.cfi_restore %r13 + movq -24(%rsi),%r12 +.cfi_restore %r12 + movq -16(%rsi),%rbp +.cfi_restore %rbp + movq -8(%rsi),%rbx +.cfi_restore %rbx + leaq (%rsi),%rsp +.cfi_def_cfa_register %rsp +L$no_data: + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +crypton_chacha20_asm_ssse3: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +L$crypton_chacha20_asm_ssse3: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + testl $2048,%r9d + jnz L$crypton_chacha20_asm_4xop + cmpq $128,%rdx + je L$crypton_chacha20_asm_128 + ja L$crypton_chacha20_asm_4x + +L$do_sse3_after_all: + subq $64+8,%rsp + andq $-16,%rsp + movdqa L$sigma(%rip),%xmm0 + movdqu (%rcx),%xmm1 + movdqu 16(%rcx),%xmm2 + movdqu (%r8),%xmm3 + movdqa L$rot16(%rip),%xmm6 + movdqa L$rot24(%rip),%xmm7 + + movdqa %xmm0,0(%rsp) + movdqa %xmm1,16(%rsp) + movdqa %xmm2,32(%rsp) + movdqa %xmm3,48(%rsp) + movq $10,%r8 + jmp L$oop_ssse3 + +.p2align 5 +L$oop_outer_ssse3: + movdqa L$one(%rip),%xmm3 + movdqa 0(%rsp),%xmm0 + movdqa 16(%rsp),%xmm1 + movdqa 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + movq $10,%r8 + movdqa %xmm3,48(%rsp) + jmp L$oop_ssse3 + +.p2align 5 +L$oop_ssse3: + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,222 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $20,%xmm1 + pslld $12,%xmm4 + por %xmm4,%xmm1 + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,223 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $25,%xmm1 + pslld $7,%xmm4 + por %xmm4,%xmm1 + pshufd $78,%xmm2,%xmm2 + pshufd $57,%xmm1,%xmm1 + pshufd $147,%xmm3,%xmm3 + nop + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,222 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $20,%xmm1 + pslld $12,%xmm4 + por %xmm4,%xmm1 + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,223 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $25,%xmm1 + pslld $7,%xmm4 + por %xmm4,%xmm1 + pshufd $78,%xmm2,%xmm2 + pshufd $147,%xmm1,%xmm1 + pshufd $57,%xmm3,%xmm3 + decq %r8 + jnz L$oop_ssse3 + paddd 0(%rsp),%xmm0 + paddd 16(%rsp),%xmm1 + paddd 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + + cmpq $64,%rdx + jb L$tail_ssse3 + + movdqu 0(%rsi),%xmm4 + movdqu 16(%rsi),%xmm5 + pxor %xmm4,%xmm0 + movdqu 32(%rsi),%xmm4 + pxor %xmm5,%xmm1 + movdqu 48(%rsi),%xmm5 + leaq 64(%rsi),%rsi + pxor %xmm4,%xmm2 + pxor %xmm5,%xmm3 + + movdqu %xmm0,0(%rdi) + movdqu %xmm1,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm3,48(%rdi) + leaq 64(%rdi),%rdi + + subq $64,%rdx + jnz L$oop_outer_ssse3 + + jmp L$done_ssse3 + +.p2align 4 +L$tail_ssse3: + movdqa %xmm0,0(%rsp) + movdqa %xmm1,16(%rsp) + movdqa %xmm2,32(%rsp) + movdqa %xmm3,48(%rsp) + xorq %r8,%r8 + +L$oop_tail_ssse3: + movzbl (%rsi,%r8,1),%eax + movzbl (%rsp,%r8,1),%ecx + leaq 1(%r8),%r8 + xorl %ecx,%eax + movb %al,-1(%rdi,%r8,1) + decq %rdx + jnz L$oop_tail_ssse3 + +L$done_ssse3: + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +L$ssse3_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +crypton_chacha20_asm_128: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +L$crypton_chacha20_asm_128: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + subq $64+8,%rsp + andq $-16,%rsp + movdqa L$sigma(%rip),%xmm8 + movdqu (%rcx),%xmm9 + movdqu 16(%rcx),%xmm2 + movdqu (%r8),%xmm3 + movdqa L$one(%rip),%xmm1 + movdqa L$rot16(%rip),%xmm6 + movdqa L$rot24(%rip),%xmm7 + + movdqa %xmm8,%xmm10 + movdqa %xmm8,0(%rsp) + movdqa %xmm9,%xmm11 + movdqa %xmm9,16(%rsp) + movdqa %xmm2,%xmm0 + movdqa %xmm2,32(%rsp) + paddd %xmm3,%xmm1 + movdqa %xmm3,48(%rsp) + movq $10,%r8 + jmp L$oop_128 + +.p2align 5 +L$oop_128: + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,222 +.byte 102,15,56,0,206 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $20,%xmm9 + movdqa %xmm11,%xmm5 + pslld $12,%xmm4 + psrld $20,%xmm11 + por %xmm4,%xmm9 + pslld $12,%xmm5 + por %xmm5,%xmm11 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,223 +.byte 102,15,56,0,207 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $25,%xmm9 + movdqa %xmm11,%xmm5 + pslld $7,%xmm4 + psrld $25,%xmm11 + por %xmm4,%xmm9 + pslld $7,%xmm5 + por %xmm5,%xmm11 + pshufd $78,%xmm2,%xmm2 + pshufd $57,%xmm9,%xmm9 + pshufd $147,%xmm3,%xmm3 + pshufd $78,%xmm0,%xmm0 + pshufd $57,%xmm11,%xmm11 + pshufd $147,%xmm1,%xmm1 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,222 +.byte 102,15,56,0,206 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $20,%xmm9 + movdqa %xmm11,%xmm5 + pslld $12,%xmm4 + psrld $20,%xmm11 + por %xmm4,%xmm9 + pslld $12,%xmm5 + por %xmm5,%xmm11 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,223 +.byte 102,15,56,0,207 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $25,%xmm9 + movdqa %xmm11,%xmm5 + pslld $7,%xmm4 + psrld $25,%xmm11 + por %xmm4,%xmm9 + pslld $7,%xmm5 + por %xmm5,%xmm11 + pshufd $78,%xmm2,%xmm2 + pshufd $147,%xmm9,%xmm9 + pshufd $57,%xmm3,%xmm3 + pshufd $78,%xmm0,%xmm0 + pshufd $147,%xmm11,%xmm11 + pshufd $57,%xmm1,%xmm1 + decq %r8 + jnz L$oop_128 + paddd 0(%rsp),%xmm8 + paddd 16(%rsp),%xmm9 + paddd 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + paddd L$one(%rip),%xmm1 + paddd 0(%rsp),%xmm10 + paddd 16(%rsp),%xmm11 + paddd 32(%rsp),%xmm0 + paddd 48(%rsp),%xmm1 + + movdqu 0(%rsi),%xmm4 + movdqu 16(%rsi),%xmm5 + pxor %xmm4,%xmm8 + movdqu 32(%rsi),%xmm4 + pxor %xmm5,%xmm9 + movdqu 48(%rsi),%xmm5 + pxor %xmm4,%xmm2 + movdqu 64(%rsi),%xmm4 + pxor %xmm5,%xmm3 + movdqu 80(%rsi),%xmm5 + pxor %xmm4,%xmm10 + movdqu 96(%rsi),%xmm4 + pxor %xmm5,%xmm11 + movdqu 112(%rsi),%xmm5 + pxor %xmm4,%xmm0 + pxor %xmm5,%xmm1 + + movdqu %xmm8,0(%rdi) + movdqu %xmm9,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm3,48(%rdi) + movdqu %xmm10,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm0,96(%rdi) + movdqu %xmm1,112(%rdi) + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +L$128_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +crypton_chacha20_asm_4x: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +L$crypton_chacha20_asm_4x: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + movq %r9,%r11 + shrq $32,%r9 + testq $32,%r9 + jnz L$crypton_chacha20_asm_8x + cmpq $192,%rdx + ja L$proceed4x + + andq $71303168,%r11 + cmpq $4194304,%r11 + je L$do_sse3_after_all + +L$proceed4x: + subq $0x140+8,%rsp + andq $-16,%rsp + movdqa L$sigma(%rip),%xmm11 + movdqu (%rcx),%xmm15 + movdqu 16(%rcx),%xmm7 + movdqu (%r8),%xmm3 + leaq 256(%rsp),%rcx + leaq L$rot16(%rip),%r9 + leaq L$rot24(%rip),%r11 + + pshufd $0x00,%xmm11,%xmm8 + pshufd $0x55,%xmm11,%xmm9 + movdqa %xmm8,64(%rsp) + pshufd $0xaa,%xmm11,%xmm10 + movdqa %xmm9,80(%rsp) + pshufd $0xff,%xmm11,%xmm11 + movdqa %xmm10,96(%rsp) + movdqa %xmm11,112(%rsp) + + pshufd $0x00,%xmm15,%xmm12 + pshufd $0x55,%xmm15,%xmm13 + movdqa %xmm12,128-256(%rcx) + pshufd $0xaa,%xmm15,%xmm14 + movdqa %xmm13,144-256(%rcx) + pshufd $0xff,%xmm15,%xmm15 + movdqa %xmm14,160-256(%rcx) + movdqa %xmm15,176-256(%rcx) + + pshufd $0x00,%xmm7,%xmm4 + pshufd $0x55,%xmm7,%xmm5 + movdqa %xmm4,192-256(%rcx) + pshufd $0xaa,%xmm7,%xmm6 + movdqa %xmm5,208-256(%rcx) + pshufd $0xff,%xmm7,%xmm7 + movdqa %xmm6,224-256(%rcx) + movdqa %xmm7,240-256(%rcx) + + pshufd $0x00,%xmm3,%xmm0 + pshufd $0x55,%xmm3,%xmm1 + paddd L$inc(%rip),%xmm0 + pshufd $0xaa,%xmm3,%xmm2 + movdqa %xmm1,272-256(%rcx) + pshufd $0xff,%xmm3,%xmm3 + movdqa %xmm2,288-256(%rcx) + movdqa %xmm3,304-256(%rcx) + + jmp L$oop_enter4x + +.p2align 5 +L$oop_outer4x: + movdqa 64(%rsp),%xmm8 + movdqa 80(%rsp),%xmm9 + movdqa 96(%rsp),%xmm10 + movdqa 112(%rsp),%xmm11 + movdqa 128-256(%rcx),%xmm12 + movdqa 144-256(%rcx),%xmm13 + movdqa 160-256(%rcx),%xmm14 + movdqa 176-256(%rcx),%xmm15 + movdqa 192-256(%rcx),%xmm4 + movdqa 208-256(%rcx),%xmm5 + movdqa 224-256(%rcx),%xmm6 + movdqa 240-256(%rcx),%xmm7 + movdqa 256-256(%rcx),%xmm0 + movdqa 272-256(%rcx),%xmm1 + movdqa 288-256(%rcx),%xmm2 + movdqa 304-256(%rcx),%xmm3 + paddd L$four(%rip),%xmm0 + +L$oop_enter4x: + movdqa %xmm6,32(%rsp) + movdqa %xmm7,48(%rsp) + movdqa (%r9),%xmm7 + movl $10,%eax + movdqa %xmm0,256-256(%rcx) + jmp L$oop4x + +.p2align 5 +L$oop4x: + paddd %xmm12,%xmm8 + paddd %xmm13,%xmm9 + pxor %xmm8,%xmm0 + pxor %xmm9,%xmm1 +.byte 102,15,56,0,199 +.byte 102,15,56,0,207 + paddd %xmm0,%xmm4 + paddd %xmm1,%xmm5 + pxor %xmm4,%xmm12 + pxor %xmm5,%xmm13 + movdqa %xmm12,%xmm6 + pslld $12,%xmm12 + psrld $20,%xmm6 + movdqa %xmm13,%xmm7 + pslld $12,%xmm13 + por %xmm6,%xmm12 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm13 + paddd %xmm12,%xmm8 + paddd %xmm13,%xmm9 + pxor %xmm8,%xmm0 + pxor %xmm9,%xmm1 +.byte 102,15,56,0,198 +.byte 102,15,56,0,206 + paddd %xmm0,%xmm4 + paddd %xmm1,%xmm5 + pxor %xmm4,%xmm12 + pxor %xmm5,%xmm13 + movdqa %xmm12,%xmm7 + pslld $7,%xmm12 + psrld $25,%xmm7 + movdqa %xmm13,%xmm6 + pslld $7,%xmm13 + por %xmm7,%xmm12 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm13 + movdqa %xmm4,0(%rsp) + movdqa %xmm5,16(%rsp) + movdqa 32(%rsp),%xmm4 + movdqa 48(%rsp),%xmm5 + paddd %xmm14,%xmm10 + paddd %xmm15,%xmm11 + pxor %xmm10,%xmm2 + pxor %xmm11,%xmm3 +.byte 102,15,56,0,215 +.byte 102,15,56,0,223 + paddd %xmm2,%xmm4 + paddd %xmm3,%xmm5 + pxor %xmm4,%xmm14 + pxor %xmm5,%xmm15 + movdqa %xmm14,%xmm6 + pslld $12,%xmm14 + psrld $20,%xmm6 + movdqa %xmm15,%xmm7 + pslld $12,%xmm15 + por %xmm6,%xmm14 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm15 + paddd %xmm14,%xmm10 + paddd %xmm15,%xmm11 + pxor %xmm10,%xmm2 + pxor %xmm11,%xmm3 +.byte 102,15,56,0,214 +.byte 102,15,56,0,222 + paddd %xmm2,%xmm4 + paddd %xmm3,%xmm5 + pxor %xmm4,%xmm14 + pxor %xmm5,%xmm15 + movdqa %xmm14,%xmm7 + pslld $7,%xmm14 + psrld $25,%xmm7 + movdqa %xmm15,%xmm6 + pslld $7,%xmm15 + por %xmm7,%xmm14 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm15 + paddd %xmm13,%xmm8 + paddd %xmm14,%xmm9 + pxor %xmm8,%xmm3 + pxor %xmm9,%xmm0 +.byte 102,15,56,0,223 +.byte 102,15,56,0,199 + paddd %xmm3,%xmm4 + paddd %xmm0,%xmm5 + pxor %xmm4,%xmm13 + pxor %xmm5,%xmm14 + movdqa %xmm13,%xmm6 + pslld $12,%xmm13 + psrld $20,%xmm6 + movdqa %xmm14,%xmm7 + pslld $12,%xmm14 + por %xmm6,%xmm13 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm14 + paddd %xmm13,%xmm8 + paddd %xmm14,%xmm9 + pxor %xmm8,%xmm3 + pxor %xmm9,%xmm0 +.byte 102,15,56,0,222 +.byte 102,15,56,0,198 + paddd %xmm3,%xmm4 + paddd %xmm0,%xmm5 + pxor %xmm4,%xmm13 + pxor %xmm5,%xmm14 + movdqa %xmm13,%xmm7 + pslld $7,%xmm13 + psrld $25,%xmm7 + movdqa %xmm14,%xmm6 + pslld $7,%xmm14 + por %xmm7,%xmm13 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm14 + movdqa %xmm4,32(%rsp) + movdqa %xmm5,48(%rsp) + movdqa 0(%rsp),%xmm4 + movdqa 16(%rsp),%xmm5 + paddd %xmm15,%xmm10 + paddd %xmm12,%xmm11 + pxor %xmm10,%xmm1 + pxor %xmm11,%xmm2 +.byte 102,15,56,0,207 +.byte 102,15,56,0,215 + paddd %xmm1,%xmm4 + paddd %xmm2,%xmm5 + pxor %xmm4,%xmm15 + pxor %xmm5,%xmm12 + movdqa %xmm15,%xmm6 + pslld $12,%xmm15 + psrld $20,%xmm6 + movdqa %xmm12,%xmm7 + pslld $12,%xmm12 + por %xmm6,%xmm15 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm12 + paddd %xmm15,%xmm10 + paddd %xmm12,%xmm11 + pxor %xmm10,%xmm1 + pxor %xmm11,%xmm2 +.byte 102,15,56,0,206 +.byte 102,15,56,0,214 + paddd %xmm1,%xmm4 + paddd %xmm2,%xmm5 + pxor %xmm4,%xmm15 + pxor %xmm5,%xmm12 + movdqa %xmm15,%xmm7 + pslld $7,%xmm15 + psrld $25,%xmm7 + movdqa %xmm12,%xmm6 + pslld $7,%xmm12 + por %xmm7,%xmm15 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm12 + decl %eax + jnz L$oop4x + + paddd 64(%rsp),%xmm8 + paddd 80(%rsp),%xmm9 + paddd 96(%rsp),%xmm10 + paddd 112(%rsp),%xmm11 + + movdqa %xmm8,%xmm6 + punpckldq %xmm9,%xmm8 + movdqa %xmm10,%xmm7 + punpckldq %xmm11,%xmm10 + punpckhdq %xmm9,%xmm6 + punpckhdq %xmm11,%xmm7 + movdqa %xmm8,%xmm9 + punpcklqdq %xmm10,%xmm8 + movdqa %xmm6,%xmm11 + punpcklqdq %xmm7,%xmm6 + punpckhqdq %xmm10,%xmm9 + punpckhqdq %xmm7,%xmm11 + paddd 128-256(%rcx),%xmm12 + paddd 144-256(%rcx),%xmm13 + paddd 160-256(%rcx),%xmm14 + paddd 176-256(%rcx),%xmm15 + + movdqa %xmm8,0(%rsp) + movdqa %xmm9,16(%rsp) + movdqa 32(%rsp),%xmm8 + movdqa 48(%rsp),%xmm9 + + movdqa %xmm12,%xmm10 + punpckldq %xmm13,%xmm12 + movdqa %xmm14,%xmm7 + punpckldq %xmm15,%xmm14 + punpckhdq %xmm13,%xmm10 + punpckhdq %xmm15,%xmm7 + movdqa %xmm12,%xmm13 + punpcklqdq %xmm14,%xmm12 + movdqa %xmm10,%xmm15 + punpcklqdq %xmm7,%xmm10 + punpckhqdq %xmm14,%xmm13 + punpckhqdq %xmm7,%xmm15 + paddd 192-256(%rcx),%xmm4 + paddd 208-256(%rcx),%xmm5 + paddd 224-256(%rcx),%xmm8 + paddd 240-256(%rcx),%xmm9 + + movdqa %xmm6,32(%rsp) + movdqa %xmm11,48(%rsp) + + movdqa %xmm4,%xmm14 + punpckldq %xmm5,%xmm4 + movdqa %xmm8,%xmm7 + punpckldq %xmm9,%xmm8 + punpckhdq %xmm5,%xmm14 + punpckhdq %xmm9,%xmm7 + movdqa %xmm4,%xmm5 + punpcklqdq %xmm8,%xmm4 + movdqa %xmm14,%xmm9 + punpcklqdq %xmm7,%xmm14 + punpckhqdq %xmm8,%xmm5 + punpckhqdq %xmm7,%xmm9 + paddd 256-256(%rcx),%xmm0 + paddd 272-256(%rcx),%xmm1 + paddd 288-256(%rcx),%xmm2 + paddd 304-256(%rcx),%xmm3 + + movdqa %xmm0,%xmm8 + punpckldq %xmm1,%xmm0 + movdqa %xmm2,%xmm7 + punpckldq %xmm3,%xmm2 + punpckhdq %xmm1,%xmm8 + punpckhdq %xmm3,%xmm7 + movdqa %xmm0,%xmm1 + punpcklqdq %xmm2,%xmm0 + movdqa %xmm8,%xmm3 + punpcklqdq %xmm7,%xmm8 + punpckhqdq %xmm2,%xmm1 + punpckhqdq %xmm7,%xmm3 + cmpq $256,%rdx + jb L$tail4x + + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + + movdqu %xmm6,64(%rdi) + movdqu 0(%rsi),%xmm6 + movdqu %xmm11,80(%rdi) + movdqu 16(%rsi),%xmm11 + movdqu %xmm2,96(%rdi) + movdqu 32(%rsi),%xmm2 + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + movdqu 48(%rsi),%xmm7 + pxor 32(%rsp),%xmm6 + pxor %xmm10,%xmm11 + pxor %xmm14,%xmm2 + pxor %xmm8,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 48(%rsp),%xmm6 + pxor %xmm15,%xmm11 + pxor %xmm9,%xmm2 + pxor %xmm3,%xmm7 + movdqu %xmm6,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm2,96(%rdi) + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + + subq $256,%rdx + jnz L$oop_outer4x + + jmp L$done4x + +L$tail4x: + cmpq $192,%rdx + jae L$192_or_more4x + cmpq $128,%rdx + jae L$128_or_more4x + cmpq $64,%rdx + jae L$64_or_more4x + + + xorq %r9,%r9 + + movdqa %xmm12,16(%rsp) + movdqa %xmm4,32(%rsp) + movdqa %xmm0,48(%rsp) + jmp L$oop_tail4x + +.p2align 5 +L$64_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + movdqu %xmm6,0(%rdi) + movdqu %xmm11,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm7,48(%rdi) + je L$done4x + + movdqa 16(%rsp),%xmm6 + leaq 64(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm13,16(%rsp) + leaq 64(%rdi),%rdi + movdqa %xmm5,32(%rsp) + subq $64,%rdx + movdqa %xmm1,48(%rsp) + jmp L$oop_tail4x + +.p2align 5 +L$128_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + movdqu %xmm6,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm2,96(%rdi) + movdqu %xmm7,112(%rdi) + je L$done4x + + movdqa 32(%rsp),%xmm6 + leaq 128(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm10,16(%rsp) + leaq 128(%rdi),%rdi + movdqa %xmm14,32(%rsp) + subq $128,%rdx + movdqa %xmm8,48(%rsp) + jmp L$oop_tail4x + +.p2align 5 +L$192_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + + movdqu %xmm6,64(%rdi) + movdqu 0(%rsi),%xmm6 + movdqu %xmm11,80(%rdi) + movdqu 16(%rsi),%xmm11 + movdqu %xmm2,96(%rdi) + movdqu 32(%rsi),%xmm2 + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + movdqu 48(%rsi),%xmm7 + pxor 32(%rsp),%xmm6 + pxor %xmm10,%xmm11 + pxor %xmm14,%xmm2 + pxor %xmm8,%xmm7 + movdqu %xmm6,0(%rdi) + movdqu %xmm11,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm7,48(%rdi) + je L$done4x + + movdqa 48(%rsp),%xmm6 + leaq 64(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm15,16(%rsp) + leaq 64(%rdi),%rdi + movdqa %xmm9,32(%rsp) + subq $192,%rdx + movdqa %xmm3,48(%rsp) + +L$oop_tail4x: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz L$oop_tail4x + +L$done4x: + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +L$4x_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +crypton_chacha20_asm_4xop: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +L$crypton_chacha20_asm_4xop: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + subq $0x140+8,%rsp + andq $-16,%rsp + vzeroupper + + vmovdqa L$sigma(%rip),%xmm11 + vmovdqu (%rcx),%xmm3 + vmovdqu 16(%rcx),%xmm15 + vmovdqu (%r8),%xmm7 + leaq 256(%rsp),%rcx + + vpshufd $0x00,%xmm11,%xmm8 + vpshufd $0x55,%xmm11,%xmm9 + vmovdqa %xmm8,64(%rsp) + vpshufd $0xaa,%xmm11,%xmm10 + vmovdqa %xmm9,80(%rsp) + vpshufd $0xff,%xmm11,%xmm11 + vmovdqa %xmm10,96(%rsp) + vmovdqa %xmm11,112(%rsp) + + vpshufd $0x00,%xmm3,%xmm0 + vpshufd $0x55,%xmm3,%xmm1 + vmovdqa %xmm0,128-256(%rcx) + vpshufd $0xaa,%xmm3,%xmm2 + vmovdqa %xmm1,144-256(%rcx) + vpshufd $0xff,%xmm3,%xmm3 + vmovdqa %xmm2,160-256(%rcx) + vmovdqa %xmm3,176-256(%rcx) + + vpshufd $0x00,%xmm15,%xmm12 + vpshufd $0x55,%xmm15,%xmm13 + vmovdqa %xmm12,192-256(%rcx) + vpshufd $0xaa,%xmm15,%xmm14 + vmovdqa %xmm13,208-256(%rcx) + vpshufd $0xff,%xmm15,%xmm15 + vmovdqa %xmm14,224-256(%rcx) + vmovdqa %xmm15,240-256(%rcx) + + vpshufd $0x00,%xmm7,%xmm4 + vpshufd $0x55,%xmm7,%xmm5 + vpaddd L$inc(%rip),%xmm4,%xmm4 + vpshufd $0xaa,%xmm7,%xmm6 + vmovdqa %xmm5,272-256(%rcx) + vpshufd $0xff,%xmm7,%xmm7 + vmovdqa %xmm6,288-256(%rcx) + vmovdqa %xmm7,304-256(%rcx) + + jmp L$oop_enter4xop + +.p2align 5 +L$oop_outer4xop: + vmovdqa 64(%rsp),%xmm8 + vmovdqa 80(%rsp),%xmm9 + vmovdqa 96(%rsp),%xmm10 + vmovdqa 112(%rsp),%xmm11 + vmovdqa 128-256(%rcx),%xmm0 + vmovdqa 144-256(%rcx),%xmm1 + vmovdqa 160-256(%rcx),%xmm2 + vmovdqa 176-256(%rcx),%xmm3 + vmovdqa 192-256(%rcx),%xmm12 + vmovdqa 208-256(%rcx),%xmm13 + vmovdqa 224-256(%rcx),%xmm14 + vmovdqa 240-256(%rcx),%xmm15 + vmovdqa 256-256(%rcx),%xmm4 + vmovdqa 272-256(%rcx),%xmm5 + vmovdqa 288-256(%rcx),%xmm6 + vmovdqa 304-256(%rcx),%xmm7 + vpaddd L$four(%rip),%xmm4,%xmm4 + +L$oop_enter4xop: + movl $10,%eax + vmovdqa %xmm4,256-256(%rcx) + jmp L$oop4xop + +.p2align 5 +L$oop4xop: + vpaddd %xmm0,%xmm8,%xmm8 + vpaddd %xmm1,%xmm9,%xmm9 + vpaddd %xmm2,%xmm10,%xmm10 + vpaddd %xmm3,%xmm11,%xmm11 + vpxor %xmm4,%xmm8,%xmm4 + vpxor %xmm5,%xmm9,%xmm5 + vpxor %xmm6,%xmm10,%xmm6 + vpxor %xmm7,%xmm11,%xmm7 +.byte 143,232,120,194,228,16 +.byte 143,232,120,194,237,16 +.byte 143,232,120,194,246,16 +.byte 143,232,120,194,255,16 + vpaddd %xmm4,%xmm12,%xmm12 + vpaddd %xmm5,%xmm13,%xmm13 + vpaddd %xmm6,%xmm14,%xmm14 + vpaddd %xmm7,%xmm15,%xmm15 + vpxor %xmm0,%xmm12,%xmm0 + vpxor %xmm1,%xmm13,%xmm1 + vpxor %xmm14,%xmm2,%xmm2 + vpxor %xmm15,%xmm3,%xmm3 +.byte 143,232,120,194,192,12 +.byte 143,232,120,194,201,12 +.byte 143,232,120,194,210,12 +.byte 143,232,120,194,219,12 + vpaddd %xmm8,%xmm0,%xmm8 + vpaddd %xmm9,%xmm1,%xmm9 + vpaddd %xmm2,%xmm10,%xmm10 + vpaddd %xmm3,%xmm11,%xmm11 + vpxor %xmm4,%xmm8,%xmm4 + vpxor %xmm5,%xmm9,%xmm5 + vpxor %xmm6,%xmm10,%xmm6 + vpxor %xmm7,%xmm11,%xmm7 +.byte 143,232,120,194,228,8 +.byte 143,232,120,194,237,8 +.byte 143,232,120,194,246,8 +.byte 143,232,120,194,255,8 + vpaddd %xmm4,%xmm12,%xmm12 + vpaddd %xmm5,%xmm13,%xmm13 + vpaddd %xmm6,%xmm14,%xmm14 + vpaddd %xmm7,%xmm15,%xmm15 + vpxor %xmm0,%xmm12,%xmm0 + vpxor %xmm1,%xmm13,%xmm1 + vpxor %xmm14,%xmm2,%xmm2 + vpxor %xmm15,%xmm3,%xmm3 +.byte 143,232,120,194,192,7 +.byte 143,232,120,194,201,7 +.byte 143,232,120,194,210,7 +.byte 143,232,120,194,219,7 + vpaddd %xmm1,%xmm8,%xmm8 + vpaddd %xmm2,%xmm9,%xmm9 + vpaddd %xmm3,%xmm10,%xmm10 + vpaddd %xmm0,%xmm11,%xmm11 + vpxor %xmm7,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm4 + vpxor %xmm5,%xmm10,%xmm5 + vpxor %xmm6,%xmm11,%xmm6 +.byte 143,232,120,194,255,16 +.byte 143,232,120,194,228,16 +.byte 143,232,120,194,237,16 +.byte 143,232,120,194,246,16 + vpaddd %xmm7,%xmm14,%xmm14 + vpaddd %xmm4,%xmm15,%xmm15 + vpaddd %xmm5,%xmm12,%xmm12 + vpaddd %xmm6,%xmm13,%xmm13 + vpxor %xmm1,%xmm14,%xmm1 + vpxor %xmm2,%xmm15,%xmm2 + vpxor %xmm12,%xmm3,%xmm3 + vpxor %xmm13,%xmm0,%xmm0 +.byte 143,232,120,194,201,12 +.byte 143,232,120,194,210,12 +.byte 143,232,120,194,219,12 +.byte 143,232,120,194,192,12 + vpaddd %xmm8,%xmm1,%xmm8 + vpaddd %xmm9,%xmm2,%xmm9 + vpaddd %xmm3,%xmm10,%xmm10 + vpaddd %xmm0,%xmm11,%xmm11 + vpxor %xmm7,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm4 + vpxor %xmm5,%xmm10,%xmm5 + vpxor %xmm6,%xmm11,%xmm6 +.byte 143,232,120,194,255,8 +.byte 143,232,120,194,228,8 +.byte 143,232,120,194,237,8 +.byte 143,232,120,194,246,8 + vpaddd %xmm7,%xmm14,%xmm14 + vpaddd %xmm4,%xmm15,%xmm15 + vpaddd %xmm5,%xmm12,%xmm12 + vpaddd %xmm6,%xmm13,%xmm13 + vpxor %xmm1,%xmm14,%xmm1 + vpxor %xmm2,%xmm15,%xmm2 + vpxor %xmm12,%xmm3,%xmm3 + vpxor %xmm13,%xmm0,%xmm0 +.byte 143,232,120,194,201,7 +.byte 143,232,120,194,210,7 +.byte 143,232,120,194,219,7 +.byte 143,232,120,194,192,7 + decl %eax + jnz L$oop4xop + + vpaddd 64(%rsp),%xmm8,%xmm8 + vpaddd 80(%rsp),%xmm9,%xmm9 + vpaddd 96(%rsp),%xmm10,%xmm10 + vpaddd 112(%rsp),%xmm11,%xmm11 + + vmovdqa %xmm14,32(%rsp) + vmovdqa %xmm15,48(%rsp) + + vpunpckldq %xmm9,%xmm8,%xmm14 + vpunpckldq %xmm11,%xmm10,%xmm15 + vpunpckhdq %xmm9,%xmm8,%xmm8 + vpunpckhdq %xmm11,%xmm10,%xmm10 + vpunpcklqdq %xmm15,%xmm14,%xmm9 + vpunpckhqdq %xmm15,%xmm14,%xmm14 + vpunpcklqdq %xmm10,%xmm8,%xmm11 + vpunpckhqdq %xmm10,%xmm8,%xmm8 + vpaddd 128-256(%rcx),%xmm0,%xmm0 + vpaddd 144-256(%rcx),%xmm1,%xmm1 + vpaddd 160-256(%rcx),%xmm2,%xmm2 + vpaddd 176-256(%rcx),%xmm3,%xmm3 + + vmovdqa %xmm9,0(%rsp) + vmovdqa %xmm14,16(%rsp) + vmovdqa 32(%rsp),%xmm9 + vmovdqa 48(%rsp),%xmm14 + + vpunpckldq %xmm1,%xmm0,%xmm10 + vpunpckldq %xmm3,%xmm2,%xmm15 + vpunpckhdq %xmm1,%xmm0,%xmm0 + vpunpckhdq %xmm3,%xmm2,%xmm2 + vpunpcklqdq %xmm15,%xmm10,%xmm1 + vpunpckhqdq %xmm15,%xmm10,%xmm10 + vpunpcklqdq %xmm2,%xmm0,%xmm3 + vpunpckhqdq %xmm2,%xmm0,%xmm0 + vpaddd 192-256(%rcx),%xmm12,%xmm12 + vpaddd 208-256(%rcx),%xmm13,%xmm13 + vpaddd 224-256(%rcx),%xmm9,%xmm9 + vpaddd 240-256(%rcx),%xmm14,%xmm14 + + vpunpckldq %xmm13,%xmm12,%xmm2 + vpunpckldq %xmm14,%xmm9,%xmm15 + vpunpckhdq %xmm13,%xmm12,%xmm12 + vpunpckhdq %xmm14,%xmm9,%xmm9 + vpunpcklqdq %xmm15,%xmm2,%xmm13 + vpunpckhqdq %xmm15,%xmm2,%xmm2 + vpunpcklqdq %xmm9,%xmm12,%xmm14 + vpunpckhqdq %xmm9,%xmm12,%xmm12 + vpaddd 256-256(%rcx),%xmm4,%xmm4 + vpaddd 272-256(%rcx),%xmm5,%xmm5 + vpaddd 288-256(%rcx),%xmm6,%xmm6 + vpaddd 304-256(%rcx),%xmm7,%xmm7 + + vpunpckldq %xmm5,%xmm4,%xmm9 + vpunpckldq %xmm7,%xmm6,%xmm15 + vpunpckhdq %xmm5,%xmm4,%xmm4 + vpunpckhdq %xmm7,%xmm6,%xmm6 + vpunpcklqdq %xmm15,%xmm9,%xmm5 + vpunpckhqdq %xmm15,%xmm9,%xmm9 + vpunpcklqdq %xmm6,%xmm4,%xmm7 + vpunpckhqdq %xmm6,%xmm4,%xmm4 + vmovdqa 0(%rsp),%xmm6 + vmovdqa 16(%rsp),%xmm15 + + cmpq $256,%rdx + jb L$tail4xop + + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + leaq 128(%rsi),%rsi + vpxor 0(%rsi),%xmm11,%xmm11 + vpxor 16(%rsi),%xmm3,%xmm3 + vpxor 32(%rsi),%xmm14,%xmm14 + vpxor 48(%rsi),%xmm7,%xmm7 + vpxor 64(%rsi),%xmm8,%xmm8 + vpxor 80(%rsi),%xmm0,%xmm0 + vpxor 96(%rsi),%xmm12,%xmm12 + vpxor 112(%rsi),%xmm4,%xmm4 + leaq 128(%rsi),%rsi + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + leaq 128(%rdi),%rdi + vmovdqu %xmm11,0(%rdi) + vmovdqu %xmm3,16(%rdi) + vmovdqu %xmm14,32(%rdi) + vmovdqu %xmm7,48(%rdi) + vmovdqu %xmm8,64(%rdi) + vmovdqu %xmm0,80(%rdi) + vmovdqu %xmm12,96(%rdi) + vmovdqu %xmm4,112(%rdi) + leaq 128(%rdi),%rdi + + subq $256,%rdx + jnz L$oop_outer4xop + + jmp L$done4xop + +.p2align 5 +L$tail4xop: + cmpq $192,%rdx + jae L$192_or_more4xop + cmpq $128,%rdx + jae L$128_or_more4xop + cmpq $64,%rdx + jae L$64_or_more4xop + + xorq %r9,%r9 + vmovdqa %xmm6,0(%rsp) + vmovdqa %xmm1,16(%rsp) + vmovdqa %xmm13,32(%rsp) + vmovdqa %xmm5,48(%rsp) + jmp L$oop_tail4xop + +.p2align 5 +L$64_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + je L$done4xop + + leaq 64(%rsi),%rsi + vmovdqa %xmm15,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm10,16(%rsp) + leaq 64(%rdi),%rdi + vmovdqa %xmm2,32(%rsp) + subq $64,%rdx + vmovdqa %xmm9,48(%rsp) + jmp L$oop_tail4xop + +.p2align 5 +L$128_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + je L$done4xop + + leaq 128(%rsi),%rsi + vmovdqa %xmm11,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm3,16(%rsp) + leaq 128(%rdi),%rdi + vmovdqa %xmm14,32(%rsp) + subq $128,%rdx + vmovdqa %xmm7,48(%rsp) + jmp L$oop_tail4xop + +.p2align 5 +L$192_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + leaq 128(%rsi),%rsi + vpxor 0(%rsi),%xmm11,%xmm11 + vpxor 16(%rsi),%xmm3,%xmm3 + vpxor 32(%rsi),%xmm14,%xmm14 + vpxor 48(%rsi),%xmm7,%xmm7 + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + leaq 128(%rdi),%rdi + vmovdqu %xmm11,0(%rdi) + vmovdqu %xmm3,16(%rdi) + vmovdqu %xmm14,32(%rdi) + vmovdqu %xmm7,48(%rdi) + je L$done4xop + + leaq 64(%rsi),%rsi + vmovdqa %xmm8,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm0,16(%rsp) + leaq 64(%rdi),%rdi + vmovdqa %xmm12,32(%rsp) + subq $192,%rdx + vmovdqa %xmm4,48(%rsp) + +L$oop_tail4xop: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz L$oop_tail4xop + +L$done4xop: + vzeroupper + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +L$4xop_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +crypton_chacha20_asm_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +L$crypton_chacha20_asm_8x: + movq %rsp,%r10 +.cfi_def_cfa_register %r10 + subq $0x280+8,%rsp + andq $-32,%rsp + vzeroupper + + + + + + + + + + + vbroadcasti128 L$sigma(%rip),%ymm11 + vbroadcasti128 (%rcx),%ymm3 + vbroadcasti128 16(%rcx),%ymm15 + vbroadcasti128 (%r8),%ymm7 + leaq 256(%rsp),%rcx + leaq 512(%rsp),%rax + leaq L$rot16(%rip),%r9 + leaq L$rot24(%rip),%r11 + + vpshufd $0x00,%ymm11,%ymm8 + vpshufd $0x55,%ymm11,%ymm9 + vmovdqa %ymm8,128-256(%rcx) + vpshufd $0xaa,%ymm11,%ymm10 + vmovdqa %ymm9,160-256(%rcx) + vpshufd $0xff,%ymm11,%ymm11 + vmovdqa %ymm10,192-256(%rcx) + vmovdqa %ymm11,224-256(%rcx) + + vpshufd $0x00,%ymm3,%ymm0 + vpshufd $0x55,%ymm3,%ymm1 + vmovdqa %ymm0,256-256(%rcx) + vpshufd $0xaa,%ymm3,%ymm2 + vmovdqa %ymm1,288-256(%rcx) + vpshufd $0xff,%ymm3,%ymm3 + vmovdqa %ymm2,320-256(%rcx) + vmovdqa %ymm3,352-256(%rcx) + + vpshufd $0x00,%ymm15,%ymm12 + vpshufd $0x55,%ymm15,%ymm13 + vmovdqa %ymm12,384-512(%rax) + vpshufd $0xaa,%ymm15,%ymm14 + vmovdqa %ymm13,416-512(%rax) + vpshufd $0xff,%ymm15,%ymm15 + vmovdqa %ymm14,448-512(%rax) + vmovdqa %ymm15,480-512(%rax) + + vpshufd $0x00,%ymm7,%ymm4 + vpshufd $0x55,%ymm7,%ymm5 + vpaddd L$incy(%rip),%ymm4,%ymm4 + vpshufd $0xaa,%ymm7,%ymm6 + vmovdqa %ymm5,544-512(%rax) + vpshufd $0xff,%ymm7,%ymm7 + vmovdqa %ymm6,576-512(%rax) + vmovdqa %ymm7,608-512(%rax) + + jmp L$oop_enter8x + +.p2align 5 +L$oop_outer8x: + vmovdqa 128-256(%rcx),%ymm8 + vmovdqa 160-256(%rcx),%ymm9 + vmovdqa 192-256(%rcx),%ymm10 + vmovdqa 224-256(%rcx),%ymm11 + vmovdqa 256-256(%rcx),%ymm0 + vmovdqa 288-256(%rcx),%ymm1 + vmovdqa 320-256(%rcx),%ymm2 + vmovdqa 352-256(%rcx),%ymm3 + vmovdqa 384-512(%rax),%ymm12 + vmovdqa 416-512(%rax),%ymm13 + vmovdqa 448-512(%rax),%ymm14 + vmovdqa 480-512(%rax),%ymm15 + vmovdqa 512-512(%rax),%ymm4 + vmovdqa 544-512(%rax),%ymm5 + vmovdqa 576-512(%rax),%ymm6 + vmovdqa 608-512(%rax),%ymm7 + vpaddd L$eight(%rip),%ymm4,%ymm4 + +L$oop_enter8x: + vmovdqa %ymm14,64(%rsp) + vmovdqa %ymm15,96(%rsp) + vbroadcasti128 (%r9),%ymm15 + vmovdqa %ymm4,512-512(%rax) + movl $10,%eax + jmp L$oop8x + +.p2align 5 +L$oop8x: + vpaddd %ymm0,%ymm8,%ymm8 + vpxor %ymm4,%ymm8,%ymm4 + vpshufb %ymm15,%ymm4,%ymm4 + vpaddd %ymm1,%ymm9,%ymm9 + vpxor %ymm5,%ymm9,%ymm5 + vpshufb %ymm15,%ymm5,%ymm5 + vpaddd %ymm4,%ymm12,%ymm12 + vpxor %ymm0,%ymm12,%ymm0 + vpslld $12,%ymm0,%ymm14 + vpsrld $20,%ymm0,%ymm0 + vpor %ymm0,%ymm14,%ymm0 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm5,%ymm13,%ymm13 + vpxor %ymm1,%ymm13,%ymm1 + vpslld $12,%ymm1,%ymm15 + vpsrld $20,%ymm1,%ymm1 + vpor %ymm1,%ymm15,%ymm1 + vpaddd %ymm0,%ymm8,%ymm8 + vpxor %ymm4,%ymm8,%ymm4 + vpshufb %ymm14,%ymm4,%ymm4 + vpaddd %ymm1,%ymm9,%ymm9 + vpxor %ymm5,%ymm9,%ymm5 + vpshufb %ymm14,%ymm5,%ymm5 + vpaddd %ymm4,%ymm12,%ymm12 + vpxor %ymm0,%ymm12,%ymm0 + vpslld $7,%ymm0,%ymm15 + vpsrld $25,%ymm0,%ymm0 + vpor %ymm0,%ymm15,%ymm0 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm5,%ymm13,%ymm13 + vpxor %ymm1,%ymm13,%ymm1 + vpslld $7,%ymm1,%ymm14 + vpsrld $25,%ymm1,%ymm1 + vpor %ymm1,%ymm14,%ymm1 + vmovdqa %ymm12,0(%rsp) + vmovdqa %ymm13,32(%rsp) + vmovdqa 64(%rsp),%ymm12 + vmovdqa 96(%rsp),%ymm13 + vpaddd %ymm2,%ymm10,%ymm10 + vpxor %ymm6,%ymm10,%ymm6 + vpshufb %ymm15,%ymm6,%ymm6 + vpaddd %ymm3,%ymm11,%ymm11 + vpxor %ymm7,%ymm11,%ymm7 + vpshufb %ymm15,%ymm7,%ymm7 + vpaddd %ymm6,%ymm12,%ymm12 + vpxor %ymm2,%ymm12,%ymm2 + vpslld $12,%ymm2,%ymm14 + vpsrld $20,%ymm2,%ymm2 + vpor %ymm2,%ymm14,%ymm2 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm7,%ymm13,%ymm13 + vpxor %ymm3,%ymm13,%ymm3 + vpslld $12,%ymm3,%ymm15 + vpsrld $20,%ymm3,%ymm3 + vpor %ymm3,%ymm15,%ymm3 + vpaddd %ymm2,%ymm10,%ymm10 + vpxor %ymm6,%ymm10,%ymm6 + vpshufb %ymm14,%ymm6,%ymm6 + vpaddd %ymm3,%ymm11,%ymm11 + vpxor %ymm7,%ymm11,%ymm7 + vpshufb %ymm14,%ymm7,%ymm7 + vpaddd %ymm6,%ymm12,%ymm12 + vpxor %ymm2,%ymm12,%ymm2 + vpslld $7,%ymm2,%ymm15 + vpsrld $25,%ymm2,%ymm2 + vpor %ymm2,%ymm15,%ymm2 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm7,%ymm13,%ymm13 + vpxor %ymm3,%ymm13,%ymm3 + vpslld $7,%ymm3,%ymm14 + vpsrld $25,%ymm3,%ymm3 + vpor %ymm3,%ymm14,%ymm3 + vpaddd %ymm1,%ymm8,%ymm8 + vpxor %ymm7,%ymm8,%ymm7 + vpshufb %ymm15,%ymm7,%ymm7 + vpaddd %ymm2,%ymm9,%ymm9 + vpxor %ymm4,%ymm9,%ymm4 + vpshufb %ymm15,%ymm4,%ymm4 + vpaddd %ymm7,%ymm12,%ymm12 + vpxor %ymm1,%ymm12,%ymm1 + vpslld $12,%ymm1,%ymm14 + vpsrld $20,%ymm1,%ymm1 + vpor %ymm1,%ymm14,%ymm1 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm4,%ymm13,%ymm13 + vpxor %ymm2,%ymm13,%ymm2 + vpslld $12,%ymm2,%ymm15 + vpsrld $20,%ymm2,%ymm2 + vpor %ymm2,%ymm15,%ymm2 + vpaddd %ymm1,%ymm8,%ymm8 + vpxor %ymm7,%ymm8,%ymm7 + vpshufb %ymm14,%ymm7,%ymm7 + vpaddd %ymm2,%ymm9,%ymm9 + vpxor %ymm4,%ymm9,%ymm4 + vpshufb %ymm14,%ymm4,%ymm4 + vpaddd %ymm7,%ymm12,%ymm12 + vpxor %ymm1,%ymm12,%ymm1 + vpslld $7,%ymm1,%ymm15 + vpsrld $25,%ymm1,%ymm1 + vpor %ymm1,%ymm15,%ymm1 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm4,%ymm13,%ymm13 + vpxor %ymm2,%ymm13,%ymm2 + vpslld $7,%ymm2,%ymm14 + vpsrld $25,%ymm2,%ymm2 + vpor %ymm2,%ymm14,%ymm2 + vmovdqa %ymm12,64(%rsp) + vmovdqa %ymm13,96(%rsp) + vmovdqa 0(%rsp),%ymm12 + vmovdqa 32(%rsp),%ymm13 + vpaddd %ymm3,%ymm10,%ymm10 + vpxor %ymm5,%ymm10,%ymm5 + vpshufb %ymm15,%ymm5,%ymm5 + vpaddd %ymm0,%ymm11,%ymm11 + vpxor %ymm6,%ymm11,%ymm6 + vpshufb %ymm15,%ymm6,%ymm6 + vpaddd %ymm5,%ymm12,%ymm12 + vpxor %ymm3,%ymm12,%ymm3 + vpslld $12,%ymm3,%ymm14 + vpsrld $20,%ymm3,%ymm3 + vpor %ymm3,%ymm14,%ymm3 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm6,%ymm13,%ymm13 + vpxor %ymm0,%ymm13,%ymm0 + vpslld $12,%ymm0,%ymm15 + vpsrld $20,%ymm0,%ymm0 + vpor %ymm0,%ymm15,%ymm0 + vpaddd %ymm3,%ymm10,%ymm10 + vpxor %ymm5,%ymm10,%ymm5 + vpshufb %ymm14,%ymm5,%ymm5 + vpaddd %ymm0,%ymm11,%ymm11 + vpxor %ymm6,%ymm11,%ymm6 + vpshufb %ymm14,%ymm6,%ymm6 + vpaddd %ymm5,%ymm12,%ymm12 + vpxor %ymm3,%ymm12,%ymm3 + vpslld $7,%ymm3,%ymm15 + vpsrld $25,%ymm3,%ymm3 + vpor %ymm3,%ymm15,%ymm3 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm6,%ymm13,%ymm13 + vpxor %ymm0,%ymm13,%ymm0 + vpslld $7,%ymm0,%ymm14 + vpsrld $25,%ymm0,%ymm0 + vpor %ymm0,%ymm14,%ymm0 + decl %eax + jnz L$oop8x + + leaq 512(%rsp),%rax + vpaddd 128-256(%rcx),%ymm8,%ymm8 + vpaddd 160-256(%rcx),%ymm9,%ymm9 + vpaddd 192-256(%rcx),%ymm10,%ymm10 + vpaddd 224-256(%rcx),%ymm11,%ymm11 + + vpunpckldq %ymm9,%ymm8,%ymm14 + vpunpckldq %ymm11,%ymm10,%ymm15 + vpunpckhdq %ymm9,%ymm8,%ymm8 + vpunpckhdq %ymm11,%ymm10,%ymm10 + vpunpcklqdq %ymm15,%ymm14,%ymm9 + vpunpckhqdq %ymm15,%ymm14,%ymm14 + vpunpcklqdq %ymm10,%ymm8,%ymm11 + vpunpckhqdq %ymm10,%ymm8,%ymm8 + vpaddd 256-256(%rcx),%ymm0,%ymm0 + vpaddd 288-256(%rcx),%ymm1,%ymm1 + vpaddd 320-256(%rcx),%ymm2,%ymm2 + vpaddd 352-256(%rcx),%ymm3,%ymm3 + + vpunpckldq %ymm1,%ymm0,%ymm10 + vpunpckldq %ymm3,%ymm2,%ymm15 + vpunpckhdq %ymm1,%ymm0,%ymm0 + vpunpckhdq %ymm3,%ymm2,%ymm2 + vpunpcklqdq %ymm15,%ymm10,%ymm1 + vpunpckhqdq %ymm15,%ymm10,%ymm10 + vpunpcklqdq %ymm2,%ymm0,%ymm3 + vpunpckhqdq %ymm2,%ymm0,%ymm0 + vperm2i128 $0x20,%ymm1,%ymm9,%ymm15 + vperm2i128 $0x31,%ymm1,%ymm9,%ymm1 + vperm2i128 $0x20,%ymm10,%ymm14,%ymm9 + vperm2i128 $0x31,%ymm10,%ymm14,%ymm10 + vperm2i128 $0x20,%ymm3,%ymm11,%ymm14 + vperm2i128 $0x31,%ymm3,%ymm11,%ymm3 + vperm2i128 $0x20,%ymm0,%ymm8,%ymm11 + vperm2i128 $0x31,%ymm0,%ymm8,%ymm0 + vmovdqa %ymm15,0(%rsp) + vmovdqa %ymm9,32(%rsp) + vmovdqa 64(%rsp),%ymm15 + vmovdqa 96(%rsp),%ymm9 + + vpaddd 384-512(%rax),%ymm12,%ymm12 + vpaddd 416-512(%rax),%ymm13,%ymm13 + vpaddd 448-512(%rax),%ymm15,%ymm15 + vpaddd 480-512(%rax),%ymm9,%ymm9 + + vpunpckldq %ymm13,%ymm12,%ymm2 + vpunpckldq %ymm9,%ymm15,%ymm8 + vpunpckhdq %ymm13,%ymm12,%ymm12 + vpunpckhdq %ymm9,%ymm15,%ymm15 + vpunpcklqdq %ymm8,%ymm2,%ymm13 + vpunpckhqdq %ymm8,%ymm2,%ymm2 + vpunpcklqdq %ymm15,%ymm12,%ymm9 + vpunpckhqdq %ymm15,%ymm12,%ymm12 + vpaddd 512-512(%rax),%ymm4,%ymm4 + vpaddd 544-512(%rax),%ymm5,%ymm5 + vpaddd 576-512(%rax),%ymm6,%ymm6 + vpaddd 608-512(%rax),%ymm7,%ymm7 + + vpunpckldq %ymm5,%ymm4,%ymm15 + vpunpckldq %ymm7,%ymm6,%ymm8 + vpunpckhdq %ymm5,%ymm4,%ymm4 + vpunpckhdq %ymm7,%ymm6,%ymm6 + vpunpcklqdq %ymm8,%ymm15,%ymm5 + vpunpckhqdq %ymm8,%ymm15,%ymm15 + vpunpcklqdq %ymm6,%ymm4,%ymm7 + vpunpckhqdq %ymm6,%ymm4,%ymm4 + vperm2i128 $0x20,%ymm5,%ymm13,%ymm8 + vperm2i128 $0x31,%ymm5,%ymm13,%ymm5 + vperm2i128 $0x20,%ymm15,%ymm2,%ymm13 + vperm2i128 $0x31,%ymm15,%ymm2,%ymm15 + vperm2i128 $0x20,%ymm7,%ymm9,%ymm2 + vperm2i128 $0x31,%ymm7,%ymm9,%ymm7 + vperm2i128 $0x20,%ymm4,%ymm12,%ymm9 + vperm2i128 $0x31,%ymm4,%ymm12,%ymm4 + vmovdqa 0(%rsp),%ymm6 + vmovdqa 32(%rsp),%ymm12 + + cmpq $512,%rdx + jb L$tail8x + + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + leaq 128(%rsi),%rsi + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm12,%ymm12 + vpxor 32(%rsi),%ymm13,%ymm13 + vpxor 64(%rsi),%ymm10,%ymm10 + vpxor 96(%rsi),%ymm15,%ymm15 + leaq 128(%rsi),%rsi + vmovdqu %ymm12,0(%rdi) + vmovdqu %ymm13,32(%rdi) + vmovdqu %ymm10,64(%rdi) + vmovdqu %ymm15,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm14,%ymm14 + vpxor 32(%rsi),%ymm2,%ymm2 + vpxor 64(%rsi),%ymm3,%ymm3 + vpxor 96(%rsi),%ymm7,%ymm7 + leaq 128(%rsi),%rsi + vmovdqu %ymm14,0(%rdi) + vmovdqu %ymm2,32(%rdi) + vmovdqu %ymm3,64(%rdi) + vmovdqu %ymm7,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm11,%ymm11 + vpxor 32(%rsi),%ymm9,%ymm9 + vpxor 64(%rsi),%ymm0,%ymm0 + vpxor 96(%rsi),%ymm4,%ymm4 + leaq 128(%rsi),%rsi + vmovdqu %ymm11,0(%rdi) + vmovdqu %ymm9,32(%rdi) + vmovdqu %ymm0,64(%rdi) + vmovdqu %ymm4,96(%rdi) + leaq 128(%rdi),%rdi + + subq $512,%rdx + jnz L$oop_outer8x + + jmp L$done8x + +L$tail8x: + cmpq $448,%rdx + jae L$448_or_more8x + cmpq $384,%rdx + jae L$384_or_more8x + cmpq $320,%rdx + jae L$320_or_more8x + cmpq $256,%rdx + jae L$256_or_more8x + cmpq $192,%rdx + jae L$192_or_more8x + cmpq $128,%rdx + jae L$128_or_more8x + cmpq $64,%rdx + jae L$64_or_more8x + + xorq %r9,%r9 + vmovdqa %ymm6,0(%rsp) + vmovdqa %ymm8,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$64_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + je L$done8x + + leaq 64(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm1,0(%rsp) + leaq 64(%rdi),%rdi + subq $64,%rdx + vmovdqa %ymm5,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$128_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + je L$done8x + + leaq 128(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm12,0(%rsp) + leaq 128(%rdi),%rdi + subq $128,%rdx + vmovdqa %ymm13,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$192_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + je L$done8x + + leaq 192(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm10,0(%rsp) + leaq 192(%rdi),%rdi + subq $192,%rdx + vmovdqa %ymm15,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$256_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + je L$done8x + + leaq 256(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm14,0(%rsp) + leaq 256(%rdi),%rdi + subq $256,%rdx + vmovdqa %ymm2,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$320_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + je L$done8x + + leaq 320(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm3,0(%rsp) + leaq 320(%rdi),%rdi + subq $320,%rdx + vmovdqa %ymm7,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$384_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vpxor 320(%rsi),%ymm3,%ymm3 + vpxor 352(%rsi),%ymm7,%ymm7 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + vmovdqu %ymm3,320(%rdi) + vmovdqu %ymm7,352(%rdi) + je L$done8x + + leaq 384(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm11,0(%rsp) + leaq 384(%rdi),%rdi + subq $384,%rdx + vmovdqa %ymm9,32(%rsp) + jmp L$oop_tail8x + +.p2align 5 +L$448_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vpxor 320(%rsi),%ymm3,%ymm3 + vpxor 352(%rsi),%ymm7,%ymm7 + vpxor 384(%rsi),%ymm11,%ymm11 + vpxor 416(%rsi),%ymm9,%ymm9 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + vmovdqu %ymm3,320(%rdi) + vmovdqu %ymm7,352(%rdi) + vmovdqu %ymm11,384(%rdi) + vmovdqu %ymm9,416(%rdi) + je L$done8x + + leaq 448(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm0,0(%rsp) + leaq 448(%rdi),%rdi + subq $448,%rdx + vmovdqa %ymm4,32(%rsp) + +L$oop_tail8x: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz L$oop_tail8x + +L$done8x: + vzeroall + leaq (%r10),%rsp +.cfi_def_cfa_register %rsp +L$avx2_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc + diff --git a/cbits/asm/chacha-x86_64-mingw64.S b/cbits/asm/chacha-x86_64-mingw64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-x86_64-mingw64.S @@ -0,0 +1,2556 @@ +.text + + + +.p2align 6 +.Lzero: +.long 0,0,0,0 +.Lone: +.long 1,0,0,0 +.Linc: +.long 0,1,2,3 +.Lfour: +.long 4,4,4,4 +.Lincy: +.long 0,2,4,6,1,3,5,7 +.Leight: +.long 8,8,8,8,8,8,8,8 +.Lrot16: +.byte 0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd +.Lrot24: +.byte 0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe +.Ltwoy: +.long 2,0,0,0, 2,0,0,0 +.p2align 6 +.Lzeroz: +.long 0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0 +.Lfourz: +.long 4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0 +.Lincz: +.long 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 +.Lsixteen: +.long 16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16 +.Lsigma: +.byte 101,120,112,97,110,100,32,51,50,45,98,121,116,101,32,107,0 +.byte 67,104,97,67,104,97,50,48,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.globl crypton_chacha20_asm_ctr32 +.def crypton_chacha20_asm_ctr32; .scl 2; .type 32; .endef +.p2align 6 +crypton_chacha20_asm_ctr32: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_chacha20_asm_ctr32: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 40(%rsp),%r8 + cmpq $0,%rdx + je .Lno_data + movq crypton_ia32cap_P+4(%rip),%r9 + testl $512,%r9d + jnz .Lcrypton_chacha20_asm_ssse3 + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + subq $64+24,%rsp + +.Lctr32_body: + + movq %rdx,%rbp + + movq 0(%rcx),%r12 + movq 8(%rcx),%r13 + movq 16(%rcx),%r14 + movq 24(%rcx),%r15 + movq 0(%r8),%rax + movq 8(%r8),%rdx + movq %r12,16(%rsp) + movq %r13,24(%rsp) + movq %r14,0(%rsp) + movq %r15,8(%rsp) + movq %rax,48(%rsp) + movq %rdx,56(%rsp) + jmp .Loop_outer + +.p2align 5 +.Loop_outer: + movl $0x61707865,%eax + movl $0x3320646e,%ebx + movl $0x79622d32,%ecx + movl $0x6b206574,%edx + movl 16(%rsp),%r8d + movl 20(%rsp),%r9d + movl 24(%rsp),%r10d + movl 28(%rsp),%r11d + movl 48(%rsp),%r12d + movl 52(%rsp),%r13d + movl 56(%rsp),%r14d + movq %r15,40(%rsp) + movl 60(%rsp),%r15d + + movq %rbp,64+0(%rsp) + movq %rsi,64+8(%rsp) + movl 0(%rsp),%esi + movq %rdi,64+16(%rsp) + movl 4(%rsp),%edi + movl $10,%ebp + jmp .Loop + +.p2align 5 +.Loop: + addl %r8d,%eax + xorl %eax,%r12d + roll $16,%r12d + addl %r9d,%ebx + xorl %ebx,%r13d + roll $16,%r13d + addl %r12d,%esi + xorl %esi,%r8d + roll $12,%r8d + addl %r13d,%edi + xorl %edi,%r9d + roll $12,%r9d + addl %r8d,%eax + xorl %eax,%r12d + roll $8,%r12d + addl %r9d,%ebx + xorl %ebx,%r13d + roll $8,%r13d + addl %r12d,%esi + xorl %esi,%r8d + roll $7,%r8d + addl %r13d,%edi + xorl %edi,%r9d + roll $7,%r9d + movl %esi,32(%rsp) + movl %edi,36(%rsp) + movl 40(%rsp),%esi + movl 44(%rsp),%edi + addl %r10d,%ecx + xorl %ecx,%r14d + roll $16,%r14d + addl %r11d,%edx + xorl %edx,%r15d + roll $16,%r15d + addl %r14d,%esi + xorl %esi,%r10d + roll $12,%r10d + addl %r15d,%edi + xorl %edi,%r11d + roll $12,%r11d + addl %r10d,%ecx + xorl %ecx,%r14d + roll $8,%r14d + addl %r11d,%edx + xorl %edx,%r15d + roll $8,%r15d + addl %r14d,%esi + xorl %esi,%r10d + roll $7,%r10d + addl %r15d,%edi + xorl %edi,%r11d + roll $7,%r11d + addl %r9d,%eax + xorl %eax,%r15d + roll $16,%r15d + addl %r10d,%ebx + xorl %ebx,%r12d + roll $16,%r12d + addl %r15d,%esi + xorl %esi,%r9d + roll $12,%r9d + addl %r12d,%edi + xorl %edi,%r10d + roll $12,%r10d + addl %r9d,%eax + xorl %eax,%r15d + roll $8,%r15d + addl %r10d,%ebx + xorl %ebx,%r12d + roll $8,%r12d + addl %r15d,%esi + xorl %esi,%r9d + roll $7,%r9d + addl %r12d,%edi + xorl %edi,%r10d + roll $7,%r10d + movl %esi,40(%rsp) + movl %edi,44(%rsp) + movl 32(%rsp),%esi + movl 36(%rsp),%edi + addl %r11d,%ecx + xorl %ecx,%r13d + roll $16,%r13d + addl %r8d,%edx + xorl %edx,%r14d + roll $16,%r14d + addl %r13d,%esi + xorl %esi,%r11d + roll $12,%r11d + addl %r14d,%edi + xorl %edi,%r8d + roll $12,%r8d + addl %r11d,%ecx + xorl %ecx,%r13d + roll $8,%r13d + addl %r8d,%edx + xorl %edx,%r14d + roll $8,%r14d + addl %r13d,%esi + xorl %esi,%r11d + roll $7,%r11d + addl %r14d,%edi + xorl %edi,%r8d + roll $7,%r8d + decl %ebp + jnz .Loop + addl 0(%rsp),%esi + addl 4(%rsp),%edi + movq 64(%rsp),%rbp + movl %esi,32(%rsp) + movq 64+8(%rsp),%rsi + movl %edi,36(%rsp) + movq 64+16(%rsp),%rdi + + addl $0x61707865,%eax + addl $0x3320646e,%ebx + addl $0x79622d32,%ecx + addl $0x6b206574,%edx + addl 16(%rsp),%r8d + addl 20(%rsp),%r9d + addl 24(%rsp),%r10d + addl 28(%rsp),%r11d + addl 48(%rsp),%r12d + addl 52(%rsp),%r13d + addl 56(%rsp),%r14d + addl 60(%rsp),%r15d + + cmpq $64,%rbp + jb .Ltail + + xorl 0(%rsi),%eax + xorl 4(%rsi),%ebx + xorl 8(%rsi),%ecx + xorl 12(%rsi),%edx + movl %eax,0(%rdi) + movl 32(%rsp),%eax + movl %ebx,4(%rdi) + movl 36(%rsp),%ebx + movl %ecx,8(%rdi) + movl 40(%rsp),%ecx + movl %edx,12(%rdi) + movl 44(%rsp),%edx + xorl 16(%rsi),%r8d + addl 8(%rsp),%ecx + xorl 20(%rsi),%r9d + addl 12(%rsp),%edx + xorl 24(%rsi),%r10d + xorl 28(%rsi),%r11d + xorl 32(%rsi),%eax + xorl 36(%rsi),%ebx + xorl 40(%rsi),%ecx + xorl 44(%rsi),%edx + xorl 48(%rsi),%r12d + xorl 52(%rsi),%r13d + xorl 56(%rsi),%r14d + xorl 60(%rsi),%r15d + leaq 64(%rsi),%rsi + + addl $1,48(%rsp) + + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + movl %eax,32(%rdi) + movl %ebx,36(%rdi) + movl %ecx,40(%rdi) + movl %edx,44(%rdi) + movl %r12d,48(%rdi) + movl %r13d,52(%rdi) + movl %r14d,56(%rdi) + movl %r15d,60(%rdi) + leaq 64(%rdi),%rdi + movq 8(%rsp),%r15 + + subq $64,%rbp + jnz .Loop_outer + + jmp .Ldone + +.p2align 4 +.Ltail: + movl %eax,0(%rsp) + movl 8(%rsp),%eax + movl %ebx,4(%rsp) + movl 12(%rsp),%ebx + movl %ecx,8(%rsp) + addl 40(%rsp),%eax + movl %edx,12(%rsp) + addl 44(%rsp),%ebx + movl %r8d,16(%rsp) + movl %r9d,20(%rsp) + movl %r10d,24(%rsp) + movl %r11d,28(%rsp) + movl %eax,40(%rsp) + movl %ebx,44(%rsp) + xorq %rbx,%rbx + movl %r12d,48(%rsp) + movl %r13d,52(%rsp) + movl %r14d,56(%rsp) + movl %r15d,60(%rsp) + +.Loop_tail: + movzbl (%rsi,%rbx,1),%eax + movzbl (%rsp,%rbx,1),%edx + leaq 1(%rbx),%rbx + xorl %edx,%eax + movb %al,-1(%rdi,%rbx,1) + decq %rbp + jnz .Loop_tail + +.Ldone: + leaq 64+24+48(%rsp),%rsi + + movq -48(%rsi),%r15 + + movq -40(%rsi),%r14 + + movq -32(%rsi),%r13 + + movq -24(%rsi),%r12 + + movq -16(%rsi),%rbp + + movq -8(%rsi),%rbx + + leaq (%rsi),%rsp + +.Lno_data: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_chacha20_asm_ctr32: +.def crypton_chacha20_asm_ssse3; .scl 3; .type 32; .endef +.p2align 5 +crypton_chacha20_asm_ssse3: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_chacha20_asm_ssse3: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 40(%rsp),%r8 +.Lcrypton_chacha20_asm_ssse3: + movq %rsp,%r10 + + testl $2048,%r9d + jnz .Lcrypton_chacha20_asm_4xop + cmpq $128,%rdx + je .Lcrypton_chacha20_asm_128 + ja .Lcrypton_chacha20_asm_4x + +.Ldo_sse3_after_all: + subq $64+40,%rsp + andq $-16,%rsp + movaps %xmm6,-40(%r10) + movaps %xmm7,-24(%r10) +.Lssse3_body: + movdqa .Lsigma(%rip),%xmm0 + movdqu (%rcx),%xmm1 + movdqu 16(%rcx),%xmm2 + movdqu (%r8),%xmm3 + movdqa .Lrot16(%rip),%xmm6 + movdqa .Lrot24(%rip),%xmm7 + + movdqa %xmm0,0(%rsp) + movdqa %xmm1,16(%rsp) + movdqa %xmm2,32(%rsp) + movdqa %xmm3,48(%rsp) + movq $10,%r8 + jmp .Loop_ssse3 + +.p2align 5 +.Loop_outer_ssse3: + movdqa .Lone(%rip),%xmm3 + movdqa 0(%rsp),%xmm0 + movdqa 16(%rsp),%xmm1 + movdqa 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + movq $10,%r8 + movdqa %xmm3,48(%rsp) + jmp .Loop_ssse3 + +.p2align 5 +.Loop_ssse3: + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,222 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $20,%xmm1 + pslld $12,%xmm4 + por %xmm4,%xmm1 + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,223 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $25,%xmm1 + pslld $7,%xmm4 + por %xmm4,%xmm1 + pshufd $78,%xmm2,%xmm2 + pshufd $57,%xmm1,%xmm1 + pshufd $147,%xmm3,%xmm3 + nop + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,222 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $20,%xmm1 + pslld $12,%xmm4 + por %xmm4,%xmm1 + paddd %xmm1,%xmm0 + pxor %xmm0,%xmm3 +.byte 102,15,56,0,223 + paddd %xmm3,%xmm2 + pxor %xmm2,%xmm1 + movdqa %xmm1,%xmm4 + psrld $25,%xmm1 + pslld $7,%xmm4 + por %xmm4,%xmm1 + pshufd $78,%xmm2,%xmm2 + pshufd $147,%xmm1,%xmm1 + pshufd $57,%xmm3,%xmm3 + decq %r8 + jnz .Loop_ssse3 + paddd 0(%rsp),%xmm0 + paddd 16(%rsp),%xmm1 + paddd 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + + cmpq $64,%rdx + jb .Ltail_ssse3 + + movdqu 0(%rsi),%xmm4 + movdqu 16(%rsi),%xmm5 + pxor %xmm4,%xmm0 + movdqu 32(%rsi),%xmm4 + pxor %xmm5,%xmm1 + movdqu 48(%rsi),%xmm5 + leaq 64(%rsi),%rsi + pxor %xmm4,%xmm2 + pxor %xmm5,%xmm3 + + movdqu %xmm0,0(%rdi) + movdqu %xmm1,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm3,48(%rdi) + leaq 64(%rdi),%rdi + + subq $64,%rdx + jnz .Loop_outer_ssse3 + + jmp .Ldone_ssse3 + +.p2align 4 +.Ltail_ssse3: + movdqa %xmm0,0(%rsp) + movdqa %xmm1,16(%rsp) + movdqa %xmm2,32(%rsp) + movdqa %xmm3,48(%rsp) + xorq %r8,%r8 + +.Loop_tail_ssse3: + movzbl (%rsi,%r8,1),%eax + movzbl (%rsp,%r8,1),%ecx + leaq 1(%r8),%r8 + xorl %ecx,%eax + movb %al,-1(%rdi,%r8,1) + decq %rdx + jnz .Loop_tail_ssse3 + +.Ldone_ssse3: + movaps -40(%r10),%xmm6 + movaps -24(%r10),%xmm7 + leaq (%r10),%rsp + +.Lssse3_epilogue: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_chacha20_asm_ssse3: +.def crypton_chacha20_asm_128; .scl 3; .type 32; .endef +.p2align 5 +crypton_chacha20_asm_128: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_chacha20_asm_128: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 40(%rsp),%r8 +.Lcrypton_chacha20_asm_128: + movq %rsp,%r10 + + subq $64+104,%rsp + andq $-16,%rsp + movaps %xmm6,-104(%r10) + movaps %xmm7,-88(%r10) + movaps %xmm8,-72(%r10) + movaps %xmm9,-56(%r10) + movaps %xmm10,-40(%r10) + movaps %xmm11,-24(%r10) +.L128_body: + movdqa .Lsigma(%rip),%xmm8 + movdqu (%rcx),%xmm9 + movdqu 16(%rcx),%xmm2 + movdqu (%r8),%xmm3 + movdqa .Lone(%rip),%xmm1 + movdqa .Lrot16(%rip),%xmm6 + movdqa .Lrot24(%rip),%xmm7 + + movdqa %xmm8,%xmm10 + movdqa %xmm8,0(%rsp) + movdqa %xmm9,%xmm11 + movdqa %xmm9,16(%rsp) + movdqa %xmm2,%xmm0 + movdqa %xmm2,32(%rsp) + paddd %xmm3,%xmm1 + movdqa %xmm3,48(%rsp) + movq $10,%r8 + jmp .Loop_128 + +.p2align 5 +.Loop_128: + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,222 +.byte 102,15,56,0,206 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $20,%xmm9 + movdqa %xmm11,%xmm5 + pslld $12,%xmm4 + psrld $20,%xmm11 + por %xmm4,%xmm9 + pslld $12,%xmm5 + por %xmm5,%xmm11 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,223 +.byte 102,15,56,0,207 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $25,%xmm9 + movdqa %xmm11,%xmm5 + pslld $7,%xmm4 + psrld $25,%xmm11 + por %xmm4,%xmm9 + pslld $7,%xmm5 + por %xmm5,%xmm11 + pshufd $78,%xmm2,%xmm2 + pshufd $57,%xmm9,%xmm9 + pshufd $147,%xmm3,%xmm3 + pshufd $78,%xmm0,%xmm0 + pshufd $57,%xmm11,%xmm11 + pshufd $147,%xmm1,%xmm1 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,222 +.byte 102,15,56,0,206 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $20,%xmm9 + movdqa %xmm11,%xmm5 + pslld $12,%xmm4 + psrld $20,%xmm11 + por %xmm4,%xmm9 + pslld $12,%xmm5 + por %xmm5,%xmm11 + paddd %xmm9,%xmm8 + pxor %xmm8,%xmm3 + paddd %xmm11,%xmm10 + pxor %xmm10,%xmm1 +.byte 102,15,56,0,223 +.byte 102,15,56,0,207 + paddd %xmm3,%xmm2 + paddd %xmm1,%xmm0 + pxor %xmm2,%xmm9 + pxor %xmm0,%xmm11 + movdqa %xmm9,%xmm4 + psrld $25,%xmm9 + movdqa %xmm11,%xmm5 + pslld $7,%xmm4 + psrld $25,%xmm11 + por %xmm4,%xmm9 + pslld $7,%xmm5 + por %xmm5,%xmm11 + pshufd $78,%xmm2,%xmm2 + pshufd $147,%xmm9,%xmm9 + pshufd $57,%xmm3,%xmm3 + pshufd $78,%xmm0,%xmm0 + pshufd $147,%xmm11,%xmm11 + pshufd $57,%xmm1,%xmm1 + decq %r8 + jnz .Loop_128 + paddd 0(%rsp),%xmm8 + paddd 16(%rsp),%xmm9 + paddd 32(%rsp),%xmm2 + paddd 48(%rsp),%xmm3 + paddd .Lone(%rip),%xmm1 + paddd 0(%rsp),%xmm10 + paddd 16(%rsp),%xmm11 + paddd 32(%rsp),%xmm0 + paddd 48(%rsp),%xmm1 + + movdqu 0(%rsi),%xmm4 + movdqu 16(%rsi),%xmm5 + pxor %xmm4,%xmm8 + movdqu 32(%rsi),%xmm4 + pxor %xmm5,%xmm9 + movdqu 48(%rsi),%xmm5 + pxor %xmm4,%xmm2 + movdqu 64(%rsi),%xmm4 + pxor %xmm5,%xmm3 + movdqu 80(%rsi),%xmm5 + pxor %xmm4,%xmm10 + movdqu 96(%rsi),%xmm4 + pxor %xmm5,%xmm11 + movdqu 112(%rsi),%xmm5 + pxor %xmm4,%xmm0 + pxor %xmm5,%xmm1 + + movdqu %xmm8,0(%rdi) + movdqu %xmm9,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm3,48(%rdi) + movdqu %xmm10,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm0,96(%rdi) + movdqu %xmm1,112(%rdi) + movaps -104(%r10),%xmm6 + movaps -88(%r10),%xmm7 + movaps -72(%r10),%xmm8 + movaps -56(%r10),%xmm9 + movaps -40(%r10),%xmm10 + movaps -24(%r10),%xmm11 + leaq (%r10),%rsp + +.L128_epilogue: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_chacha20_asm_128: +.def crypton_chacha20_asm_4x; .scl 3; .type 32; .endef +.p2align 5 +crypton_chacha20_asm_4x: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_chacha20_asm_4x: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 40(%rsp),%r8 +.Lcrypton_chacha20_asm_4x: + movq %rsp,%r10 + + movq %r9,%r11 + shrq $32,%r9 + testq $32,%r9 + jnz .Lcrypton_chacha20_asm_8x + cmpq $192,%rdx + ja .Lproceed4x + + andq $71303168,%r11 + cmpq $4194304,%r11 + je .Ldo_sse3_after_all + +.Lproceed4x: + subq $0x140+168,%rsp + andq $-16,%rsp + movaps %xmm6,-168(%r10) + movaps %xmm7,-152(%r10) + movaps %xmm8,-136(%r10) + movaps %xmm9,-120(%r10) + movaps %xmm10,-104(%r10) + movaps %xmm11,-88(%r10) + movaps %xmm12,-72(%r10) + movaps %xmm13,-56(%r10) + movaps %xmm14,-40(%r10) + movaps %xmm15,-24(%r10) +.L4x_body: + movdqa .Lsigma(%rip),%xmm11 + movdqu (%rcx),%xmm15 + movdqu 16(%rcx),%xmm7 + movdqu (%r8),%xmm3 + leaq 256(%rsp),%rcx + leaq .Lrot16(%rip),%r9 + leaq .Lrot24(%rip),%r11 + + pshufd $0x00,%xmm11,%xmm8 + pshufd $0x55,%xmm11,%xmm9 + movdqa %xmm8,64(%rsp) + pshufd $0xaa,%xmm11,%xmm10 + movdqa %xmm9,80(%rsp) + pshufd $0xff,%xmm11,%xmm11 + movdqa %xmm10,96(%rsp) + movdqa %xmm11,112(%rsp) + + pshufd $0x00,%xmm15,%xmm12 + pshufd $0x55,%xmm15,%xmm13 + movdqa %xmm12,128-256(%rcx) + pshufd $0xaa,%xmm15,%xmm14 + movdqa %xmm13,144-256(%rcx) + pshufd $0xff,%xmm15,%xmm15 + movdqa %xmm14,160-256(%rcx) + movdqa %xmm15,176-256(%rcx) + + pshufd $0x00,%xmm7,%xmm4 + pshufd $0x55,%xmm7,%xmm5 + movdqa %xmm4,192-256(%rcx) + pshufd $0xaa,%xmm7,%xmm6 + movdqa %xmm5,208-256(%rcx) + pshufd $0xff,%xmm7,%xmm7 + movdqa %xmm6,224-256(%rcx) + movdqa %xmm7,240-256(%rcx) + + pshufd $0x00,%xmm3,%xmm0 + pshufd $0x55,%xmm3,%xmm1 + paddd .Linc(%rip),%xmm0 + pshufd $0xaa,%xmm3,%xmm2 + movdqa %xmm1,272-256(%rcx) + pshufd $0xff,%xmm3,%xmm3 + movdqa %xmm2,288-256(%rcx) + movdqa %xmm3,304-256(%rcx) + + jmp .Loop_enter4x + +.p2align 5 +.Loop_outer4x: + movdqa 64(%rsp),%xmm8 + movdqa 80(%rsp),%xmm9 + movdqa 96(%rsp),%xmm10 + movdqa 112(%rsp),%xmm11 + movdqa 128-256(%rcx),%xmm12 + movdqa 144-256(%rcx),%xmm13 + movdqa 160-256(%rcx),%xmm14 + movdqa 176-256(%rcx),%xmm15 + movdqa 192-256(%rcx),%xmm4 + movdqa 208-256(%rcx),%xmm5 + movdqa 224-256(%rcx),%xmm6 + movdqa 240-256(%rcx),%xmm7 + movdqa 256-256(%rcx),%xmm0 + movdqa 272-256(%rcx),%xmm1 + movdqa 288-256(%rcx),%xmm2 + movdqa 304-256(%rcx),%xmm3 + paddd .Lfour(%rip),%xmm0 + +.Loop_enter4x: + movdqa %xmm6,32(%rsp) + movdqa %xmm7,48(%rsp) + movdqa (%r9),%xmm7 + movl $10,%eax + movdqa %xmm0,256-256(%rcx) + jmp .Loop4x + +.p2align 5 +.Loop4x: + paddd %xmm12,%xmm8 + paddd %xmm13,%xmm9 + pxor %xmm8,%xmm0 + pxor %xmm9,%xmm1 +.byte 102,15,56,0,199 +.byte 102,15,56,0,207 + paddd %xmm0,%xmm4 + paddd %xmm1,%xmm5 + pxor %xmm4,%xmm12 + pxor %xmm5,%xmm13 + movdqa %xmm12,%xmm6 + pslld $12,%xmm12 + psrld $20,%xmm6 + movdqa %xmm13,%xmm7 + pslld $12,%xmm13 + por %xmm6,%xmm12 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm13 + paddd %xmm12,%xmm8 + paddd %xmm13,%xmm9 + pxor %xmm8,%xmm0 + pxor %xmm9,%xmm1 +.byte 102,15,56,0,198 +.byte 102,15,56,0,206 + paddd %xmm0,%xmm4 + paddd %xmm1,%xmm5 + pxor %xmm4,%xmm12 + pxor %xmm5,%xmm13 + movdqa %xmm12,%xmm7 + pslld $7,%xmm12 + psrld $25,%xmm7 + movdqa %xmm13,%xmm6 + pslld $7,%xmm13 + por %xmm7,%xmm12 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm13 + movdqa %xmm4,0(%rsp) + movdqa %xmm5,16(%rsp) + movdqa 32(%rsp),%xmm4 + movdqa 48(%rsp),%xmm5 + paddd %xmm14,%xmm10 + paddd %xmm15,%xmm11 + pxor %xmm10,%xmm2 + pxor %xmm11,%xmm3 +.byte 102,15,56,0,215 +.byte 102,15,56,0,223 + paddd %xmm2,%xmm4 + paddd %xmm3,%xmm5 + pxor %xmm4,%xmm14 + pxor %xmm5,%xmm15 + movdqa %xmm14,%xmm6 + pslld $12,%xmm14 + psrld $20,%xmm6 + movdqa %xmm15,%xmm7 + pslld $12,%xmm15 + por %xmm6,%xmm14 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm15 + paddd %xmm14,%xmm10 + paddd %xmm15,%xmm11 + pxor %xmm10,%xmm2 + pxor %xmm11,%xmm3 +.byte 102,15,56,0,214 +.byte 102,15,56,0,222 + paddd %xmm2,%xmm4 + paddd %xmm3,%xmm5 + pxor %xmm4,%xmm14 + pxor %xmm5,%xmm15 + movdqa %xmm14,%xmm7 + pslld $7,%xmm14 + psrld $25,%xmm7 + movdqa %xmm15,%xmm6 + pslld $7,%xmm15 + por %xmm7,%xmm14 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm15 + paddd %xmm13,%xmm8 + paddd %xmm14,%xmm9 + pxor %xmm8,%xmm3 + pxor %xmm9,%xmm0 +.byte 102,15,56,0,223 +.byte 102,15,56,0,199 + paddd %xmm3,%xmm4 + paddd %xmm0,%xmm5 + pxor %xmm4,%xmm13 + pxor %xmm5,%xmm14 + movdqa %xmm13,%xmm6 + pslld $12,%xmm13 + psrld $20,%xmm6 + movdqa %xmm14,%xmm7 + pslld $12,%xmm14 + por %xmm6,%xmm13 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm14 + paddd %xmm13,%xmm8 + paddd %xmm14,%xmm9 + pxor %xmm8,%xmm3 + pxor %xmm9,%xmm0 +.byte 102,15,56,0,222 +.byte 102,15,56,0,198 + paddd %xmm3,%xmm4 + paddd %xmm0,%xmm5 + pxor %xmm4,%xmm13 + pxor %xmm5,%xmm14 + movdqa %xmm13,%xmm7 + pslld $7,%xmm13 + psrld $25,%xmm7 + movdqa %xmm14,%xmm6 + pslld $7,%xmm14 + por %xmm7,%xmm13 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm14 + movdqa %xmm4,32(%rsp) + movdqa %xmm5,48(%rsp) + movdqa 0(%rsp),%xmm4 + movdqa 16(%rsp),%xmm5 + paddd %xmm15,%xmm10 + paddd %xmm12,%xmm11 + pxor %xmm10,%xmm1 + pxor %xmm11,%xmm2 +.byte 102,15,56,0,207 +.byte 102,15,56,0,215 + paddd %xmm1,%xmm4 + paddd %xmm2,%xmm5 + pxor %xmm4,%xmm15 + pxor %xmm5,%xmm12 + movdqa %xmm15,%xmm6 + pslld $12,%xmm15 + psrld $20,%xmm6 + movdqa %xmm12,%xmm7 + pslld $12,%xmm12 + por %xmm6,%xmm15 + psrld $20,%xmm7 + movdqa (%r11),%xmm6 + por %xmm7,%xmm12 + paddd %xmm15,%xmm10 + paddd %xmm12,%xmm11 + pxor %xmm10,%xmm1 + pxor %xmm11,%xmm2 +.byte 102,15,56,0,206 +.byte 102,15,56,0,214 + paddd %xmm1,%xmm4 + paddd %xmm2,%xmm5 + pxor %xmm4,%xmm15 + pxor %xmm5,%xmm12 + movdqa %xmm15,%xmm7 + pslld $7,%xmm15 + psrld $25,%xmm7 + movdqa %xmm12,%xmm6 + pslld $7,%xmm12 + por %xmm7,%xmm15 + psrld $25,%xmm6 + movdqa (%r9),%xmm7 + por %xmm6,%xmm12 + decl %eax + jnz .Loop4x + + paddd 64(%rsp),%xmm8 + paddd 80(%rsp),%xmm9 + paddd 96(%rsp),%xmm10 + paddd 112(%rsp),%xmm11 + + movdqa %xmm8,%xmm6 + punpckldq %xmm9,%xmm8 + movdqa %xmm10,%xmm7 + punpckldq %xmm11,%xmm10 + punpckhdq %xmm9,%xmm6 + punpckhdq %xmm11,%xmm7 + movdqa %xmm8,%xmm9 + punpcklqdq %xmm10,%xmm8 + movdqa %xmm6,%xmm11 + punpcklqdq %xmm7,%xmm6 + punpckhqdq %xmm10,%xmm9 + punpckhqdq %xmm7,%xmm11 + paddd 128-256(%rcx),%xmm12 + paddd 144-256(%rcx),%xmm13 + paddd 160-256(%rcx),%xmm14 + paddd 176-256(%rcx),%xmm15 + + movdqa %xmm8,0(%rsp) + movdqa %xmm9,16(%rsp) + movdqa 32(%rsp),%xmm8 + movdqa 48(%rsp),%xmm9 + + movdqa %xmm12,%xmm10 + punpckldq %xmm13,%xmm12 + movdqa %xmm14,%xmm7 + punpckldq %xmm15,%xmm14 + punpckhdq %xmm13,%xmm10 + punpckhdq %xmm15,%xmm7 + movdqa %xmm12,%xmm13 + punpcklqdq %xmm14,%xmm12 + movdqa %xmm10,%xmm15 + punpcklqdq %xmm7,%xmm10 + punpckhqdq %xmm14,%xmm13 + punpckhqdq %xmm7,%xmm15 + paddd 192-256(%rcx),%xmm4 + paddd 208-256(%rcx),%xmm5 + paddd 224-256(%rcx),%xmm8 + paddd 240-256(%rcx),%xmm9 + + movdqa %xmm6,32(%rsp) + movdqa %xmm11,48(%rsp) + + movdqa %xmm4,%xmm14 + punpckldq %xmm5,%xmm4 + movdqa %xmm8,%xmm7 + punpckldq %xmm9,%xmm8 + punpckhdq %xmm5,%xmm14 + punpckhdq %xmm9,%xmm7 + movdqa %xmm4,%xmm5 + punpcklqdq %xmm8,%xmm4 + movdqa %xmm14,%xmm9 + punpcklqdq %xmm7,%xmm14 + punpckhqdq %xmm8,%xmm5 + punpckhqdq %xmm7,%xmm9 + paddd 256-256(%rcx),%xmm0 + paddd 272-256(%rcx),%xmm1 + paddd 288-256(%rcx),%xmm2 + paddd 304-256(%rcx),%xmm3 + + movdqa %xmm0,%xmm8 + punpckldq %xmm1,%xmm0 + movdqa %xmm2,%xmm7 + punpckldq %xmm3,%xmm2 + punpckhdq %xmm1,%xmm8 + punpckhdq %xmm3,%xmm7 + movdqa %xmm0,%xmm1 + punpcklqdq %xmm2,%xmm0 + movdqa %xmm8,%xmm3 + punpcklqdq %xmm7,%xmm8 + punpckhqdq %xmm2,%xmm1 + punpckhqdq %xmm7,%xmm3 + cmpq $256,%rdx + jb .Ltail4x + + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + + movdqu %xmm6,64(%rdi) + movdqu 0(%rsi),%xmm6 + movdqu %xmm11,80(%rdi) + movdqu 16(%rsi),%xmm11 + movdqu %xmm2,96(%rdi) + movdqu 32(%rsi),%xmm2 + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + movdqu 48(%rsi),%xmm7 + pxor 32(%rsp),%xmm6 + pxor %xmm10,%xmm11 + pxor %xmm14,%xmm2 + pxor %xmm8,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 48(%rsp),%xmm6 + pxor %xmm15,%xmm11 + pxor %xmm9,%xmm2 + pxor %xmm3,%xmm7 + movdqu %xmm6,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm2,96(%rdi) + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + + subq $256,%rdx + jnz .Loop_outer4x + + jmp .Ldone4x + +.Ltail4x: + cmpq $192,%rdx + jae .L192_or_more4x + cmpq $128,%rdx + jae .L128_or_more4x + cmpq $64,%rdx + jae .L64_or_more4x + + + xorq %r9,%r9 + + movdqa %xmm12,16(%rsp) + movdqa %xmm4,32(%rsp) + movdqa %xmm0,48(%rsp) + jmp .Loop_tail4x + +.p2align 5 +.L64_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + movdqu %xmm6,0(%rdi) + movdqu %xmm11,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm7,48(%rdi) + je .Ldone4x + + movdqa 16(%rsp),%xmm6 + leaq 64(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm13,16(%rsp) + leaq 64(%rdi),%rdi + movdqa %xmm5,32(%rsp) + subq $64,%rdx + movdqa %xmm1,48(%rsp) + jmp .Loop_tail4x + +.p2align 5 +.L128_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + movdqu %xmm6,64(%rdi) + movdqu %xmm11,80(%rdi) + movdqu %xmm2,96(%rdi) + movdqu %xmm7,112(%rdi) + je .Ldone4x + + movdqa 32(%rsp),%xmm6 + leaq 128(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm10,16(%rsp) + leaq 128(%rdi),%rdi + movdqa %xmm14,32(%rsp) + subq $128,%rdx + movdqa %xmm8,48(%rsp) + jmp .Loop_tail4x + +.p2align 5 +.L192_or_more4x: + movdqu 0(%rsi),%xmm6 + movdqu 16(%rsi),%xmm11 + movdqu 32(%rsi),%xmm2 + movdqu 48(%rsi),%xmm7 + pxor 0(%rsp),%xmm6 + pxor %xmm12,%xmm11 + pxor %xmm4,%xmm2 + pxor %xmm0,%xmm7 + + movdqu %xmm6,0(%rdi) + movdqu 64(%rsi),%xmm6 + movdqu %xmm11,16(%rdi) + movdqu 80(%rsi),%xmm11 + movdqu %xmm2,32(%rdi) + movdqu 96(%rsi),%xmm2 + movdqu %xmm7,48(%rdi) + movdqu 112(%rsi),%xmm7 + leaq 128(%rsi),%rsi + pxor 16(%rsp),%xmm6 + pxor %xmm13,%xmm11 + pxor %xmm5,%xmm2 + pxor %xmm1,%xmm7 + + movdqu %xmm6,64(%rdi) + movdqu 0(%rsi),%xmm6 + movdqu %xmm11,80(%rdi) + movdqu 16(%rsi),%xmm11 + movdqu %xmm2,96(%rdi) + movdqu 32(%rsi),%xmm2 + movdqu %xmm7,112(%rdi) + leaq 128(%rdi),%rdi + movdqu 48(%rsi),%xmm7 + pxor 32(%rsp),%xmm6 + pxor %xmm10,%xmm11 + pxor %xmm14,%xmm2 + pxor %xmm8,%xmm7 + movdqu %xmm6,0(%rdi) + movdqu %xmm11,16(%rdi) + movdqu %xmm2,32(%rdi) + movdqu %xmm7,48(%rdi) + je .Ldone4x + + movdqa 48(%rsp),%xmm6 + leaq 64(%rsi),%rsi + xorq %r9,%r9 + movdqa %xmm6,0(%rsp) + movdqa %xmm15,16(%rsp) + leaq 64(%rdi),%rdi + movdqa %xmm9,32(%rsp) + subq $192,%rdx + movdqa %xmm3,48(%rsp) + +.Loop_tail4x: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz .Loop_tail4x + +.Ldone4x: + movaps -168(%r10),%xmm6 + movaps -152(%r10),%xmm7 + movaps -136(%r10),%xmm8 + movaps -120(%r10),%xmm9 + movaps -104(%r10),%xmm10 + movaps -88(%r10),%xmm11 + movaps -72(%r10),%xmm12 + movaps -56(%r10),%xmm13 + movaps -40(%r10),%xmm14 + movaps -24(%r10),%xmm15 + leaq (%r10),%rsp + +.L4x_epilogue: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_chacha20_asm_4x: +.def crypton_chacha20_asm_4xop; .scl 3; .type 32; .endef +.p2align 5 +crypton_chacha20_asm_4xop: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_chacha20_asm_4xop: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 40(%rsp),%r8 +.Lcrypton_chacha20_asm_4xop: + movq %rsp,%r10 + + subq $0x140+168,%rsp + andq $-16,%rsp + movaps %xmm6,-168(%r10) + movaps %xmm7,-152(%r10) + movaps %xmm8,-136(%r10) + movaps %xmm9,-120(%r10) + movaps %xmm10,-104(%r10) + movaps %xmm11,-88(%r10) + movaps %xmm12,-72(%r10) + movaps %xmm13,-56(%r10) + movaps %xmm14,-40(%r10) + movaps %xmm15,-24(%r10) +.L4xop_body: + vzeroupper + + vmovdqa .Lsigma(%rip),%xmm11 + vmovdqu (%rcx),%xmm3 + vmovdqu 16(%rcx),%xmm15 + vmovdqu (%r8),%xmm7 + leaq 256(%rsp),%rcx + + vpshufd $0x00,%xmm11,%xmm8 + vpshufd $0x55,%xmm11,%xmm9 + vmovdqa %xmm8,64(%rsp) + vpshufd $0xaa,%xmm11,%xmm10 + vmovdqa %xmm9,80(%rsp) + vpshufd $0xff,%xmm11,%xmm11 + vmovdqa %xmm10,96(%rsp) + vmovdqa %xmm11,112(%rsp) + + vpshufd $0x00,%xmm3,%xmm0 + vpshufd $0x55,%xmm3,%xmm1 + vmovdqa %xmm0,128-256(%rcx) + vpshufd $0xaa,%xmm3,%xmm2 + vmovdqa %xmm1,144-256(%rcx) + vpshufd $0xff,%xmm3,%xmm3 + vmovdqa %xmm2,160-256(%rcx) + vmovdqa %xmm3,176-256(%rcx) + + vpshufd $0x00,%xmm15,%xmm12 + vpshufd $0x55,%xmm15,%xmm13 + vmovdqa %xmm12,192-256(%rcx) + vpshufd $0xaa,%xmm15,%xmm14 + vmovdqa %xmm13,208-256(%rcx) + vpshufd $0xff,%xmm15,%xmm15 + vmovdqa %xmm14,224-256(%rcx) + vmovdqa %xmm15,240-256(%rcx) + + vpshufd $0x00,%xmm7,%xmm4 + vpshufd $0x55,%xmm7,%xmm5 + vpaddd .Linc(%rip),%xmm4,%xmm4 + vpshufd $0xaa,%xmm7,%xmm6 + vmovdqa %xmm5,272-256(%rcx) + vpshufd $0xff,%xmm7,%xmm7 + vmovdqa %xmm6,288-256(%rcx) + vmovdqa %xmm7,304-256(%rcx) + + jmp .Loop_enter4xop + +.p2align 5 +.Loop_outer4xop: + vmovdqa 64(%rsp),%xmm8 + vmovdqa 80(%rsp),%xmm9 + vmovdqa 96(%rsp),%xmm10 + vmovdqa 112(%rsp),%xmm11 + vmovdqa 128-256(%rcx),%xmm0 + vmovdqa 144-256(%rcx),%xmm1 + vmovdqa 160-256(%rcx),%xmm2 + vmovdqa 176-256(%rcx),%xmm3 + vmovdqa 192-256(%rcx),%xmm12 + vmovdqa 208-256(%rcx),%xmm13 + vmovdqa 224-256(%rcx),%xmm14 + vmovdqa 240-256(%rcx),%xmm15 + vmovdqa 256-256(%rcx),%xmm4 + vmovdqa 272-256(%rcx),%xmm5 + vmovdqa 288-256(%rcx),%xmm6 + vmovdqa 304-256(%rcx),%xmm7 + vpaddd .Lfour(%rip),%xmm4,%xmm4 + +.Loop_enter4xop: + movl $10,%eax + vmovdqa %xmm4,256-256(%rcx) + jmp .Loop4xop + +.p2align 5 +.Loop4xop: + vpaddd %xmm0,%xmm8,%xmm8 + vpaddd %xmm1,%xmm9,%xmm9 + vpaddd %xmm2,%xmm10,%xmm10 + vpaddd %xmm3,%xmm11,%xmm11 + vpxor %xmm4,%xmm8,%xmm4 + vpxor %xmm5,%xmm9,%xmm5 + vpxor %xmm6,%xmm10,%xmm6 + vpxor %xmm7,%xmm11,%xmm7 +.byte 143,232,120,194,228,16 +.byte 143,232,120,194,237,16 +.byte 143,232,120,194,246,16 +.byte 143,232,120,194,255,16 + vpaddd %xmm4,%xmm12,%xmm12 + vpaddd %xmm5,%xmm13,%xmm13 + vpaddd %xmm6,%xmm14,%xmm14 + vpaddd %xmm7,%xmm15,%xmm15 + vpxor %xmm0,%xmm12,%xmm0 + vpxor %xmm1,%xmm13,%xmm1 + vpxor %xmm14,%xmm2,%xmm2 + vpxor %xmm15,%xmm3,%xmm3 +.byte 143,232,120,194,192,12 +.byte 143,232,120,194,201,12 +.byte 143,232,120,194,210,12 +.byte 143,232,120,194,219,12 + vpaddd %xmm8,%xmm0,%xmm8 + vpaddd %xmm9,%xmm1,%xmm9 + vpaddd %xmm2,%xmm10,%xmm10 + vpaddd %xmm3,%xmm11,%xmm11 + vpxor %xmm4,%xmm8,%xmm4 + vpxor %xmm5,%xmm9,%xmm5 + vpxor %xmm6,%xmm10,%xmm6 + vpxor %xmm7,%xmm11,%xmm7 +.byte 143,232,120,194,228,8 +.byte 143,232,120,194,237,8 +.byte 143,232,120,194,246,8 +.byte 143,232,120,194,255,8 + vpaddd %xmm4,%xmm12,%xmm12 + vpaddd %xmm5,%xmm13,%xmm13 + vpaddd %xmm6,%xmm14,%xmm14 + vpaddd %xmm7,%xmm15,%xmm15 + vpxor %xmm0,%xmm12,%xmm0 + vpxor %xmm1,%xmm13,%xmm1 + vpxor %xmm14,%xmm2,%xmm2 + vpxor %xmm15,%xmm3,%xmm3 +.byte 143,232,120,194,192,7 +.byte 143,232,120,194,201,7 +.byte 143,232,120,194,210,7 +.byte 143,232,120,194,219,7 + vpaddd %xmm1,%xmm8,%xmm8 + vpaddd %xmm2,%xmm9,%xmm9 + vpaddd %xmm3,%xmm10,%xmm10 + vpaddd %xmm0,%xmm11,%xmm11 + vpxor %xmm7,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm4 + vpxor %xmm5,%xmm10,%xmm5 + vpxor %xmm6,%xmm11,%xmm6 +.byte 143,232,120,194,255,16 +.byte 143,232,120,194,228,16 +.byte 143,232,120,194,237,16 +.byte 143,232,120,194,246,16 + vpaddd %xmm7,%xmm14,%xmm14 + vpaddd %xmm4,%xmm15,%xmm15 + vpaddd %xmm5,%xmm12,%xmm12 + vpaddd %xmm6,%xmm13,%xmm13 + vpxor %xmm1,%xmm14,%xmm1 + vpxor %xmm2,%xmm15,%xmm2 + vpxor %xmm12,%xmm3,%xmm3 + vpxor %xmm13,%xmm0,%xmm0 +.byte 143,232,120,194,201,12 +.byte 143,232,120,194,210,12 +.byte 143,232,120,194,219,12 +.byte 143,232,120,194,192,12 + vpaddd %xmm8,%xmm1,%xmm8 + vpaddd %xmm9,%xmm2,%xmm9 + vpaddd %xmm3,%xmm10,%xmm10 + vpaddd %xmm0,%xmm11,%xmm11 + vpxor %xmm7,%xmm8,%xmm7 + vpxor %xmm4,%xmm9,%xmm4 + vpxor %xmm5,%xmm10,%xmm5 + vpxor %xmm6,%xmm11,%xmm6 +.byte 143,232,120,194,255,8 +.byte 143,232,120,194,228,8 +.byte 143,232,120,194,237,8 +.byte 143,232,120,194,246,8 + vpaddd %xmm7,%xmm14,%xmm14 + vpaddd %xmm4,%xmm15,%xmm15 + vpaddd %xmm5,%xmm12,%xmm12 + vpaddd %xmm6,%xmm13,%xmm13 + vpxor %xmm1,%xmm14,%xmm1 + vpxor %xmm2,%xmm15,%xmm2 + vpxor %xmm12,%xmm3,%xmm3 + vpxor %xmm13,%xmm0,%xmm0 +.byte 143,232,120,194,201,7 +.byte 143,232,120,194,210,7 +.byte 143,232,120,194,219,7 +.byte 143,232,120,194,192,7 + decl %eax + jnz .Loop4xop + + vpaddd 64(%rsp),%xmm8,%xmm8 + vpaddd 80(%rsp),%xmm9,%xmm9 + vpaddd 96(%rsp),%xmm10,%xmm10 + vpaddd 112(%rsp),%xmm11,%xmm11 + + vmovdqa %xmm14,32(%rsp) + vmovdqa %xmm15,48(%rsp) + + vpunpckldq %xmm9,%xmm8,%xmm14 + vpunpckldq %xmm11,%xmm10,%xmm15 + vpunpckhdq %xmm9,%xmm8,%xmm8 + vpunpckhdq %xmm11,%xmm10,%xmm10 + vpunpcklqdq %xmm15,%xmm14,%xmm9 + vpunpckhqdq %xmm15,%xmm14,%xmm14 + vpunpcklqdq %xmm10,%xmm8,%xmm11 + vpunpckhqdq %xmm10,%xmm8,%xmm8 + vpaddd 128-256(%rcx),%xmm0,%xmm0 + vpaddd 144-256(%rcx),%xmm1,%xmm1 + vpaddd 160-256(%rcx),%xmm2,%xmm2 + vpaddd 176-256(%rcx),%xmm3,%xmm3 + + vmovdqa %xmm9,0(%rsp) + vmovdqa %xmm14,16(%rsp) + vmovdqa 32(%rsp),%xmm9 + vmovdqa 48(%rsp),%xmm14 + + vpunpckldq %xmm1,%xmm0,%xmm10 + vpunpckldq %xmm3,%xmm2,%xmm15 + vpunpckhdq %xmm1,%xmm0,%xmm0 + vpunpckhdq %xmm3,%xmm2,%xmm2 + vpunpcklqdq %xmm15,%xmm10,%xmm1 + vpunpckhqdq %xmm15,%xmm10,%xmm10 + vpunpcklqdq %xmm2,%xmm0,%xmm3 + vpunpckhqdq %xmm2,%xmm0,%xmm0 + vpaddd 192-256(%rcx),%xmm12,%xmm12 + vpaddd 208-256(%rcx),%xmm13,%xmm13 + vpaddd 224-256(%rcx),%xmm9,%xmm9 + vpaddd 240-256(%rcx),%xmm14,%xmm14 + + vpunpckldq %xmm13,%xmm12,%xmm2 + vpunpckldq %xmm14,%xmm9,%xmm15 + vpunpckhdq %xmm13,%xmm12,%xmm12 + vpunpckhdq %xmm14,%xmm9,%xmm9 + vpunpcklqdq %xmm15,%xmm2,%xmm13 + vpunpckhqdq %xmm15,%xmm2,%xmm2 + vpunpcklqdq %xmm9,%xmm12,%xmm14 + vpunpckhqdq %xmm9,%xmm12,%xmm12 + vpaddd 256-256(%rcx),%xmm4,%xmm4 + vpaddd 272-256(%rcx),%xmm5,%xmm5 + vpaddd 288-256(%rcx),%xmm6,%xmm6 + vpaddd 304-256(%rcx),%xmm7,%xmm7 + + vpunpckldq %xmm5,%xmm4,%xmm9 + vpunpckldq %xmm7,%xmm6,%xmm15 + vpunpckhdq %xmm5,%xmm4,%xmm4 + vpunpckhdq %xmm7,%xmm6,%xmm6 + vpunpcklqdq %xmm15,%xmm9,%xmm5 + vpunpckhqdq %xmm15,%xmm9,%xmm9 + vpunpcklqdq %xmm6,%xmm4,%xmm7 + vpunpckhqdq %xmm6,%xmm4,%xmm4 + vmovdqa 0(%rsp),%xmm6 + vmovdqa 16(%rsp),%xmm15 + + cmpq $256,%rdx + jb .Ltail4xop + + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + leaq 128(%rsi),%rsi + vpxor 0(%rsi),%xmm11,%xmm11 + vpxor 16(%rsi),%xmm3,%xmm3 + vpxor 32(%rsi),%xmm14,%xmm14 + vpxor 48(%rsi),%xmm7,%xmm7 + vpxor 64(%rsi),%xmm8,%xmm8 + vpxor 80(%rsi),%xmm0,%xmm0 + vpxor 96(%rsi),%xmm12,%xmm12 + vpxor 112(%rsi),%xmm4,%xmm4 + leaq 128(%rsi),%rsi + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + leaq 128(%rdi),%rdi + vmovdqu %xmm11,0(%rdi) + vmovdqu %xmm3,16(%rdi) + vmovdqu %xmm14,32(%rdi) + vmovdqu %xmm7,48(%rdi) + vmovdqu %xmm8,64(%rdi) + vmovdqu %xmm0,80(%rdi) + vmovdqu %xmm12,96(%rdi) + vmovdqu %xmm4,112(%rdi) + leaq 128(%rdi),%rdi + + subq $256,%rdx + jnz .Loop_outer4xop + + jmp .Ldone4xop + +.p2align 5 +.Ltail4xop: + cmpq $192,%rdx + jae .L192_or_more4xop + cmpq $128,%rdx + jae .L128_or_more4xop + cmpq $64,%rdx + jae .L64_or_more4xop + + xorq %r9,%r9 + vmovdqa %xmm6,0(%rsp) + vmovdqa %xmm1,16(%rsp) + vmovdqa %xmm13,32(%rsp) + vmovdqa %xmm5,48(%rsp) + jmp .Loop_tail4xop + +.p2align 5 +.L64_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + je .Ldone4xop + + leaq 64(%rsi),%rsi + vmovdqa %xmm15,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm10,16(%rsp) + leaq 64(%rdi),%rdi + vmovdqa %xmm2,32(%rsp) + subq $64,%rdx + vmovdqa %xmm9,48(%rsp) + jmp .Loop_tail4xop + +.p2align 5 +.L128_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + je .Ldone4xop + + leaq 128(%rsi),%rsi + vmovdqa %xmm11,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm3,16(%rsp) + leaq 128(%rdi),%rdi + vmovdqa %xmm14,32(%rsp) + subq $128,%rdx + vmovdqa %xmm7,48(%rsp) + jmp .Loop_tail4xop + +.p2align 5 +.L192_or_more4xop: + vpxor 0(%rsi),%xmm6,%xmm6 + vpxor 16(%rsi),%xmm1,%xmm1 + vpxor 32(%rsi),%xmm13,%xmm13 + vpxor 48(%rsi),%xmm5,%xmm5 + vpxor 64(%rsi),%xmm15,%xmm15 + vpxor 80(%rsi),%xmm10,%xmm10 + vpxor 96(%rsi),%xmm2,%xmm2 + vpxor 112(%rsi),%xmm9,%xmm9 + leaq 128(%rsi),%rsi + vpxor 0(%rsi),%xmm11,%xmm11 + vpxor 16(%rsi),%xmm3,%xmm3 + vpxor 32(%rsi),%xmm14,%xmm14 + vpxor 48(%rsi),%xmm7,%xmm7 + + vmovdqu %xmm6,0(%rdi) + vmovdqu %xmm1,16(%rdi) + vmovdqu %xmm13,32(%rdi) + vmovdqu %xmm5,48(%rdi) + vmovdqu %xmm15,64(%rdi) + vmovdqu %xmm10,80(%rdi) + vmovdqu %xmm2,96(%rdi) + vmovdqu %xmm9,112(%rdi) + leaq 128(%rdi),%rdi + vmovdqu %xmm11,0(%rdi) + vmovdqu %xmm3,16(%rdi) + vmovdqu %xmm14,32(%rdi) + vmovdqu %xmm7,48(%rdi) + je .Ldone4xop + + leaq 64(%rsi),%rsi + vmovdqa %xmm8,0(%rsp) + xorq %r9,%r9 + vmovdqa %xmm0,16(%rsp) + leaq 64(%rdi),%rdi + vmovdqa %xmm12,32(%rsp) + subq $192,%rdx + vmovdqa %xmm4,48(%rsp) + +.Loop_tail4xop: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz .Loop_tail4xop + +.Ldone4xop: + vzeroupper + movaps -168(%r10),%xmm6 + movaps -152(%r10),%xmm7 + movaps -136(%r10),%xmm8 + movaps -120(%r10),%xmm9 + movaps -104(%r10),%xmm10 + movaps -88(%r10),%xmm11 + movaps -72(%r10),%xmm12 + movaps -56(%r10),%xmm13 + movaps -40(%r10),%xmm14 + movaps -24(%r10),%xmm15 + leaq (%r10),%rsp + +.L4xop_epilogue: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_chacha20_asm_4xop: +.def crypton_chacha20_asm_avx2; .scl 3; .type 32; .endef +.p2align 5 +crypton_chacha20_asm_avx2: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_chacha20_asm_avx2: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movq 40(%rsp),%r8 +.Lcrypton_chacha20_asm_8x: + movq %rsp,%r10 + + subq $0x280+168,%rsp + andq $-32,%rsp + movaps %xmm6,-168(%r10) + movaps %xmm7,-152(%r10) + movaps %xmm8,-136(%r10) + movaps %xmm9,-120(%r10) + movaps %xmm10,-104(%r10) + movaps %xmm11,-88(%r10) + movaps %xmm12,-72(%r10) + movaps %xmm13,-56(%r10) + movaps %xmm14,-40(%r10) + movaps %xmm15,-24(%r10) +.Lavx2_body: + vzeroupper + + + + + + + + + + + vbroadcasti128 .Lsigma(%rip),%ymm11 + vbroadcasti128 (%rcx),%ymm3 + vbroadcasti128 16(%rcx),%ymm15 + vbroadcasti128 (%r8),%ymm7 + leaq 256(%rsp),%rcx + leaq 512(%rsp),%rax + leaq .Lrot16(%rip),%r9 + leaq .Lrot24(%rip),%r11 + + vpshufd $0x00,%ymm11,%ymm8 + vpshufd $0x55,%ymm11,%ymm9 + vmovdqa %ymm8,128-256(%rcx) + vpshufd $0xaa,%ymm11,%ymm10 + vmovdqa %ymm9,160-256(%rcx) + vpshufd $0xff,%ymm11,%ymm11 + vmovdqa %ymm10,192-256(%rcx) + vmovdqa %ymm11,224-256(%rcx) + + vpshufd $0x00,%ymm3,%ymm0 + vpshufd $0x55,%ymm3,%ymm1 + vmovdqa %ymm0,256-256(%rcx) + vpshufd $0xaa,%ymm3,%ymm2 + vmovdqa %ymm1,288-256(%rcx) + vpshufd $0xff,%ymm3,%ymm3 + vmovdqa %ymm2,320-256(%rcx) + vmovdqa %ymm3,352-256(%rcx) + + vpshufd $0x00,%ymm15,%ymm12 + vpshufd $0x55,%ymm15,%ymm13 + vmovdqa %ymm12,384-512(%rax) + vpshufd $0xaa,%ymm15,%ymm14 + vmovdqa %ymm13,416-512(%rax) + vpshufd $0xff,%ymm15,%ymm15 + vmovdqa %ymm14,448-512(%rax) + vmovdqa %ymm15,480-512(%rax) + + vpshufd $0x00,%ymm7,%ymm4 + vpshufd $0x55,%ymm7,%ymm5 + vpaddd .Lincy(%rip),%ymm4,%ymm4 + vpshufd $0xaa,%ymm7,%ymm6 + vmovdqa %ymm5,544-512(%rax) + vpshufd $0xff,%ymm7,%ymm7 + vmovdqa %ymm6,576-512(%rax) + vmovdqa %ymm7,608-512(%rax) + + jmp .Loop_enter8x + +.p2align 5 +.Loop_outer8x: + vmovdqa 128-256(%rcx),%ymm8 + vmovdqa 160-256(%rcx),%ymm9 + vmovdqa 192-256(%rcx),%ymm10 + vmovdqa 224-256(%rcx),%ymm11 + vmovdqa 256-256(%rcx),%ymm0 + vmovdqa 288-256(%rcx),%ymm1 + vmovdqa 320-256(%rcx),%ymm2 + vmovdqa 352-256(%rcx),%ymm3 + vmovdqa 384-512(%rax),%ymm12 + vmovdqa 416-512(%rax),%ymm13 + vmovdqa 448-512(%rax),%ymm14 + vmovdqa 480-512(%rax),%ymm15 + vmovdqa 512-512(%rax),%ymm4 + vmovdqa 544-512(%rax),%ymm5 + vmovdqa 576-512(%rax),%ymm6 + vmovdqa 608-512(%rax),%ymm7 + vpaddd .Leight(%rip),%ymm4,%ymm4 + +.Loop_enter8x: + vmovdqa %ymm14,64(%rsp) + vmovdqa %ymm15,96(%rsp) + vbroadcasti128 (%r9),%ymm15 + vmovdqa %ymm4,512-512(%rax) + movl $10,%eax + jmp .Loop8x + +.p2align 5 +.Loop8x: + vpaddd %ymm0,%ymm8,%ymm8 + vpxor %ymm4,%ymm8,%ymm4 + vpshufb %ymm15,%ymm4,%ymm4 + vpaddd %ymm1,%ymm9,%ymm9 + vpxor %ymm5,%ymm9,%ymm5 + vpshufb %ymm15,%ymm5,%ymm5 + vpaddd %ymm4,%ymm12,%ymm12 + vpxor %ymm0,%ymm12,%ymm0 + vpslld $12,%ymm0,%ymm14 + vpsrld $20,%ymm0,%ymm0 + vpor %ymm0,%ymm14,%ymm0 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm5,%ymm13,%ymm13 + vpxor %ymm1,%ymm13,%ymm1 + vpslld $12,%ymm1,%ymm15 + vpsrld $20,%ymm1,%ymm1 + vpor %ymm1,%ymm15,%ymm1 + vpaddd %ymm0,%ymm8,%ymm8 + vpxor %ymm4,%ymm8,%ymm4 + vpshufb %ymm14,%ymm4,%ymm4 + vpaddd %ymm1,%ymm9,%ymm9 + vpxor %ymm5,%ymm9,%ymm5 + vpshufb %ymm14,%ymm5,%ymm5 + vpaddd %ymm4,%ymm12,%ymm12 + vpxor %ymm0,%ymm12,%ymm0 + vpslld $7,%ymm0,%ymm15 + vpsrld $25,%ymm0,%ymm0 + vpor %ymm0,%ymm15,%ymm0 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm5,%ymm13,%ymm13 + vpxor %ymm1,%ymm13,%ymm1 + vpslld $7,%ymm1,%ymm14 + vpsrld $25,%ymm1,%ymm1 + vpor %ymm1,%ymm14,%ymm1 + vmovdqa %ymm12,0(%rsp) + vmovdqa %ymm13,32(%rsp) + vmovdqa 64(%rsp),%ymm12 + vmovdqa 96(%rsp),%ymm13 + vpaddd %ymm2,%ymm10,%ymm10 + vpxor %ymm6,%ymm10,%ymm6 + vpshufb %ymm15,%ymm6,%ymm6 + vpaddd %ymm3,%ymm11,%ymm11 + vpxor %ymm7,%ymm11,%ymm7 + vpshufb %ymm15,%ymm7,%ymm7 + vpaddd %ymm6,%ymm12,%ymm12 + vpxor %ymm2,%ymm12,%ymm2 + vpslld $12,%ymm2,%ymm14 + vpsrld $20,%ymm2,%ymm2 + vpor %ymm2,%ymm14,%ymm2 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm7,%ymm13,%ymm13 + vpxor %ymm3,%ymm13,%ymm3 + vpslld $12,%ymm3,%ymm15 + vpsrld $20,%ymm3,%ymm3 + vpor %ymm3,%ymm15,%ymm3 + vpaddd %ymm2,%ymm10,%ymm10 + vpxor %ymm6,%ymm10,%ymm6 + vpshufb %ymm14,%ymm6,%ymm6 + vpaddd %ymm3,%ymm11,%ymm11 + vpxor %ymm7,%ymm11,%ymm7 + vpshufb %ymm14,%ymm7,%ymm7 + vpaddd %ymm6,%ymm12,%ymm12 + vpxor %ymm2,%ymm12,%ymm2 + vpslld $7,%ymm2,%ymm15 + vpsrld $25,%ymm2,%ymm2 + vpor %ymm2,%ymm15,%ymm2 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm7,%ymm13,%ymm13 + vpxor %ymm3,%ymm13,%ymm3 + vpslld $7,%ymm3,%ymm14 + vpsrld $25,%ymm3,%ymm3 + vpor %ymm3,%ymm14,%ymm3 + vpaddd %ymm1,%ymm8,%ymm8 + vpxor %ymm7,%ymm8,%ymm7 + vpshufb %ymm15,%ymm7,%ymm7 + vpaddd %ymm2,%ymm9,%ymm9 + vpxor %ymm4,%ymm9,%ymm4 + vpshufb %ymm15,%ymm4,%ymm4 + vpaddd %ymm7,%ymm12,%ymm12 + vpxor %ymm1,%ymm12,%ymm1 + vpslld $12,%ymm1,%ymm14 + vpsrld $20,%ymm1,%ymm1 + vpor %ymm1,%ymm14,%ymm1 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm4,%ymm13,%ymm13 + vpxor %ymm2,%ymm13,%ymm2 + vpslld $12,%ymm2,%ymm15 + vpsrld $20,%ymm2,%ymm2 + vpor %ymm2,%ymm15,%ymm2 + vpaddd %ymm1,%ymm8,%ymm8 + vpxor %ymm7,%ymm8,%ymm7 + vpshufb %ymm14,%ymm7,%ymm7 + vpaddd %ymm2,%ymm9,%ymm9 + vpxor %ymm4,%ymm9,%ymm4 + vpshufb %ymm14,%ymm4,%ymm4 + vpaddd %ymm7,%ymm12,%ymm12 + vpxor %ymm1,%ymm12,%ymm1 + vpslld $7,%ymm1,%ymm15 + vpsrld $25,%ymm1,%ymm1 + vpor %ymm1,%ymm15,%ymm1 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm4,%ymm13,%ymm13 + vpxor %ymm2,%ymm13,%ymm2 + vpslld $7,%ymm2,%ymm14 + vpsrld $25,%ymm2,%ymm2 + vpor %ymm2,%ymm14,%ymm2 + vmovdqa %ymm12,64(%rsp) + vmovdqa %ymm13,96(%rsp) + vmovdqa 0(%rsp),%ymm12 + vmovdqa 32(%rsp),%ymm13 + vpaddd %ymm3,%ymm10,%ymm10 + vpxor %ymm5,%ymm10,%ymm5 + vpshufb %ymm15,%ymm5,%ymm5 + vpaddd %ymm0,%ymm11,%ymm11 + vpxor %ymm6,%ymm11,%ymm6 + vpshufb %ymm15,%ymm6,%ymm6 + vpaddd %ymm5,%ymm12,%ymm12 + vpxor %ymm3,%ymm12,%ymm3 + vpslld $12,%ymm3,%ymm14 + vpsrld $20,%ymm3,%ymm3 + vpor %ymm3,%ymm14,%ymm3 + vbroadcasti128 (%r11),%ymm14 + vpaddd %ymm6,%ymm13,%ymm13 + vpxor %ymm0,%ymm13,%ymm0 + vpslld $12,%ymm0,%ymm15 + vpsrld $20,%ymm0,%ymm0 + vpor %ymm0,%ymm15,%ymm0 + vpaddd %ymm3,%ymm10,%ymm10 + vpxor %ymm5,%ymm10,%ymm5 + vpshufb %ymm14,%ymm5,%ymm5 + vpaddd %ymm0,%ymm11,%ymm11 + vpxor %ymm6,%ymm11,%ymm6 + vpshufb %ymm14,%ymm6,%ymm6 + vpaddd %ymm5,%ymm12,%ymm12 + vpxor %ymm3,%ymm12,%ymm3 + vpslld $7,%ymm3,%ymm15 + vpsrld $25,%ymm3,%ymm3 + vpor %ymm3,%ymm15,%ymm3 + vbroadcasti128 (%r9),%ymm15 + vpaddd %ymm6,%ymm13,%ymm13 + vpxor %ymm0,%ymm13,%ymm0 + vpslld $7,%ymm0,%ymm14 + vpsrld $25,%ymm0,%ymm0 + vpor %ymm0,%ymm14,%ymm0 + decl %eax + jnz .Loop8x + + leaq 512(%rsp),%rax + vpaddd 128-256(%rcx),%ymm8,%ymm8 + vpaddd 160-256(%rcx),%ymm9,%ymm9 + vpaddd 192-256(%rcx),%ymm10,%ymm10 + vpaddd 224-256(%rcx),%ymm11,%ymm11 + + vpunpckldq %ymm9,%ymm8,%ymm14 + vpunpckldq %ymm11,%ymm10,%ymm15 + vpunpckhdq %ymm9,%ymm8,%ymm8 + vpunpckhdq %ymm11,%ymm10,%ymm10 + vpunpcklqdq %ymm15,%ymm14,%ymm9 + vpunpckhqdq %ymm15,%ymm14,%ymm14 + vpunpcklqdq %ymm10,%ymm8,%ymm11 + vpunpckhqdq %ymm10,%ymm8,%ymm8 + vpaddd 256-256(%rcx),%ymm0,%ymm0 + vpaddd 288-256(%rcx),%ymm1,%ymm1 + vpaddd 320-256(%rcx),%ymm2,%ymm2 + vpaddd 352-256(%rcx),%ymm3,%ymm3 + + vpunpckldq %ymm1,%ymm0,%ymm10 + vpunpckldq %ymm3,%ymm2,%ymm15 + vpunpckhdq %ymm1,%ymm0,%ymm0 + vpunpckhdq %ymm3,%ymm2,%ymm2 + vpunpcklqdq %ymm15,%ymm10,%ymm1 + vpunpckhqdq %ymm15,%ymm10,%ymm10 + vpunpcklqdq %ymm2,%ymm0,%ymm3 + vpunpckhqdq %ymm2,%ymm0,%ymm0 + vperm2i128 $0x20,%ymm1,%ymm9,%ymm15 + vperm2i128 $0x31,%ymm1,%ymm9,%ymm1 + vperm2i128 $0x20,%ymm10,%ymm14,%ymm9 + vperm2i128 $0x31,%ymm10,%ymm14,%ymm10 + vperm2i128 $0x20,%ymm3,%ymm11,%ymm14 + vperm2i128 $0x31,%ymm3,%ymm11,%ymm3 + vperm2i128 $0x20,%ymm0,%ymm8,%ymm11 + vperm2i128 $0x31,%ymm0,%ymm8,%ymm0 + vmovdqa %ymm15,0(%rsp) + vmovdqa %ymm9,32(%rsp) + vmovdqa 64(%rsp),%ymm15 + vmovdqa 96(%rsp),%ymm9 + + vpaddd 384-512(%rax),%ymm12,%ymm12 + vpaddd 416-512(%rax),%ymm13,%ymm13 + vpaddd 448-512(%rax),%ymm15,%ymm15 + vpaddd 480-512(%rax),%ymm9,%ymm9 + + vpunpckldq %ymm13,%ymm12,%ymm2 + vpunpckldq %ymm9,%ymm15,%ymm8 + vpunpckhdq %ymm13,%ymm12,%ymm12 + vpunpckhdq %ymm9,%ymm15,%ymm15 + vpunpcklqdq %ymm8,%ymm2,%ymm13 + vpunpckhqdq %ymm8,%ymm2,%ymm2 + vpunpcklqdq %ymm15,%ymm12,%ymm9 + vpunpckhqdq %ymm15,%ymm12,%ymm12 + vpaddd 512-512(%rax),%ymm4,%ymm4 + vpaddd 544-512(%rax),%ymm5,%ymm5 + vpaddd 576-512(%rax),%ymm6,%ymm6 + vpaddd 608-512(%rax),%ymm7,%ymm7 + + vpunpckldq %ymm5,%ymm4,%ymm15 + vpunpckldq %ymm7,%ymm6,%ymm8 + vpunpckhdq %ymm5,%ymm4,%ymm4 + vpunpckhdq %ymm7,%ymm6,%ymm6 + vpunpcklqdq %ymm8,%ymm15,%ymm5 + vpunpckhqdq %ymm8,%ymm15,%ymm15 + vpunpcklqdq %ymm6,%ymm4,%ymm7 + vpunpckhqdq %ymm6,%ymm4,%ymm4 + vperm2i128 $0x20,%ymm5,%ymm13,%ymm8 + vperm2i128 $0x31,%ymm5,%ymm13,%ymm5 + vperm2i128 $0x20,%ymm15,%ymm2,%ymm13 + vperm2i128 $0x31,%ymm15,%ymm2,%ymm15 + vperm2i128 $0x20,%ymm7,%ymm9,%ymm2 + vperm2i128 $0x31,%ymm7,%ymm9,%ymm7 + vperm2i128 $0x20,%ymm4,%ymm12,%ymm9 + vperm2i128 $0x31,%ymm4,%ymm12,%ymm4 + vmovdqa 0(%rsp),%ymm6 + vmovdqa 32(%rsp),%ymm12 + + cmpq $512,%rdx + jb .Ltail8x + + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + leaq 128(%rsi),%rsi + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm12,%ymm12 + vpxor 32(%rsi),%ymm13,%ymm13 + vpxor 64(%rsi),%ymm10,%ymm10 + vpxor 96(%rsi),%ymm15,%ymm15 + leaq 128(%rsi),%rsi + vmovdqu %ymm12,0(%rdi) + vmovdqu %ymm13,32(%rdi) + vmovdqu %ymm10,64(%rdi) + vmovdqu %ymm15,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm14,%ymm14 + vpxor 32(%rsi),%ymm2,%ymm2 + vpxor 64(%rsi),%ymm3,%ymm3 + vpxor 96(%rsi),%ymm7,%ymm7 + leaq 128(%rsi),%rsi + vmovdqu %ymm14,0(%rdi) + vmovdqu %ymm2,32(%rdi) + vmovdqu %ymm3,64(%rdi) + vmovdqu %ymm7,96(%rdi) + leaq 128(%rdi),%rdi + + vpxor 0(%rsi),%ymm11,%ymm11 + vpxor 32(%rsi),%ymm9,%ymm9 + vpxor 64(%rsi),%ymm0,%ymm0 + vpxor 96(%rsi),%ymm4,%ymm4 + leaq 128(%rsi),%rsi + vmovdqu %ymm11,0(%rdi) + vmovdqu %ymm9,32(%rdi) + vmovdqu %ymm0,64(%rdi) + vmovdqu %ymm4,96(%rdi) + leaq 128(%rdi),%rdi + + subq $512,%rdx + jnz .Loop_outer8x + + jmp .Ldone8x + +.Ltail8x: + cmpq $448,%rdx + jae .L448_or_more8x + cmpq $384,%rdx + jae .L384_or_more8x + cmpq $320,%rdx + jae .L320_or_more8x + cmpq $256,%rdx + jae .L256_or_more8x + cmpq $192,%rdx + jae .L192_or_more8x + cmpq $128,%rdx + jae .L128_or_more8x + cmpq $64,%rdx + jae .L64_or_more8x + + xorq %r9,%r9 + vmovdqa %ymm6,0(%rsp) + vmovdqa %ymm8,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L64_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + je .Ldone8x + + leaq 64(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm1,0(%rsp) + leaq 64(%rdi),%rdi + subq $64,%rdx + vmovdqa %ymm5,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L128_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + je .Ldone8x + + leaq 128(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm12,0(%rsp) + leaq 128(%rdi),%rdi + subq $128,%rdx + vmovdqa %ymm13,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L192_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + je .Ldone8x + + leaq 192(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm10,0(%rsp) + leaq 192(%rdi),%rdi + subq $192,%rdx + vmovdqa %ymm15,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L256_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + je .Ldone8x + + leaq 256(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm14,0(%rsp) + leaq 256(%rdi),%rdi + subq $256,%rdx + vmovdqa %ymm2,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L320_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + je .Ldone8x + + leaq 320(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm3,0(%rsp) + leaq 320(%rdi),%rdi + subq $320,%rdx + vmovdqa %ymm7,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L384_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vpxor 320(%rsi),%ymm3,%ymm3 + vpxor 352(%rsi),%ymm7,%ymm7 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + vmovdqu %ymm3,320(%rdi) + vmovdqu %ymm7,352(%rdi) + je .Ldone8x + + leaq 384(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm11,0(%rsp) + leaq 384(%rdi),%rdi + subq $384,%rdx + vmovdqa %ymm9,32(%rsp) + jmp .Loop_tail8x + +.p2align 5 +.L448_or_more8x: + vpxor 0(%rsi),%ymm6,%ymm6 + vpxor 32(%rsi),%ymm8,%ymm8 + vpxor 64(%rsi),%ymm1,%ymm1 + vpxor 96(%rsi),%ymm5,%ymm5 + vpxor 128(%rsi),%ymm12,%ymm12 + vpxor 160(%rsi),%ymm13,%ymm13 + vpxor 192(%rsi),%ymm10,%ymm10 + vpxor 224(%rsi),%ymm15,%ymm15 + vpxor 256(%rsi),%ymm14,%ymm14 + vpxor 288(%rsi),%ymm2,%ymm2 + vpxor 320(%rsi),%ymm3,%ymm3 + vpxor 352(%rsi),%ymm7,%ymm7 + vpxor 384(%rsi),%ymm11,%ymm11 + vpxor 416(%rsi),%ymm9,%ymm9 + vmovdqu %ymm6,0(%rdi) + vmovdqu %ymm8,32(%rdi) + vmovdqu %ymm1,64(%rdi) + vmovdqu %ymm5,96(%rdi) + vmovdqu %ymm12,128(%rdi) + vmovdqu %ymm13,160(%rdi) + vmovdqu %ymm10,192(%rdi) + vmovdqu %ymm15,224(%rdi) + vmovdqu %ymm14,256(%rdi) + vmovdqu %ymm2,288(%rdi) + vmovdqu %ymm3,320(%rdi) + vmovdqu %ymm7,352(%rdi) + vmovdqu %ymm11,384(%rdi) + vmovdqu %ymm9,416(%rdi) + je .Ldone8x + + leaq 448(%rsi),%rsi + xorq %r9,%r9 + vmovdqa %ymm0,0(%rsp) + leaq 448(%rdi),%rdi + subq $448,%rdx + vmovdqa %ymm4,32(%rsp) + +.Loop_tail8x: + movzbl (%rsi,%r9,1),%eax + movzbl (%rsp,%r9,1),%ecx + leaq 1(%r9),%r9 + xorl %ecx,%eax + movb %al,-1(%rdi,%r9,1) + decq %rdx + jnz .Loop_tail8x + +.Ldone8x: + vzeroall + movaps -168(%r10),%xmm6 + movaps -152(%r10),%xmm7 + movaps -136(%r10),%xmm8 + movaps -120(%r10),%xmm9 + movaps -104(%r10),%xmm10 + movaps -88(%r10),%xmm11 + movaps -72(%r10),%xmm12 + movaps -56(%r10),%xmm13 + movaps -40(%r10),%xmm14 + movaps -24(%r10),%xmm15 + leaq (%r10),%rsp + +.Lavx2_epilogue: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_chacha20_asm_avx2: + +.def se_handler; .scl 3; .type 32; .endef +.p2align 4 +se_handler: + .byte 0xf3,0x0f,0x1e,0xfa + + pushq %rsi + pushq %rdi + pushq %rbx + pushq %rbp + pushq %r12 + pushq %r13 + pushq %r14 + pushq %r15 + pushfq + subq $64,%rsp + + movq 120(%r8),%rax + movq 248(%r8),%rbx + + movq 8(%r9),%rsi + movq 56(%r9),%r11 + + leaq .Lctr32_body(%rip),%r10 + cmpq %r10,%rbx + jb .Lcommon_seh_tail + + movq 152(%r8),%rax + + leaq .Lno_data(%rip),%r10 + cmpq %r10,%rbx + jae .Lcommon_seh_tail + + leaq 64+24+48(%rax),%rax + + movq -8(%rax),%rbx + movq -16(%rax),%rbp + movq -24(%rax),%r12 + movq -32(%rax),%r13 + movq -40(%rax),%r14 + movq -48(%rax),%r15 + movq %rbx,144(%r8) + movq %rbp,160(%r8) + movq %r12,216(%r8) + movq %r13,224(%r8) + movq %r14,232(%r8) + movq %r15,240(%r8) + +.Lcommon_seh_tail: + movq 8(%rax),%rdi + movq 16(%rax),%rsi + movq %rax,152(%r8) + movq %rsi,168(%r8) + movq %rdi,176(%r8) + + movq 40(%r9),%rdi + movq %r8,%rsi + movl $154,%ecx +.long 0xa548f3fc + + movq %r9,%rsi + xorq %rcx,%rcx + movq 8(%rsi),%rdx + movq 0(%rsi),%r8 + movq 16(%rsi),%r9 + movq 40(%rsi),%r10 + leaq 56(%rsi),%r11 + leaq 24(%rsi),%r12 + movq %r10,32(%rsp) + movq %r11,40(%rsp) + movq %r12,48(%rsp) + movq %rcx,56(%rsp) + call *__imp_RtlVirtualUnwind(%rip) + + movl $1,%eax + addq $64,%rsp + popfq + popq %r15 + popq %r14 + popq %r13 + popq %r12 + popq %rbp + popq %rbx + popq %rdi + popq %rsi + .byte 0xf3,0xc3 + + +.def simd_handler; .scl 3; .type 32; .endef +.p2align 4 +simd_handler: + .byte 0xf3,0x0f,0x1e,0xfa + + pushq %rsi + pushq %rdi + pushq %rbx + pushq %rbp + pushq %r12 + pushq %r13 + pushq %r14 + pushq %r15 + pushfq + subq $64,%rsp + + movq 120(%r8),%rax + movq 248(%r8),%rbx + + movq 8(%r9),%rsi + movq 56(%r9),%r11 + + movl 0(%r11),%r10d + leaq (%rsi,%r10,1),%r10 + cmpq %r10,%rbx + jb .Lcommon_seh_tail + + movq 200(%r8),%rax + + movl 4(%r11),%r10d + movl 8(%r11),%ecx + leaq (%rsi,%r10,1),%r10 + cmpq %r10,%rbx + jae .Lcommon_seh_tail + + negq %rcx + leaq -8(%rax,%rcx,1),%rsi + leaq 512(%r8),%rdi + negl %ecx + shrl $3,%ecx +.long 0xa548f3fc + + jmp .Lcommon_seh_tail + + +.section .pdata +.p2align 2 +.rva .LSEH_begin_crypton_chacha20_asm_ctr32 +.rva .LSEH_end_crypton_chacha20_asm_ctr32 +.rva .LSEH_info_crypton_chacha20_asm_ctr32 + +.rva .LSEH_begin_crypton_chacha20_asm_ssse3 +.rva .LSEH_end_crypton_chacha20_asm_ssse3 +.rva .LSEH_info_crypton_chacha20_asm_ssse3 + +.rva .LSEH_begin_crypton_chacha20_asm_128 +.rva .LSEH_end_crypton_chacha20_asm_128 +.rva .LSEH_info_crypton_chacha20_asm_128 + +.rva .LSEH_begin_crypton_chacha20_asm_4x +.rva .LSEH_end_crypton_chacha20_asm_4x +.rva .LSEH_info_crypton_chacha20_asm_4x +.rva .LSEH_begin_crypton_chacha20_asm_4xop +.rva .LSEH_end_crypton_chacha20_asm_4xop +.rva .LSEH_info_crypton_chacha20_asm_4xop +.rva .LSEH_begin_crypton_chacha20_asm_avx2 +.rva .LSEH_end_crypton_chacha20_asm_avx2 +.rva .LSEH_info_crypton_chacha20_asm_avx2 +.section .xdata +.p2align 3 +.LSEH_info_crypton_chacha20_asm_ctr32: +.byte 9,0,0,0 +.rva se_handler + +.LSEH_info_crypton_chacha20_asm_ssse3: +.byte 9,0,0,0 +.rva simd_handler +.rva .Lssse3_body,.Lssse3_epilogue +.long 0x20,0 + +.LSEH_info_crypton_chacha20_asm_128: +.byte 9,0,0,0 +.rva simd_handler +.rva .L128_body,.L128_epilogue +.long 0x60,0 + +.LSEH_info_crypton_chacha20_asm_4x: +.byte 9,0,0,0 +.rva simd_handler +.rva .L4x_body,.L4x_epilogue +.long 0xa0,0 +.LSEH_info_crypton_chacha20_asm_4xop: +.byte 9,0,0,0 +.rva simd_handler +.rva .L4xop_body,.L4xop_epilogue +.long 0xa0,0 +.LSEH_info_crypton_chacha20_asm_avx2: +.byte 9,0,0,0 +.rva simd_handler +.rva .Lavx2_body,.Lavx2_epilogue +.long 0xa0,0 diff --git a/cbits/asm/chacha-x86_64.pl b/cbits/asm/chacha-x86_64.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/chacha-x86_64.pl @@ -0,0 +1,4044 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# November 2014 +# +# ChaCha20 for x86_64. +# +# December 2016 +# +# Add AVX512F code path. +# +# December 2017 +# +# Add AVX512VL code path. +# +# Performance in cycles per byte out of large buffer. +# +# IALU/gcc 4.8(i) 1x/2xSSSE3(ii) 4xSSSE3 NxAVX(v) +# +# P4 9.48/+99% - - +# Core2 7.83/+55% 7.90/5.76 4.35 +# Westmere 7.19/+50% 5.60/4.50 3.00 +# Sandy Bridge 8.31/+42% 5.45/4.00 2.72 +# Ivy Bridge 6.71/+46% 5.40/? 2.41 +# Haswell 5.92/+43% 5.20/3.45 2.42 1.23 +# Skylake[-X] 5.87/+39% 4.70/3.22 2.31 1.19[0.80(vi)] +# Cannon Lake 5.87/+39% 4.60/3.20 2.26 0.80(vi) +# Rocket Lake 5.86/+39% ? 2.30 0.58 +# Silvermont 12.0/+33% 7.75/6.90 7.03(iii) +# Knights L 11.7/- ? 9.60(iii) 0.80 +# Goldmont 10.6/+17% 5.10/3.52 3.28 +# Sledgehammer 7.28/+52% - - +# Bulldozer 9.66/+28% 9.85/5.35(iv) 3.06(iv) +# Ryzen 5.96/+50% 5.19/3.00 2.40 2.09 +# VIA Nano 10.5/+46% 6.72/6.88 6.05 +# +# (i) compared to older gcc 3.x one can observe >2x improvement on +# most platforms; +# (ii) 2xSSSE3 is code path optimized specifically for 128 bytes used +# by chacha20_poly1305_tls_cipher, results are EVP-free; +# (iii) this is not optimal result for Atom because of MSROM +# limitations, SSE2 can do better, but gain is considered too +# low to justify the [maintenance] effort; +# (iv) Bulldozer actually executes 4xXOP code path that delivers 2.20 +# and 4.85 for 128-byte inputs; +# (v) 8xAVX2, 8xAVX512VL or 16xAVX512F, whichever best applicable; +# (vi) even though Skylake-X can execute AVX512F code and deliver 0.57 +# cpb in single thread, the corresponding capability is suppressed; + +$flavour = shift; +$output = shift; +if ($flavour =~ /\./) { $output = $flavour; undef $flavour; } + +$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/); + +$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; +( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or +( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or +die "can't locate x86_64-xlate.pl"; + +$avx=undef; + +if (!defined($avx) && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) && + ($ENV{ASM} //= "nasm") && + `"$ENV{ASM}" -v 2>&1` =~ /NASM version ([0-9]+\.[0-9]+)(?:\.([0-9]+))?/) { + $avx = ($1>=2.09) + ($1>=2.10) + ($1>=2.12); + $avx += 1 if ($1==2.11 && $2>=8); +} + +if (!defined($avx) && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) && + ($ENV{ASM} //= "ml64") && + `"$ENV{ASM}" 2>&1` =~ /Version ([0-9]+)\./) { + $avx = ($1>=10) + ($1>=11) + ($1>=14); +} + +$ENV{CC} //= "cc"; +if (!defined($avx) && `$ENV{CC} -Wa,-v -c -o /dev/zero -x assembler /dev/null 2>&1` + =~ /GNU assembler version ([0-9]+)\.([0-9]+)/) { + my $ver = $1 + $2/100.0; # 3.1->3.01, 3.10->3.10 + $avx = ($ver>=2.19) + ($ver>=2.22) + ($ver>=2.25); +} + +if (!defined($avx) && `$ENV{CC} -v 2>&1` + =~ /((?:^clang|LLVM) version|.*based on LLVM) ([0-9]+)\.([0-9]+)/) { + my $ver = $2 + $3/100.0; # 3.1->3.01, 3.10->3.10 + $avx = ($ver>=3.0) + ($ver>3.0); + $avx += ($ver>=7.0) if ($1 =~ /^clang/); +} + +open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\""; +*STDOUT=*OUT; + +# input parameter block +($out,$inp,$len,$key,$counter)=("%rdi","%rsi","%rdx","%rcx","%r8"); + +$code.=<<___; +.text + +.extern OPENSSL_ia32cap_P + +.align 64 +.Lzero: +.long 0,0,0,0 +.Lone: +.long 1,0,0,0 +.Linc: +.long 0,1,2,3 +.Lfour: +.long 4,4,4,4 +.Lincy: +.long 0,2,4,6,1,3,5,7 +.Leight: +.long 8,8,8,8,8,8,8,8 +.Lrot16: +.byte 0x2,0x3,0x0,0x1, 0x6,0x7,0x4,0x5, 0xa,0xb,0x8,0x9, 0xe,0xf,0xc,0xd +.Lrot24: +.byte 0x3,0x0,0x1,0x2, 0x7,0x4,0x5,0x6, 0xb,0x8,0x9,0xa, 0xf,0xc,0xd,0xe +.Ltwoy: +.long 2,0,0,0, 2,0,0,0 +.align 64 +.Lzeroz: +.long 0,0,0,0, 1,0,0,0, 2,0,0,0, 3,0,0,0 +.Lfourz: +.long 4,0,0,0, 4,0,0,0, 4,0,0,0, 4,0,0,0 +.Lincz: +.long 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15 +.Lsixteen: +.long 16,16,16,16,16,16,16,16,16,16,16,16,16,16,16,16 +.Lsigma: +.asciz "expand 32-byte k" +.asciz "ChaCha20 for x86_64, CRYPTOGAMS by \@dot-asm" +___ + +sub AUTOLOAD() # thunk [simplified] 32-bit style perlasm +{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://; + my $arg = pop; + $arg = "\$$arg" if ($arg*1 eq $arg); + $code .= "\t$opcode\t".join(',',$arg,reverse @_)."\n"; +} + +@x=("%eax","%ebx","%ecx","%edx",map("%r${_}d",(8..11)), + "%nox","%nox","%nox","%nox",map("%r${_}d",(12..15))); +@t=("%esi","%edi"); + +sub ROUND { # critical path is 24 cycles per round +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); +my ($xc,$xc_)=map("\"$_\"",@t); +my @x=map("\"$_\"",@x); + + # Consider order in which variables are addressed by their + # index: + # + # a b c d + # + # 0 4 8 12 < even round + # 1 5 9 13 + # 2 6 10 14 + # 3 7 11 15 + # 0 5 10 15 < odd round + # 1 6 11 12 + # 2 7 8 13 + # 3 4 9 14 + # + # 'a', 'b' and 'd's are permanently allocated in registers, + # @x[0..7,12..15], while 'c's are maintained in memory. If + # you observe 'c' column, you'll notice that pair of 'c's is + # invariant between rounds. This means that we have to reload + # them once per round, in the middle. This is why you'll see + # bunch of 'c' stores and loads in the middle, but none in + # the beginning or end. + + # Normally instructions would be interleaved to favour in-order + # execution. Generally out-of-order cores manage it gracefully, + # but not this time for some reason. As in-order execution + # cores are dying breed, old Atom is the only one around, + # instructions are left uninterleaved. Besides, Atom is better + # off executing 1xSSSE3 code anyway... + + ( + "&add (@x[$a0],@x[$b0])", # Q1 + "&xor (@x[$d0],@x[$a0])", + "&rol (@x[$d0],16)", + "&add (@x[$a1],@x[$b1])", # Q2 + "&xor (@x[$d1],@x[$a1])", + "&rol (@x[$d1],16)", + + "&add ($xc,@x[$d0])", + "&xor (@x[$b0],$xc)", + "&rol (@x[$b0],12)", + "&add ($xc_,@x[$d1])", + "&xor (@x[$b1],$xc_)", + "&rol (@x[$b1],12)", + + "&add (@x[$a0],@x[$b0])", + "&xor (@x[$d0],@x[$a0])", + "&rol (@x[$d0],8)", + "&add (@x[$a1],@x[$b1])", + "&xor (@x[$d1],@x[$a1])", + "&rol (@x[$d1],8)", + + "&add ($xc,@x[$d0])", + "&xor (@x[$b0],$xc)", + "&rol (@x[$b0],7)", + "&add ($xc_,@x[$d1])", + "&xor (@x[$b1],$xc_)", + "&rol (@x[$b1],7)", + + "&mov (\"4*$c0(%rsp)\",$xc)", # reload pair of 'c's + "&mov (\"4*$c1(%rsp)\",$xc_)", + "&mov ($xc,\"4*$c2(%rsp)\")", + "&mov ($xc_,\"4*$c3(%rsp)\")", + + "&add (@x[$a2],@x[$b2])", # Q3 + "&xor (@x[$d2],@x[$a2])", + "&rol (@x[$d2],16)", + "&add (@x[$a3],@x[$b3])", # Q4 + "&xor (@x[$d3],@x[$a3])", + "&rol (@x[$d3],16)", + + "&add ($xc,@x[$d2])", + "&xor (@x[$b2],$xc)", + "&rol (@x[$b2],12)", + "&add ($xc_,@x[$d3])", + "&xor (@x[$b3],$xc_)", + "&rol (@x[$b3],12)", + + "&add (@x[$a2],@x[$b2])", + "&xor (@x[$d2],@x[$a2])", + "&rol (@x[$d2],8)", + "&add (@x[$a3],@x[$b3])", + "&xor (@x[$d3],@x[$a3])", + "&rol (@x[$d3],8)", + + "&add ($xc,@x[$d2])", + "&xor (@x[$b2],$xc)", + "&rol (@x[$b2],7)", + "&add ($xc_,@x[$d3])", + "&xor (@x[$b3],$xc_)", + "&rol (@x[$b3],7)" + ); +} + +######################################################################## +# Generic code path that handles all lengths on pre-SSSE3 processors. +$code.=<<___; +.globl ChaCha20_ctr32 +.type ChaCha20_ctr32,\@function,5 +.align 64 +ChaCha20_ctr32: +.cfi_startproc + cmp \$0,$len + je .Lno_data +___ + if ($flavour !~ /kernel/) { +$code.=<<___; + mov OPENSSL_ia32cap_P+4(%rip),%r9 +___ +$code.=<<___ if ($avx>2); + bt \$48,%r9 # check for AVX512F + jc .LChaCha20_avx512 + test %r9,%r9 # check for AVX512VL + js .LChaCha20_avx512vl +___ +$code.=<<___; + test \$`1<<(41-32)`,%r9d + jnz .LChaCha20_ssse3 +___ + } +$code.=<<___; + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + sub \$64+24,%rsp +.cfi_adjust_cfa_offset 64+24 +.Lctr32_body: + + mov $len,%rbp # reassign $len + + mov 0($key),%r12 # copy key and counter to stack + mov 8($key),%r13 + mov 16($key),%r14 + mov 24($key),%r15 + mov 0($counter),%rax + mov 8($counter),%rdx + mov %r12,4*4(%rsp) + mov %r13,4*6(%rsp) + mov %r14,4*0(%rsp) + mov %r15,4*2(%rsp) + mov %rax,4*12(%rsp) + mov %rdx,4*14(%rsp) + jmp .Loop_outer + +.align 32 +.Loop_outer: + mov \$0x61707865,@x[0] # 'expa' + mov \$0x3320646e,@x[1] # 'nd 3' + mov \$0x79622d32,@x[2] # '2-by' + mov \$0x6b206574,@x[3] # 'te k' + mov 4*4(%rsp),@x[4] + mov 4*5(%rsp),@x[5] + mov 4*6(%rsp),@x[6] + mov 4*7(%rsp),@x[7] + mov 4*12(%rsp),@x[12] + mov 4*13(%rsp),@x[13] + mov 4*14(%rsp),@x[14] + mov %r15,4*10(%rsp) # "@x[10]:@x[11]" + mov 4*15(%rsp),@x[15] + + mov %rbp,64+0(%rsp) # save len + mov $inp,64+8(%rsp) # save inp + mov 0(%rsp),@t[0] # "@x[8]" + mov $out,64+16(%rsp) # save out + mov 4(%rsp),@t[1] # "@x[9]" + mov \$10,%ebp + jmp .Loop + +.align 32 +.Loop: +___ + foreach (&ROUND (0, 4, 8,12)) { eval; } + foreach (&ROUND (0, 5,10,15)) { eval; } + &dec ("%ebp"); + &jnz (".Loop"); + +$code.=<<___; + add 4*0(%rsp),@t[0] # modulo-scheduled + add 4*1(%rsp),@t[1] + mov 64(%rsp),%rbp # load len + mov @t[0],4*8(%rsp) + mov 64+8(%rsp),$inp # load inp + mov @t[1],4*9(%rsp) + mov 64+16(%rsp),$out # load out + + add \$0x61707865,@x[0] # 'expa' + add \$0x3320646e,@x[1] # 'nd 3' + add \$0x79622d32,@x[2] # '2-by' + add \$0x6b206574,@x[3] # 'te k' + add 4*4(%rsp),@x[4] + add 4*5(%rsp),@x[5] + add 4*6(%rsp),@x[6] + add 4*7(%rsp),@x[7] + add 4*12(%rsp),@x[12] + add 4*13(%rsp),@x[13] + add 4*14(%rsp),@x[14] + add 4*15(%rsp),@x[15] + + cmp \$64,%rbp + jb .Ltail + + xor 4*0($inp),@x[0] # xor with input + xor 4*1($inp),@x[1] + xor 4*2($inp),@x[2] + xor 4*3($inp),@x[3] + mov @x[0],4*0($out) # write output + mov 4*8(%rsp),@x[0] # load @x[8]-@x[11] + mov @x[1],4*1($out) + mov 4*9(%rsp),@x[1] + mov @x[2],4*2($out) + mov 4*10(%rsp),@x[2] + mov @x[3],4*3($out) + mov 4*11(%rsp),@x[3] + xor 4*4($inp),@x[4] + add 4*2(%rsp),@x[2] + xor 4*5($inp),@x[5] + add 4*3(%rsp),@x[3] + xor 4*6($inp),@x[6] + xor 4*7($inp),@x[7] + xor 4*8($inp),@x[0] + xor 4*9($inp),@x[1] + xor 4*10($inp),@x[2] + xor 4*11($inp),@x[3] + xor 4*12($inp),@x[12] + xor 4*13($inp),@x[13] + xor 4*14($inp),@x[14] + xor 4*15($inp),@x[15] + lea 4*16($inp),$inp # inp+=64 + + addl \$1,4*12(%rsp) # increment counter + + mov @x[4],4*4($out) + mov @x[5],4*5($out) + mov @x[6],4*6($out) + mov @x[7],4*7($out) + mov @x[0],4*8($out) + mov @x[1],4*9($out) + mov @x[2],4*10($out) + mov @x[3],4*11($out) + mov @x[12],4*12($out) + mov @x[13],4*13($out) + mov @x[14],4*14($out) + mov @x[15],4*15($out) + lea 4*16($out),$out # out+=64 + mov 4*2(%rsp),%r15 + + sub \$64,%rbp + jnz .Loop_outer + + jmp .Ldone + +.align 16 +.Ltail: + mov @x[0],4*0(%rsp) + mov 4*2(%rsp),@x[0] + mov @x[1],4*1(%rsp) + mov 4*3(%rsp),@x[1] + mov @x[2],4*2(%rsp) + add 4*10(%rsp),@x[0] + mov @x[3],4*3(%rsp) + add 4*11(%rsp),@x[1] + mov @x[4],4*4(%rsp) + mov @x[5],4*5(%rsp) + mov @x[6],4*6(%rsp) + mov @x[7],4*7(%rsp) + mov @x[0],4*10(%rsp) + mov @x[1],4*11(%rsp) + xor %rbx,%rbx + mov @x[12],4*12(%rsp) + mov @x[13],4*13(%rsp) + mov @x[14],4*14(%rsp) + mov @x[15],4*15(%rsp) + +.Loop_tail: + movzb ($inp,%rbx),%eax + movzb (%rsp,%rbx),%edx + lea 1(%rbx),%rbx + xor %edx,%eax + mov %al,-1($out,%rbx) + dec %rbp + jnz .Loop_tail + +.Ldone: + lea 64+24+48(%rsp),%rsi +.cfi_def_cfa %rsi,8 + mov -48(%rsi),%r15 +.cfi_restore %r15 + mov -40(%rsi),%r14 +.cfi_restore %r14 + mov -32(%rsi),%r13 +.cfi_restore %r13 + mov -24(%rsi),%r12 +.cfi_restore %r12 + mov -16(%rsi),%rbp +.cfi_restore %rbp + mov -8(%rsi),%rbx +.cfi_restore %rbx + lea (%rsi),%rsp +.cfi_def_cfa_register %rsp +.Lno_data: + ret +.cfi_endproc +.size ChaCha20_ctr32,.-ChaCha20_ctr32 +___ + +######################################################################## +# SSSE3 code path that handles shorter lengths +{ +my ($a,$b,$c,$d,$t,$t1,$rot16,$rot24)=map("%xmm$_",(0..7)); + +sub SSSE3ROUND { # critical path is 20 "SIMD ticks" per round + &paddd ($a,$b); + &pxor ($d,$a); + &pshufb ($d,$rot16); + + &paddd ($c,$d); + &pxor ($b,$c); + &movdqa ($t,$b); + &psrld ($b,20); + &pslld ($t,12); + &por ($b,$t); + + &paddd ($a,$b); + &pxor ($d,$a); + &pshufb ($d,$rot24); + + &paddd ($c,$d); + &pxor ($b,$c); + &movdqa ($t,$b); + &psrld ($b,25); + &pslld ($t,7); + &por ($b,$t); +} + +my $xframe = $win64 ? 32+8 : 8; + +$code.=<<___ if ($flavour =~ /kernel/); +.globl ChaCha20_ssse3 +___ +$code.=<<___; +.type ChaCha20_ssse3,\@function,5 +.align 32 +ChaCha20_ssse3: +.cfi_startproc +.LChaCha20_ssse3: + mov %rsp,%r10 # frame pointer +.cfi_def_cfa_register %r10 +___ +$code.=<<___ if ($avx && $flavour !~ /kernel/); + test \$`1<<(43-32)`,%r9d + jnz .LChaCha20_4xop # XOP is fastest even if we use 1/4 +___ +$code.=<<___; + cmp \$128,$len # we might throw away some data, + je .LChaCha20_128 + ja .LChaCha20_4x # but overall it won't be slower + +.Ldo_sse3_after_all: + sub \$64+$xframe,%rsp + and \$-16,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0x28(%r10) + movaps %xmm7,-0x18(%r10) +.Lssse3_body: +___ +$code.=<<___; + movdqa .Lsigma(%rip),$a + movdqu ($key),$b + movdqu 16($key),$c + movdqu ($counter),$d + movdqa .Lrot16(%rip),$rot16 + movdqa .Lrot24(%rip),$rot24 + + movdqa $a,0x00(%rsp) + movdqa $b,0x10(%rsp) + movdqa $c,0x20(%rsp) + movdqa $d,0x30(%rsp) + mov \$10,$counter # reuse $counter + jmp .Loop_ssse3 + +.align 32 +.Loop_outer_ssse3: + movdqa .Lone(%rip),$d + movdqa 0x00(%rsp),$a + movdqa 0x10(%rsp),$b + movdqa 0x20(%rsp),$c + paddd 0x30(%rsp),$d + mov \$10,$counter + movdqa $d,0x30(%rsp) + jmp .Loop_ssse3 + +.align 32 +.Loop_ssse3: +___ + &SSSE3ROUND(); + &pshufd ($c,$c,0b01001110); + &pshufd ($b,$b,0b00111001); + &pshufd ($d,$d,0b10010011); + &nop (); + + &SSSE3ROUND(); + &pshufd ($c,$c,0b01001110); + &pshufd ($b,$b,0b10010011); + &pshufd ($d,$d,0b00111001); + + &dec ($counter); + &jnz (".Loop_ssse3"); + +$code.=<<___; + paddd 0x00(%rsp),$a + paddd 0x10(%rsp),$b + paddd 0x20(%rsp),$c + paddd 0x30(%rsp),$d + + cmp \$64,$len + jb .Ltail_ssse3 + + movdqu 0x00($inp),$t + movdqu 0x10($inp),$t1 + pxor $t,$a # xor with input + movdqu 0x20($inp),$t + pxor $t1,$b + movdqu 0x30($inp),$t1 + lea 0x40($inp),$inp # inp+=64 + pxor $t,$c + pxor $t1,$d + + movdqu $a,0x00($out) # write output + movdqu $b,0x10($out) + movdqu $c,0x20($out) + movdqu $d,0x30($out) + lea 0x40($out),$out # out+=64 + + sub \$64,$len + jnz .Loop_outer_ssse3 + + jmp .Ldone_ssse3 + +.align 16 +.Ltail_ssse3: + movdqa $a,0x00(%rsp) + movdqa $b,0x10(%rsp) + movdqa $c,0x20(%rsp) + movdqa $d,0x30(%rsp) + xor $counter,$counter + +.Loop_tail_ssse3: + movzb ($inp,$counter),%eax + movzb (%rsp,$counter),%ecx + lea 1($counter),$counter + xor %ecx,%eax + mov %al,-1($out,$counter) + dec $len + jnz .Loop_tail_ssse3 + +.Ldone_ssse3: +___ +$code.=<<___ if ($win64); + movaps -0x28(%r10),%xmm6 + movaps -0x18(%r10),%xmm7 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.Lssse3_epilogue: + ret +.cfi_endproc +.size ChaCha20_ssse3,.-ChaCha20_ssse3 +___ +} + +######################################################################## +# SSSE3 code path that handles 128-byte inputs +{ +my ($a,$b,$c,$d,$t,$t1,$rot16,$rot24)=map("%xmm$_",(8,9,2..7)); +my ($a1,$b1,$c1,$d1)=map("%xmm$_",(10,11,0,1)); + +sub SSSE3ROUND_2x { + &paddd ($a,$b); + &pxor ($d,$a); + &paddd ($a1,$b1); + &pxor ($d1,$a1); + &pshufb ($d,$rot16); + &pshufb($d1,$rot16); + + &paddd ($c,$d); + &paddd ($c1,$d1); + &pxor ($b,$c); + &pxor ($b1,$c1); + &movdqa ($t,$b); + &psrld ($b,20); + &movdqa($t1,$b1); + &pslld ($t,12); + &psrld ($b1,20); + &por ($b,$t); + &pslld ($t1,12); + &por ($b1,$t1); + + &paddd ($a,$b); + &pxor ($d,$a); + &paddd ($a1,$b1); + &pxor ($d1,$a1); + &pshufb ($d,$rot24); + &pshufb($d1,$rot24); + + &paddd ($c,$d); + &paddd ($c1,$d1); + &pxor ($b,$c); + &pxor ($b1,$c1); + &movdqa ($t,$b); + &psrld ($b,25); + &movdqa($t1,$b1); + &pslld ($t,7); + &psrld ($b1,25); + &por ($b,$t); + &pslld ($t1,7); + &por ($b1,$t1); +} + +my $xframe = $win64 ? 0x68 : 8; + +$code.=<<___; +.type ChaCha20_128,\@function,5 +.align 32 +ChaCha20_128: +.cfi_startproc +.LChaCha20_128: + mov %rsp,%r10 # frame pointer +.cfi_def_cfa_register %r10 + sub \$64+$xframe,%rsp + and \$-16,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0x68(%r10) + movaps %xmm7,-0x58(%r10) + movaps %xmm8,-0x48(%r10) + movaps %xmm9,-0x38(%r10) + movaps %xmm10,-0x28(%r10) + movaps %xmm11,-0x18(%r10) +.L128_body: +___ +$code.=<<___; + movdqa .Lsigma(%rip),$a + movdqu ($key),$b + movdqu 16($key),$c + movdqu ($counter),$d + movdqa .Lone(%rip),$d1 + movdqa .Lrot16(%rip),$rot16 + movdqa .Lrot24(%rip),$rot24 + + movdqa $a,$a1 + movdqa $a,0x00(%rsp) + movdqa $b,$b1 + movdqa $b,0x10(%rsp) + movdqa $c,$c1 + movdqa $c,0x20(%rsp) + paddd $d,$d1 + movdqa $d,0x30(%rsp) + mov \$10,$counter # reuse $counter + jmp .Loop_128 + +.align 32 +.Loop_128: +___ + &SSSE3ROUND_2x(); + &pshufd ($c,$c,0b01001110); + &pshufd ($b,$b,0b00111001); + &pshufd ($d,$d,0b10010011); + &pshufd ($c1,$c1,0b01001110); + &pshufd ($b1,$b1,0b00111001); + &pshufd ($d1,$d1,0b10010011); + + &SSSE3ROUND_2x(); + &pshufd ($c,$c,0b01001110); + &pshufd ($b,$b,0b10010011); + &pshufd ($d,$d,0b00111001); + &pshufd ($c1,$c1,0b01001110); + &pshufd ($b1,$b1,0b10010011); + &pshufd ($d1,$d1,0b00111001); + + &dec ($counter); + &jnz (".Loop_128"); + +$code.=<<___; + paddd 0x00(%rsp),$a + paddd 0x10(%rsp),$b + paddd 0x20(%rsp),$c + paddd 0x30(%rsp),$d + paddd .Lone(%rip),$d1 + paddd 0x00(%rsp),$a1 + paddd 0x10(%rsp),$b1 + paddd 0x20(%rsp),$c1 + paddd 0x30(%rsp),$d1 + + movdqu 0x00($inp),$t + movdqu 0x10($inp),$t1 + pxor $t,$a # xor with input + movdqu 0x20($inp),$t + pxor $t1,$b + movdqu 0x30($inp),$t1 + pxor $t,$c + movdqu 0x40($inp),$t + pxor $t1,$d + movdqu 0x50($inp),$t1 + pxor $t,$a1 + movdqu 0x60($inp),$t + pxor $t1,$b1 + movdqu 0x70($inp),$t1 + pxor $t,$c1 + pxor $t1,$d1 + + movdqu $a,0x00($out) # write output + movdqu $b,0x10($out) + movdqu $c,0x20($out) + movdqu $d,0x30($out) + movdqu $a1,0x40($out) + movdqu $b1,0x50($out) + movdqu $c1,0x60($out) + movdqu $d1,0x70($out) +___ +$code.=<<___ if ($win64); + movaps -0x68(%r10),%xmm6 + movaps -0x58(%r10),%xmm7 + movaps -0x48(%r10),%xmm8 + movaps -0x38(%r10),%xmm9 + movaps -0x28(%r10),%xmm10 + movaps -0x18(%r10),%xmm11 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.L128_epilogue: + ret +.cfi_endproc +.size ChaCha20_128,.-ChaCha20_128 +___ +} + +######################################################################## +# SSSE3 code path that handles longer messages. +{ +# assign variables to favor Atom front-end +my ($xd0,$xd1,$xd2,$xd3, $xt0,$xt1,$xt2,$xt3, + $xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3)=map("%xmm$_",(0..15)); +my @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + "%nox","%nox","%nox","%nox", $xd0,$xd1,$xd2,$xd3); + +sub SSSE3_lane_ROUND { +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); +my ($xc,$xc_,$t0,$t1)=map("\"$_\"",$xt0,$xt1,$xt2,$xt3); +my @x=map("\"$_\"",@xx); + + # Consider order in which variables are addressed by their + # index: + # + # a b c d + # + # 0 4 8 12 < even round + # 1 5 9 13 + # 2 6 10 14 + # 3 7 11 15 + # 0 5 10 15 < odd round + # 1 6 11 12 + # 2 7 8 13 + # 3 4 9 14 + # + # 'a', 'b' and 'd's are permanently allocated in registers, + # @x[0..7,12..15], while 'c's are maintained in memory. If + # you observe 'c' column, you'll notice that pair of 'c's is + # invariant between rounds. This means that we have to reload + # them once per round, in the middle. This is why you'll see + # bunch of 'c' stores and loads in the middle, but none in + # the beginning or end. + + ( + "&paddd (@x[$a0],@x[$b0])", # Q1 + "&paddd (@x[$a1],@x[$b1])", # Q2 + "&pxor (@x[$d0],@x[$a0])", + "&pxor (@x[$d1],@x[$a1])", + "&pshufb (@x[$d0],$t1)", + "&pshufb (@x[$d1],$t1)", + + "&paddd ($xc,@x[$d0])", + "&paddd ($xc_,@x[$d1])", + "&pxor (@x[$b0],$xc)", + "&pxor (@x[$b1],$xc_)", + "&movdqa ($t0,@x[$b0])", + "&pslld (@x[$b0],12)", + "&psrld ($t0,20)", + "&movdqa ($t1,@x[$b1])", + "&pslld (@x[$b1],12)", + "&por (@x[$b0],$t0)", + "&psrld ($t1,20)", + "&movdqa ($t0,'(%r11)')", # .Lrot24(%rip) + "&por (@x[$b1],$t1)", + + "&paddd (@x[$a0],@x[$b0])", + "&paddd (@x[$a1],@x[$b1])", + "&pxor (@x[$d0],@x[$a0])", + "&pxor (@x[$d1],@x[$a1])", + "&pshufb (@x[$d0],$t0)", + "&pshufb (@x[$d1],$t0)", + + "&paddd ($xc,@x[$d0])", + "&paddd ($xc_,@x[$d1])", + "&pxor (@x[$b0],$xc)", + "&pxor (@x[$b1],$xc_)", + "&movdqa ($t1,@x[$b0])", + "&pslld (@x[$b0],7)", + "&psrld ($t1,25)", + "&movdqa ($t0,@x[$b1])", + "&pslld (@x[$b1],7)", + "&por (@x[$b0],$t1)", + "&psrld ($t0,25)", + "&movdqa ($t1,'(%r9)')", # .Lrot16(%rip) + "&por (@x[$b1],$t0)", + + "&movdqa (\"`16*($c0-8)`(%rsp)\",$xc)", # reload pair of 'c's + "&movdqa (\"`16*($c1-8)`(%rsp)\",$xc_)", + "&movdqa ($xc,\"`16*($c2-8)`(%rsp)\")", + "&movdqa ($xc_,\"`16*($c3-8)`(%rsp)\")", + + "&paddd (@x[$a2],@x[$b2])", # Q3 + "&paddd (@x[$a3],@x[$b3])", # Q4 + "&pxor (@x[$d2],@x[$a2])", + "&pxor (@x[$d3],@x[$a3])", + "&pshufb (@x[$d2],$t1)", + "&pshufb (@x[$d3],$t1)", + + "&paddd ($xc,@x[$d2])", + "&paddd ($xc_,@x[$d3])", + "&pxor (@x[$b2],$xc)", + "&pxor (@x[$b3],$xc_)", + "&movdqa ($t0,@x[$b2])", + "&pslld (@x[$b2],12)", + "&psrld ($t0,20)", + "&movdqa ($t1,@x[$b3])", + "&pslld (@x[$b3],12)", + "&por (@x[$b2],$t0)", + "&psrld ($t1,20)", + "&movdqa ($t0,'(%r11)')", # .Lrot24(%rip) + "&por (@x[$b3],$t1)", + + "&paddd (@x[$a2],@x[$b2])", + "&paddd (@x[$a3],@x[$b3])", + "&pxor (@x[$d2],@x[$a2])", + "&pxor (@x[$d3],@x[$a3])", + "&pshufb (@x[$d2],$t0)", + "&pshufb (@x[$d3],$t0)", + + "&paddd ($xc,@x[$d2])", + "&paddd ($xc_,@x[$d3])", + "&pxor (@x[$b2],$xc)", + "&pxor (@x[$b3],$xc_)", + "&movdqa ($t1,@x[$b2])", + "&pslld (@x[$b2],7)", + "&psrld ($t1,25)", + "&movdqa ($t0,@x[$b3])", + "&pslld (@x[$b3],7)", + "&por (@x[$b2],$t1)", + "&psrld ($t0,25)", + "&movdqa ($t1,'(%r9)')", # .Lrot16(%rip) + "&por (@x[$b3],$t0)" + ); +} + +my $xframe = $win64 ? 0xa8 : 8; + +$code.=<<___; +.type ChaCha20_4x,\@function,5 +.align 32 +ChaCha20_4x: +.cfi_startproc +.LChaCha20_4x: + mov %rsp,%r10 # frame pointer +.cfi_def_cfa_register %r10 + mov %r9,%r11 +___ +$code.=<<___ if ($avx>1 && $flavour !~ /kernel/); + shr \$32,%r9 # OPENSSL_ia32cap_P+8 + test \$`1<<5`,%r9 # test AVX2 + jnz .LChaCha20_8x +___ +$code.=<<___; + cmp \$192,$len + ja .Lproceed4x + + and \$`1<<26|1<<22`,%r11 # isolate XSAVE+MOVBE + cmp \$`1<<22`,%r11 # check for MOVBE without XSAVE + je .Ldo_sse3_after_all # to detect Atom + +.Lproceed4x: + sub \$0x140+$xframe,%rsp + and \$-16,%rsp +___ + ################ stack layout + # +0x00 SIMD equivalent of @x[8-12] + # ... + # +0x40 constant copy of key[0-2] smashed by lanes + # ... + # +0x100 SIMD counters (with nonce smashed by lanes) + # ... + # +0x140 +$code.=<<___ if ($win64); + movaps %xmm6,-0xa8(%r10) + movaps %xmm7,-0x98(%r10) + movaps %xmm8,-0x88(%r10) + movaps %xmm9,-0x78(%r10) + movaps %xmm10,-0x68(%r10) + movaps %xmm11,-0x58(%r10) + movaps %xmm12,-0x48(%r10) + movaps %xmm13,-0x38(%r10) + movaps %xmm14,-0x28(%r10) + movaps %xmm15,-0x18(%r10) +.L4x_body: +___ +$code.=<<___; + movdqa .Lsigma(%rip),$xa3 # key[0] + movdqu ($key),$xb3 # key[1] + movdqu 16($key),$xt3 # key[2] + movdqu ($counter),$xd3 # key[3] + lea 0x100(%rsp),%rcx # size optimization + lea .Lrot16(%rip),%r9 + lea .Lrot24(%rip),%r11 + + pshufd \$0x00,$xa3,$xa0 # smash key by lanes... + pshufd \$0x55,$xa3,$xa1 + movdqa $xa0,0x40(%rsp) # ... and offload + pshufd \$0xaa,$xa3,$xa2 + movdqa $xa1,0x50(%rsp) + pshufd \$0xff,$xa3,$xa3 + movdqa $xa2,0x60(%rsp) + movdqa $xa3,0x70(%rsp) + + pshufd \$0x00,$xb3,$xb0 + pshufd \$0x55,$xb3,$xb1 + movdqa $xb0,0x80-0x100(%rcx) + pshufd \$0xaa,$xb3,$xb2 + movdqa $xb1,0x90-0x100(%rcx) + pshufd \$0xff,$xb3,$xb3 + movdqa $xb2,0xa0-0x100(%rcx) + movdqa $xb3,0xb0-0x100(%rcx) + + pshufd \$0x00,$xt3,$xt0 # "$xc0" + pshufd \$0x55,$xt3,$xt1 # "$xc1" + movdqa $xt0,0xc0-0x100(%rcx) + pshufd \$0xaa,$xt3,$xt2 # "$xc2" + movdqa $xt1,0xd0-0x100(%rcx) + pshufd \$0xff,$xt3,$xt3 # "$xc3" + movdqa $xt2,0xe0-0x100(%rcx) + movdqa $xt3,0xf0-0x100(%rcx) + + pshufd \$0x00,$xd3,$xd0 + pshufd \$0x55,$xd3,$xd1 + paddd .Linc(%rip),$xd0 # don't save counters yet + pshufd \$0xaa,$xd3,$xd2 + movdqa $xd1,0x110-0x100(%rcx) + pshufd \$0xff,$xd3,$xd3 + movdqa $xd2,0x120-0x100(%rcx) + movdqa $xd3,0x130-0x100(%rcx) + + jmp .Loop_enter4x + +.align 32 +.Loop_outer4x: + movdqa 0x40(%rsp),$xa0 # re-load smashed key + movdqa 0x50(%rsp),$xa1 + movdqa 0x60(%rsp),$xa2 + movdqa 0x70(%rsp),$xa3 + movdqa 0x80-0x100(%rcx),$xb0 + movdqa 0x90-0x100(%rcx),$xb1 + movdqa 0xa0-0x100(%rcx),$xb2 + movdqa 0xb0-0x100(%rcx),$xb3 + movdqa 0xc0-0x100(%rcx),$xt0 # "$xc0" + movdqa 0xd0-0x100(%rcx),$xt1 # "$xc1" + movdqa 0xe0-0x100(%rcx),$xt2 # "$xc2" + movdqa 0xf0-0x100(%rcx),$xt3 # "$xc3" + movdqa 0x100-0x100(%rcx),$xd0 + movdqa 0x110-0x100(%rcx),$xd1 + movdqa 0x120-0x100(%rcx),$xd2 + movdqa 0x130-0x100(%rcx),$xd3 + paddd .Lfour(%rip),$xd0 # next SIMD counters + +.Loop_enter4x: + movdqa $xt2,0x20(%rsp) # SIMD equivalent of "@x[10]" + movdqa $xt3,0x30(%rsp) # SIMD equivalent of "@x[11]" + movdqa (%r9),$xt3 # .Lrot16(%rip) + mov \$10,%eax + movdqa $xd0,0x100-0x100(%rcx) # save SIMD counters + jmp .Loop4x + +.align 32 +.Loop4x: +___ + foreach (&SSSE3_lane_ROUND(0, 4, 8,12)) { eval; } + foreach (&SSSE3_lane_ROUND(0, 5,10,15)) { eval; } +$code.=<<___; + dec %eax + jnz .Loop4x + + paddd 0x40(%rsp),$xa0 # accumulate key material + paddd 0x50(%rsp),$xa1 + paddd 0x60(%rsp),$xa2 + paddd 0x70(%rsp),$xa3 + + movdqa $xa0,$xt2 # "de-interlace" data + punpckldq $xa1,$xa0 + movdqa $xa2,$xt3 + punpckldq $xa3,$xa2 + punpckhdq $xa1,$xt2 + punpckhdq $xa3,$xt3 + movdqa $xa0,$xa1 + punpcklqdq $xa2,$xa0 # "a0" + movdqa $xt2,$xa3 + punpcklqdq $xt3,$xt2 # "a2" + punpckhqdq $xa2,$xa1 # "a1" + punpckhqdq $xt3,$xa3 # "a3" +___ + ($xa2,$xt2)=($xt2,$xa2); +$code.=<<___; + paddd 0x80-0x100(%rcx),$xb0 + paddd 0x90-0x100(%rcx),$xb1 + paddd 0xa0-0x100(%rcx),$xb2 + paddd 0xb0-0x100(%rcx),$xb3 + + movdqa $xa0,0x00(%rsp) # offload $xaN + movdqa $xa1,0x10(%rsp) + movdqa 0x20(%rsp),$xa0 # "xc2" + movdqa 0x30(%rsp),$xa1 # "xc3" + + movdqa $xb0,$xt2 + punpckldq $xb1,$xb0 + movdqa $xb2,$xt3 + punpckldq $xb3,$xb2 + punpckhdq $xb1,$xt2 + punpckhdq $xb3,$xt3 + movdqa $xb0,$xb1 + punpcklqdq $xb2,$xb0 # "b0" + movdqa $xt2,$xb3 + punpcklqdq $xt3,$xt2 # "b2" + punpckhqdq $xb2,$xb1 # "b1" + punpckhqdq $xt3,$xb3 # "b3" +___ + ($xb2,$xt2)=($xt2,$xb2); + my ($xc0,$xc1,$xc2,$xc3)=($xt0,$xt1,$xa0,$xa1); +$code.=<<___; + paddd 0xc0-0x100(%rcx),$xc0 + paddd 0xd0-0x100(%rcx),$xc1 + paddd 0xe0-0x100(%rcx),$xc2 + paddd 0xf0-0x100(%rcx),$xc3 + + movdqa $xa2,0x20(%rsp) # keep offloading $xaN + movdqa $xa3,0x30(%rsp) + + movdqa $xc0,$xt2 + punpckldq $xc1,$xc0 + movdqa $xc2,$xt3 + punpckldq $xc3,$xc2 + punpckhdq $xc1,$xt2 + punpckhdq $xc3,$xt3 + movdqa $xc0,$xc1 + punpcklqdq $xc2,$xc0 # "c0" + movdqa $xt2,$xc3 + punpcklqdq $xt3,$xt2 # "c2" + punpckhqdq $xc2,$xc1 # "c1" + punpckhqdq $xt3,$xc3 # "c3" +___ + ($xc2,$xt2)=($xt2,$xc2); + ($xt0,$xt1)=($xa2,$xa3); # use $xaN as temporary +$code.=<<___; + paddd 0x100-0x100(%rcx),$xd0 + paddd 0x110-0x100(%rcx),$xd1 + paddd 0x120-0x100(%rcx),$xd2 + paddd 0x130-0x100(%rcx),$xd3 + + movdqa $xd0,$xt2 + punpckldq $xd1,$xd0 + movdqa $xd2,$xt3 + punpckldq $xd3,$xd2 + punpckhdq $xd1,$xt2 + punpckhdq $xd3,$xt3 + movdqa $xd0,$xd1 + punpcklqdq $xd2,$xd0 # "d0" + movdqa $xt2,$xd3 + punpcklqdq $xt3,$xt2 # "d2" + punpckhqdq $xd2,$xd1 # "d1" + punpckhqdq $xt3,$xd3 # "d3" +___ + ($xd2,$xt2)=($xt2,$xd2); +$code.=<<___; + cmp \$64*4,$len + jb .Ltail4x + + movdqu 0x00($inp),$xt0 # xor with input + movdqu 0x10($inp),$xt1 + movdqu 0x20($inp),$xt2 + movdqu 0x30($inp),$xt3 + pxor 0x00(%rsp),$xt0 # $xaN is offloaded, remember? + pxor $xb0,$xt1 + pxor $xc0,$xt2 + pxor $xd0,$xt3 + + movdqu $xt0,0x00($out) + movdqu 0x40($inp),$xt0 + movdqu $xt1,0x10($out) + movdqu 0x50($inp),$xt1 + movdqu $xt2,0x20($out) + movdqu 0x60($inp),$xt2 + movdqu $xt3,0x30($out) + movdqu 0x70($inp),$xt3 + lea 0x80($inp),$inp # size optimization + pxor 0x10(%rsp),$xt0 + pxor $xb1,$xt1 + pxor $xc1,$xt2 + pxor $xd1,$xt3 + + movdqu $xt0,0x40($out) + movdqu 0x00($inp),$xt0 + movdqu $xt1,0x50($out) + movdqu 0x10($inp),$xt1 + movdqu $xt2,0x60($out) + movdqu 0x20($inp),$xt2 + movdqu $xt3,0x70($out) + lea 0x80($out),$out # size optimization + movdqu 0x30($inp),$xt3 + pxor 0x20(%rsp),$xt0 + pxor $xb2,$xt1 + pxor $xc2,$xt2 + pxor $xd2,$xt3 + + movdqu $xt0,0x00($out) + movdqu 0x40($inp),$xt0 + movdqu $xt1,0x10($out) + movdqu 0x50($inp),$xt1 + movdqu $xt2,0x20($out) + movdqu 0x60($inp),$xt2 + movdqu $xt3,0x30($out) + movdqu 0x70($inp),$xt3 + lea 0x80($inp),$inp # inp+=64*4 + pxor 0x30(%rsp),$xt0 + pxor $xb3,$xt1 + pxor $xc3,$xt2 + pxor $xd3,$xt3 + movdqu $xt0,0x40($out) + movdqu $xt1,0x50($out) + movdqu $xt2,0x60($out) + movdqu $xt3,0x70($out) + lea 0x80($out),$out # out+=64*4 + + sub \$64*4,$len + jnz .Loop_outer4x + + jmp .Ldone4x + +.Ltail4x: + cmp \$192,$len + jae .L192_or_more4x + cmp \$128,$len + jae .L128_or_more4x + cmp \$64,$len + jae .L64_or_more4x + + #movdqa 0x00(%rsp),$xt0 # $xaN is offloaded, remember? + xor %r9,%r9 + #movdqa $xt0,0x00(%rsp) + movdqa $xb0,0x10(%rsp) + movdqa $xc0,0x20(%rsp) + movdqa $xd0,0x30(%rsp) + jmp .Loop_tail4x + +.align 32 +.L64_or_more4x: + movdqu 0x00($inp),$xt0 # xor with input + movdqu 0x10($inp),$xt1 + movdqu 0x20($inp),$xt2 + movdqu 0x30($inp),$xt3 + pxor 0x00(%rsp),$xt0 # $xaxN is offloaded, remember? + pxor $xb0,$xt1 + pxor $xc0,$xt2 + pxor $xd0,$xt3 + movdqu $xt0,0x00($out) + movdqu $xt1,0x10($out) + movdqu $xt2,0x20($out) + movdqu $xt3,0x30($out) + je .Ldone4x + + movdqa 0x10(%rsp),$xt0 # $xaN is offloaded, remember? + lea 0x40($inp),$inp # inp+=64*1 + xor %r9,%r9 + movdqa $xt0,0x00(%rsp) + movdqa $xb1,0x10(%rsp) + lea 0x40($out),$out # out+=64*1 + movdqa $xc1,0x20(%rsp) + sub \$64,$len # len-=64*1 + movdqa $xd1,0x30(%rsp) + jmp .Loop_tail4x + +.align 32 +.L128_or_more4x: + movdqu 0x00($inp),$xt0 # xor with input + movdqu 0x10($inp),$xt1 + movdqu 0x20($inp),$xt2 + movdqu 0x30($inp),$xt3 + pxor 0x00(%rsp),$xt0 # $xaN is offloaded, remember? + pxor $xb0,$xt1 + pxor $xc0,$xt2 + pxor $xd0,$xt3 + + movdqu $xt0,0x00($out) + movdqu 0x40($inp),$xt0 + movdqu $xt1,0x10($out) + movdqu 0x50($inp),$xt1 + movdqu $xt2,0x20($out) + movdqu 0x60($inp),$xt2 + movdqu $xt3,0x30($out) + movdqu 0x70($inp),$xt3 + pxor 0x10(%rsp),$xt0 + pxor $xb1,$xt1 + pxor $xc1,$xt2 + pxor $xd1,$xt3 + movdqu $xt0,0x40($out) + movdqu $xt1,0x50($out) + movdqu $xt2,0x60($out) + movdqu $xt3,0x70($out) + je .Ldone4x + + movdqa 0x20(%rsp),$xt0 # $xaN is offloaded, remember? + lea 0x80($inp),$inp # inp+=64*2 + xor %r9,%r9 + movdqa $xt0,0x00(%rsp) + movdqa $xb2,0x10(%rsp) + lea 0x80($out),$out # out+=64*2 + movdqa $xc2,0x20(%rsp) + sub \$128,$len # len-=64*2 + movdqa $xd2,0x30(%rsp) + jmp .Loop_tail4x + +.align 32 +.L192_or_more4x: + movdqu 0x00($inp),$xt0 # xor with input + movdqu 0x10($inp),$xt1 + movdqu 0x20($inp),$xt2 + movdqu 0x30($inp),$xt3 + pxor 0x00(%rsp),$xt0 # $xaN is offloaded, remember? + pxor $xb0,$xt1 + pxor $xc0,$xt2 + pxor $xd0,$xt3 + + movdqu $xt0,0x00($out) + movdqu 0x40($inp),$xt0 + movdqu $xt1,0x10($out) + movdqu 0x50($inp),$xt1 + movdqu $xt2,0x20($out) + movdqu 0x60($inp),$xt2 + movdqu $xt3,0x30($out) + movdqu 0x70($inp),$xt3 + lea 0x80($inp),$inp # size optimization + pxor 0x10(%rsp),$xt0 + pxor $xb1,$xt1 + pxor $xc1,$xt2 + pxor $xd1,$xt3 + + movdqu $xt0,0x40($out) + movdqu 0x00($inp),$xt0 + movdqu $xt1,0x50($out) + movdqu 0x10($inp),$xt1 + movdqu $xt2,0x60($out) + movdqu 0x20($inp),$xt2 + movdqu $xt3,0x70($out) + lea 0x80($out),$out # size optimization + movdqu 0x30($inp),$xt3 + pxor 0x20(%rsp),$xt0 + pxor $xb2,$xt1 + pxor $xc2,$xt2 + pxor $xd2,$xt3 + movdqu $xt0,0x00($out) + movdqu $xt1,0x10($out) + movdqu $xt2,0x20($out) + movdqu $xt3,0x30($out) + je .Ldone4x + + movdqa 0x30(%rsp),$xt0 # $xaN is offloaded, remember? + lea 0x40($inp),$inp # inp+=64*3 + xor %r9,%r9 + movdqa $xt0,0x00(%rsp) + movdqa $xb3,0x10(%rsp) + lea 0x40($out),$out # out+=64*3 + movdqa $xc3,0x20(%rsp) + sub \$192,$len # len-=64*3 + movdqa $xd3,0x30(%rsp) + +.Loop_tail4x: + movzb ($inp,%r9),%eax + movzb (%rsp,%r9),%ecx + lea 1(%r9),%r9 + xor %ecx,%eax + mov %al,-1($out,%r9) + dec $len + jnz .Loop_tail4x + +.Ldone4x: +___ +$code.=<<___ if ($win64); + movaps -0xa8(%r10),%xmm6 + movaps -0x98(%r10),%xmm7 + movaps -0x88(%r10),%xmm8 + movaps -0x78(%r10),%xmm9 + movaps -0x68(%r10),%xmm10 + movaps -0x58(%r10),%xmm11 + movaps -0x48(%r10),%xmm12 + movaps -0x38(%r10),%xmm13 + movaps -0x28(%r10),%xmm14 + movaps -0x18(%r10),%xmm15 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.L4x_epilogue: + ret +.cfi_endproc +.size ChaCha20_4x,.-ChaCha20_4x +___ +} + +######################################################################## +# XOP code path that handles all lengths. +if ($avx) { +# There is some "anomaly" observed depending on instructions' size or +# alignment. If you look closely at below code you'll notice that +# sometimes argument order varies. The order affects instruction +# encoding by making it larger, and such fiddling gives 5% performance +# improvement. This is on FX-4100... + +my ($xb0,$xb1,$xb2,$xb3, $xd0,$xd1,$xd2,$xd3, + $xa0,$xa1,$xa2,$xa3, $xt0,$xt1,$xt2,$xt3)=map("%xmm$_",(0..15)); +my @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xt0,$xt1,$xt2,$xt3, $xd0,$xd1,$xd2,$xd3); + +sub XOP_lane_ROUND { +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); +my @x=map("\"$_\"",@xx); + + ( + "&vpaddd (@x[$a0],@x[$a0],@x[$b0])", # Q1 + "&vpaddd (@x[$a1],@x[$a1],@x[$b1])", # Q2 + "&vpaddd (@x[$a2],@x[$a2],@x[$b2])", # Q3 + "&vpaddd (@x[$a3],@x[$a3],@x[$b3])", # Q4 + "&vpxor (@x[$d0],@x[$a0],@x[$d0])", + "&vpxor (@x[$d1],@x[$a1],@x[$d1])", + "&vpxor (@x[$d2],@x[$a2],@x[$d2])", + "&vpxor (@x[$d3],@x[$a3],@x[$d3])", + "&vprotd (@x[$d0],@x[$d0],16)", + "&vprotd (@x[$d1],@x[$d1],16)", + "&vprotd (@x[$d2],@x[$d2],16)", + "&vprotd (@x[$d3],@x[$d3],16)", + + "&vpaddd (@x[$c0],@x[$c0],@x[$d0])", + "&vpaddd (@x[$c1],@x[$c1],@x[$d1])", + "&vpaddd (@x[$c2],@x[$c2],@x[$d2])", + "&vpaddd (@x[$c3],@x[$c3],@x[$d3])", + "&vpxor (@x[$b0],@x[$c0],@x[$b0])", + "&vpxor (@x[$b1],@x[$c1],@x[$b1])", + "&vpxor (@x[$b2],@x[$b2],@x[$c2])", # flip + "&vpxor (@x[$b3],@x[$b3],@x[$c3])", # flip + "&vprotd (@x[$b0],@x[$b0],12)", + "&vprotd (@x[$b1],@x[$b1],12)", + "&vprotd (@x[$b2],@x[$b2],12)", + "&vprotd (@x[$b3],@x[$b3],12)", + + "&vpaddd (@x[$a0],@x[$b0],@x[$a0])", # flip + "&vpaddd (@x[$a1],@x[$b1],@x[$a1])", # flip + "&vpaddd (@x[$a2],@x[$a2],@x[$b2])", + "&vpaddd (@x[$a3],@x[$a3],@x[$b3])", + "&vpxor (@x[$d0],@x[$a0],@x[$d0])", + "&vpxor (@x[$d1],@x[$a1],@x[$d1])", + "&vpxor (@x[$d2],@x[$a2],@x[$d2])", + "&vpxor (@x[$d3],@x[$a3],@x[$d3])", + "&vprotd (@x[$d0],@x[$d0],8)", + "&vprotd (@x[$d1],@x[$d1],8)", + "&vprotd (@x[$d2],@x[$d2],8)", + "&vprotd (@x[$d3],@x[$d3],8)", + + "&vpaddd (@x[$c0],@x[$c0],@x[$d0])", + "&vpaddd (@x[$c1],@x[$c1],@x[$d1])", + "&vpaddd (@x[$c2],@x[$c2],@x[$d2])", + "&vpaddd (@x[$c3],@x[$c3],@x[$d3])", + "&vpxor (@x[$b0],@x[$c0],@x[$b0])", + "&vpxor (@x[$b1],@x[$c1],@x[$b1])", + "&vpxor (@x[$b2],@x[$b2],@x[$c2])", # flip + "&vpxor (@x[$b3],@x[$b3],@x[$c3])", # flip + "&vprotd (@x[$b0],@x[$b0],7)", + "&vprotd (@x[$b1],@x[$b1],7)", + "&vprotd (@x[$b2],@x[$b2],7)", + "&vprotd (@x[$b3],@x[$b3],7)" + ); +} + +my $xframe = $win64 ? 0xa8 : 8; + +$code.=<<___ if ($flavour =~ /kernel/); +.globl ChaCha20_4xop +___ +$code.=<<___; +.type ChaCha20_4xop,\@function,5 +.align 32 +ChaCha20_4xop: +.cfi_startproc +.LChaCha20_4xop: + mov %rsp,%r10 # frame pointer +.cfi_def_cfa_register %r10 + sub \$0x140+$xframe,%rsp + and \$-16,%rsp +___ + ################ stack layout + # +0x00 SIMD equivalent of @x[8-12] + # ... + # +0x40 constant copy of key[0-2] smashed by lanes + # ... + # +0x100 SIMD counters (with nonce smashed by lanes) + # ... + # +0x140 +$code.=<<___ if ($win64); + movaps %xmm6,-0xa8(%r10) + movaps %xmm7,-0x98(%r10) + movaps %xmm8,-0x88(%r10) + movaps %xmm9,-0x78(%r10) + movaps %xmm10,-0x68(%r10) + movaps %xmm11,-0x58(%r10) + movaps %xmm12,-0x48(%r10) + movaps %xmm13,-0x38(%r10) + movaps %xmm14,-0x28(%r10) + movaps %xmm15,-0x18(%r10) +.L4xop_body: +___ +$code.=<<___; + vzeroupper + + vmovdqa .Lsigma(%rip),$xa3 # key[0] + vmovdqu ($key),$xb3 # key[1] + vmovdqu 16($key),$xt3 # key[2] + vmovdqu ($counter),$xd3 # key[3] + lea 0x100(%rsp),%rcx # size optimization + + vpshufd \$0x00,$xa3,$xa0 # smash key by lanes... + vpshufd \$0x55,$xa3,$xa1 + vmovdqa $xa0,0x40(%rsp) # ... and offload + vpshufd \$0xaa,$xa3,$xa2 + vmovdqa $xa1,0x50(%rsp) + vpshufd \$0xff,$xa3,$xa3 + vmovdqa $xa2,0x60(%rsp) + vmovdqa $xa3,0x70(%rsp) + + vpshufd \$0x00,$xb3,$xb0 + vpshufd \$0x55,$xb3,$xb1 + vmovdqa $xb0,0x80-0x100(%rcx) + vpshufd \$0xaa,$xb3,$xb2 + vmovdqa $xb1,0x90-0x100(%rcx) + vpshufd \$0xff,$xb3,$xb3 + vmovdqa $xb2,0xa0-0x100(%rcx) + vmovdqa $xb3,0xb0-0x100(%rcx) + + vpshufd \$0x00,$xt3,$xt0 # "$xc0" + vpshufd \$0x55,$xt3,$xt1 # "$xc1" + vmovdqa $xt0,0xc0-0x100(%rcx) + vpshufd \$0xaa,$xt3,$xt2 # "$xc2" + vmovdqa $xt1,0xd0-0x100(%rcx) + vpshufd \$0xff,$xt3,$xt3 # "$xc3" + vmovdqa $xt2,0xe0-0x100(%rcx) + vmovdqa $xt3,0xf0-0x100(%rcx) + + vpshufd \$0x00,$xd3,$xd0 + vpshufd \$0x55,$xd3,$xd1 + vpaddd .Linc(%rip),$xd0,$xd0 # don't save counters yet + vpshufd \$0xaa,$xd3,$xd2 + vmovdqa $xd1,0x110-0x100(%rcx) + vpshufd \$0xff,$xd3,$xd3 + vmovdqa $xd2,0x120-0x100(%rcx) + vmovdqa $xd3,0x130-0x100(%rcx) + + jmp .Loop_enter4xop + +.align 32 +.Loop_outer4xop: + vmovdqa 0x40(%rsp),$xa0 # re-load smashed key + vmovdqa 0x50(%rsp),$xa1 + vmovdqa 0x60(%rsp),$xa2 + vmovdqa 0x70(%rsp),$xa3 + vmovdqa 0x80-0x100(%rcx),$xb0 + vmovdqa 0x90-0x100(%rcx),$xb1 + vmovdqa 0xa0-0x100(%rcx),$xb2 + vmovdqa 0xb0-0x100(%rcx),$xb3 + vmovdqa 0xc0-0x100(%rcx),$xt0 # "$xc0" + vmovdqa 0xd0-0x100(%rcx),$xt1 # "$xc1" + vmovdqa 0xe0-0x100(%rcx),$xt2 # "$xc2" + vmovdqa 0xf0-0x100(%rcx),$xt3 # "$xc3" + vmovdqa 0x100-0x100(%rcx),$xd0 + vmovdqa 0x110-0x100(%rcx),$xd1 + vmovdqa 0x120-0x100(%rcx),$xd2 + vmovdqa 0x130-0x100(%rcx),$xd3 + vpaddd .Lfour(%rip),$xd0,$xd0 # next SIMD counters + +.Loop_enter4xop: + mov \$10,%eax + vmovdqa $xd0,0x100-0x100(%rcx) # save SIMD counters + jmp .Loop4xop + +.align 32 +.Loop4xop: +___ + foreach (&XOP_lane_ROUND(0, 4, 8,12)) { eval; } + foreach (&XOP_lane_ROUND(0, 5,10,15)) { eval; } +$code.=<<___; + dec %eax + jnz .Loop4xop + + vpaddd 0x40(%rsp),$xa0,$xa0 # accumulate key material + vpaddd 0x50(%rsp),$xa1,$xa1 + vpaddd 0x60(%rsp),$xa2,$xa2 + vpaddd 0x70(%rsp),$xa3,$xa3 + + vmovdqa $xt2,0x20(%rsp) # offload $xc2,3 + vmovdqa $xt3,0x30(%rsp) + + vpunpckldq $xa1,$xa0,$xt2 # "de-interlace" data + vpunpckldq $xa3,$xa2,$xt3 + vpunpckhdq $xa1,$xa0,$xa0 + vpunpckhdq $xa3,$xa2,$xa2 + vpunpcklqdq $xt3,$xt2,$xa1 # "a0" + vpunpckhqdq $xt3,$xt2,$xt2 # "a1" + vpunpcklqdq $xa2,$xa0,$xa3 # "a2" + vpunpckhqdq $xa2,$xa0,$xa0 # "a3" +___ + ($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2); +$code.=<<___; + vpaddd 0x80-0x100(%rcx),$xb0,$xb0 + vpaddd 0x90-0x100(%rcx),$xb1,$xb1 + vpaddd 0xa0-0x100(%rcx),$xb2,$xb2 + vpaddd 0xb0-0x100(%rcx),$xb3,$xb3 + + vmovdqa $xa0,0x00(%rsp) # offload $xa0,1 + vmovdqa $xa1,0x10(%rsp) + vmovdqa 0x20(%rsp),$xa0 # "xc2" + vmovdqa 0x30(%rsp),$xa1 # "xc3" + + vpunpckldq $xb1,$xb0,$xt2 + vpunpckldq $xb3,$xb2,$xt3 + vpunpckhdq $xb1,$xb0,$xb0 + vpunpckhdq $xb3,$xb2,$xb2 + vpunpcklqdq $xt3,$xt2,$xb1 # "b0" + vpunpckhqdq $xt3,$xt2,$xt2 # "b1" + vpunpcklqdq $xb2,$xb0,$xb3 # "b2" + vpunpckhqdq $xb2,$xb0,$xb0 # "b3" +___ + ($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2); + my ($xc0,$xc1,$xc2,$xc3)=($xt0,$xt1,$xa0,$xa1); +$code.=<<___; + vpaddd 0xc0-0x100(%rcx),$xc0,$xc0 + vpaddd 0xd0-0x100(%rcx),$xc1,$xc1 + vpaddd 0xe0-0x100(%rcx),$xc2,$xc2 + vpaddd 0xf0-0x100(%rcx),$xc3,$xc3 + + vpunpckldq $xc1,$xc0,$xt2 + vpunpckldq $xc3,$xc2,$xt3 + vpunpckhdq $xc1,$xc0,$xc0 + vpunpckhdq $xc3,$xc2,$xc2 + vpunpcklqdq $xt3,$xt2,$xc1 # "c0" + vpunpckhqdq $xt3,$xt2,$xt2 # "c1" + vpunpcklqdq $xc2,$xc0,$xc3 # "c2" + vpunpckhqdq $xc2,$xc0,$xc0 # "c3" +___ + ($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2); +$code.=<<___; + vpaddd 0x100-0x100(%rcx),$xd0,$xd0 + vpaddd 0x110-0x100(%rcx),$xd1,$xd1 + vpaddd 0x120-0x100(%rcx),$xd2,$xd2 + vpaddd 0x130-0x100(%rcx),$xd3,$xd3 + + vpunpckldq $xd1,$xd0,$xt2 + vpunpckldq $xd3,$xd2,$xt3 + vpunpckhdq $xd1,$xd0,$xd0 + vpunpckhdq $xd3,$xd2,$xd2 + vpunpcklqdq $xt3,$xt2,$xd1 # "d0" + vpunpckhqdq $xt3,$xt2,$xt2 # "d1" + vpunpcklqdq $xd2,$xd0,$xd3 # "d2" + vpunpckhqdq $xd2,$xd0,$xd0 # "d3" +___ + ($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2); + ($xa0,$xa1)=($xt2,$xt3); +$code.=<<___; + vmovdqa 0x00(%rsp),$xa0 # restore $xa0,1 + vmovdqa 0x10(%rsp),$xa1 + + cmp \$64*4,$len + jb .Ltail4xop + + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x10($inp),$xb0,$xb0 + vpxor 0x20($inp),$xc0,$xc0 + vpxor 0x30($inp),$xd0,$xd0 + vpxor 0x40($inp),$xa1,$xa1 + vpxor 0x50($inp),$xb1,$xb1 + vpxor 0x60($inp),$xc1,$xc1 + vpxor 0x70($inp),$xd1,$xd1 + lea 0x80($inp),$inp # size optimization + vpxor 0x00($inp),$xa2,$xa2 + vpxor 0x10($inp),$xb2,$xb2 + vpxor 0x20($inp),$xc2,$xc2 + vpxor 0x30($inp),$xd2,$xd2 + vpxor 0x40($inp),$xa3,$xa3 + vpxor 0x50($inp),$xb3,$xb3 + vpxor 0x60($inp),$xc3,$xc3 + vpxor 0x70($inp),$xd3,$xd3 + lea 0x80($inp),$inp # inp+=64*4 + + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x10($out) + vmovdqu $xc0,0x20($out) + vmovdqu $xd0,0x30($out) + vmovdqu $xa1,0x40($out) + vmovdqu $xb1,0x50($out) + vmovdqu $xc1,0x60($out) + vmovdqu $xd1,0x70($out) + lea 0x80($out),$out # size optimization + vmovdqu $xa2,0x00($out) + vmovdqu $xb2,0x10($out) + vmovdqu $xc2,0x20($out) + vmovdqu $xd2,0x30($out) + vmovdqu $xa3,0x40($out) + vmovdqu $xb3,0x50($out) + vmovdqu $xc3,0x60($out) + vmovdqu $xd3,0x70($out) + lea 0x80($out),$out # out+=64*4 + + sub \$64*4,$len + jnz .Loop_outer4xop + + jmp .Ldone4xop + +.align 32 +.Ltail4xop: + cmp \$192,$len + jae .L192_or_more4xop + cmp \$128,$len + jae .L128_or_more4xop + cmp \$64,$len + jae .L64_or_more4xop + + xor %r9,%r9 + vmovdqa $xa0,0x00(%rsp) + vmovdqa $xb0,0x10(%rsp) + vmovdqa $xc0,0x20(%rsp) + vmovdqa $xd0,0x30(%rsp) + jmp .Loop_tail4xop + +.align 32 +.L64_or_more4xop: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x10($inp),$xb0,$xb0 + vpxor 0x20($inp),$xc0,$xc0 + vpxor 0x30($inp),$xd0,$xd0 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x10($out) + vmovdqu $xc0,0x20($out) + vmovdqu $xd0,0x30($out) + je .Ldone4xop + + lea 0x40($inp),$inp # inp+=64*1 + vmovdqa $xa1,0x00(%rsp) + xor %r9,%r9 + vmovdqa $xb1,0x10(%rsp) + lea 0x40($out),$out # out+=64*1 + vmovdqa $xc1,0x20(%rsp) + sub \$64,$len # len-=64*1 + vmovdqa $xd1,0x30(%rsp) + jmp .Loop_tail4xop + +.align 32 +.L128_or_more4xop: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x10($inp),$xb0,$xb0 + vpxor 0x20($inp),$xc0,$xc0 + vpxor 0x30($inp),$xd0,$xd0 + vpxor 0x40($inp),$xa1,$xa1 + vpxor 0x50($inp),$xb1,$xb1 + vpxor 0x60($inp),$xc1,$xc1 + vpxor 0x70($inp),$xd1,$xd1 + + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x10($out) + vmovdqu $xc0,0x20($out) + vmovdqu $xd0,0x30($out) + vmovdqu $xa1,0x40($out) + vmovdqu $xb1,0x50($out) + vmovdqu $xc1,0x60($out) + vmovdqu $xd1,0x70($out) + je .Ldone4xop + + lea 0x80($inp),$inp # inp+=64*2 + vmovdqa $xa2,0x00(%rsp) + xor %r9,%r9 + vmovdqa $xb2,0x10(%rsp) + lea 0x80($out),$out # out+=64*2 + vmovdqa $xc2,0x20(%rsp) + sub \$128,$len # len-=64*2 + vmovdqa $xd2,0x30(%rsp) + jmp .Loop_tail4xop + +.align 32 +.L192_or_more4xop: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x10($inp),$xb0,$xb0 + vpxor 0x20($inp),$xc0,$xc0 + vpxor 0x30($inp),$xd0,$xd0 + vpxor 0x40($inp),$xa1,$xa1 + vpxor 0x50($inp),$xb1,$xb1 + vpxor 0x60($inp),$xc1,$xc1 + vpxor 0x70($inp),$xd1,$xd1 + lea 0x80($inp),$inp # size optimization + vpxor 0x00($inp),$xa2,$xa2 + vpxor 0x10($inp),$xb2,$xb2 + vpxor 0x20($inp),$xc2,$xc2 + vpxor 0x30($inp),$xd2,$xd2 + + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x10($out) + vmovdqu $xc0,0x20($out) + vmovdqu $xd0,0x30($out) + vmovdqu $xa1,0x40($out) + vmovdqu $xb1,0x50($out) + vmovdqu $xc1,0x60($out) + vmovdqu $xd1,0x70($out) + lea 0x80($out),$out # size optimization + vmovdqu $xa2,0x00($out) + vmovdqu $xb2,0x10($out) + vmovdqu $xc2,0x20($out) + vmovdqu $xd2,0x30($out) + je .Ldone4xop + + lea 0x40($inp),$inp # inp+=64*3 + vmovdqa $xa3,0x00(%rsp) + xor %r9,%r9 + vmovdqa $xb3,0x10(%rsp) + lea 0x40($out),$out # out+=64*3 + vmovdqa $xc3,0x20(%rsp) + sub \$192,$len # len-=64*3 + vmovdqa $xd3,0x30(%rsp) + +.Loop_tail4xop: + movzb ($inp,%r9),%eax + movzb (%rsp,%r9),%ecx + lea 1(%r9),%r9 + xor %ecx,%eax + mov %al,-1($out,%r9) + dec $len + jnz .Loop_tail4xop + +.Ldone4xop: + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0xa8(%r10),%xmm6 + movaps -0x98(%r10),%xmm7 + movaps -0x88(%r10),%xmm8 + movaps -0x78(%r10),%xmm9 + movaps -0x68(%r10),%xmm10 + movaps -0x58(%r10),%xmm11 + movaps -0x48(%r10),%xmm12 + movaps -0x38(%r10),%xmm13 + movaps -0x28(%r10),%xmm14 + movaps -0x18(%r10),%xmm15 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.L4xop_epilogue: + ret +.cfi_endproc +.size ChaCha20_4xop,.-ChaCha20_4xop +___ +} + +######################################################################## +# AVX2 code path +if ($avx>1) { +my ($xb0,$xb1,$xb2,$xb3, $xd0,$xd1,$xd2,$xd3, + $xa0,$xa1,$xa2,$xa3, $xt0,$xt1,$xt2,$xt3)=map("%ymm$_",(0..15)); +my @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + "%nox","%nox","%nox","%nox", $xd0,$xd1,$xd2,$xd3); + +sub AVX2_lane_ROUND { +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); +my ($xc,$xc_,$t0,$t1)=map("\"$_\"",$xt0,$xt1,$xt2,$xt3); +my @x=map("\"$_\"",@xx); + + # Consider order in which variables are addressed by their + # index: + # + # a b c d + # + # 0 4 8 12 < even round + # 1 5 9 13 + # 2 6 10 14 + # 3 7 11 15 + # 0 5 10 15 < odd round + # 1 6 11 12 + # 2 7 8 13 + # 3 4 9 14 + # + # 'a', 'b' and 'd's are permanently allocated in registers, + # @x[0..7,12..15], while 'c's are maintained in memory. If + # you observe 'c' column, you'll notice that pair of 'c's is + # invariant between rounds. This means that we have to reload + # them once per round, in the middle. This is why you'll see + # bunch of 'c' stores and loads in the middle, but none in + # the beginning or end. + + ( + "&vpaddd (@x[$a0],@x[$a0],@x[$b0])", # Q1 + "&vpxor (@x[$d0],@x[$a0],@x[$d0])", + "&vpshufb (@x[$d0],@x[$d0],$t1)", + "&vpaddd (@x[$a1],@x[$a1],@x[$b1])", # Q2 + "&vpxor (@x[$d1],@x[$a1],@x[$d1])", + "&vpshufb (@x[$d1],@x[$d1],$t1)", + + "&vpaddd ($xc,$xc,@x[$d0])", + "&vpxor (@x[$b0],$xc,@x[$b0])", + "&vpslld ($t0,@x[$b0],12)", + "&vpsrld (@x[$b0],@x[$b0],20)", + "&vpor (@x[$b0],$t0,@x[$b0])", + "&vbroadcasti128($t0,'(%r11)')", # .Lrot24(%rip) + "&vpaddd ($xc_,$xc_,@x[$d1])", + "&vpxor (@x[$b1],$xc_,@x[$b1])", + "&vpslld ($t1,@x[$b1],12)", + "&vpsrld (@x[$b1],@x[$b1],20)", + "&vpor (@x[$b1],$t1,@x[$b1])", + + "&vpaddd (@x[$a0],@x[$a0],@x[$b0])", + "&vpxor (@x[$d0],@x[$a0],@x[$d0])", + "&vpshufb (@x[$d0],@x[$d0],$t0)", + "&vpaddd (@x[$a1],@x[$a1],@x[$b1])", + "&vpxor (@x[$d1],@x[$a1],@x[$d1])", + "&vpshufb (@x[$d1],@x[$d1],$t0)", + + "&vpaddd ($xc,$xc,@x[$d0])", + "&vpxor (@x[$b0],$xc,@x[$b0])", + "&vpslld ($t1,@x[$b0],7)", + "&vpsrld (@x[$b0],@x[$b0],25)", + "&vpor (@x[$b0],$t1,@x[$b0])", + "&vbroadcasti128($t1,'(%r9)')", # .Lrot16(%rip) + "&vpaddd ($xc_,$xc_,@x[$d1])", + "&vpxor (@x[$b1],$xc_,@x[$b1])", + "&vpslld ($t0,@x[$b1],7)", + "&vpsrld (@x[$b1],@x[$b1],25)", + "&vpor (@x[$b1],$t0,@x[$b1])", + + "&vmovdqa (\"`32*($c0-8)`(%rsp)\",$xc)", # reload pair of 'c's + "&vmovdqa (\"`32*($c1-8)`(%rsp)\",$xc_)", + "&vmovdqa ($xc,\"`32*($c2-8)`(%rsp)\")", + "&vmovdqa ($xc_,\"`32*($c3-8)`(%rsp)\")", + + "&vpaddd (@x[$a2],@x[$a2],@x[$b2])", # Q3 + "&vpxor (@x[$d2],@x[$a2],@x[$d2])", + "&vpshufb (@x[$d2],@x[$d2],$t1)", + "&vpaddd (@x[$a3],@x[$a3],@x[$b3])", # Q4 + "&vpxor (@x[$d3],@x[$a3],@x[$d3])", + "&vpshufb (@x[$d3],@x[$d3],$t1)", + + "&vpaddd ($xc,$xc,@x[$d2])", + "&vpxor (@x[$b2],$xc,@x[$b2])", + "&vpslld ($t0,@x[$b2],12)", + "&vpsrld (@x[$b2],@x[$b2],20)", + "&vpor (@x[$b2],$t0,@x[$b2])", + "&vbroadcasti128($t0,'(%r11)')", # .Lrot24(%rip) + "&vpaddd ($xc_,$xc_,@x[$d3])", + "&vpxor (@x[$b3],$xc_,@x[$b3])", + "&vpslld ($t1,@x[$b3],12)", + "&vpsrld (@x[$b3],@x[$b3],20)", + "&vpor (@x[$b3],$t1,@x[$b3])", + + "&vpaddd (@x[$a2],@x[$a2],@x[$b2])", + "&vpxor (@x[$d2],@x[$a2],@x[$d2])", + "&vpshufb (@x[$d2],@x[$d2],$t0)", + "&vpaddd (@x[$a3],@x[$a3],@x[$b3])", + "&vpxor (@x[$d3],@x[$a3],@x[$d3])", + "&vpshufb (@x[$d3],@x[$d3],$t0)", + + "&vpaddd ($xc,$xc,@x[$d2])", + "&vpxor (@x[$b2],$xc,@x[$b2])", + "&vpslld ($t1,@x[$b2],7)", + "&vpsrld (@x[$b2],@x[$b2],25)", + "&vpor (@x[$b2],$t1,@x[$b2])", + "&vbroadcasti128($t1,'(%r9)')", # .Lrot16(%rip) + "&vpaddd ($xc_,$xc_,@x[$d3])", + "&vpxor (@x[$b3],$xc_,@x[$b3])", + "&vpslld ($t0,@x[$b3],7)", + "&vpsrld (@x[$b3],@x[$b3],25)", + "&vpor (@x[$b3],$t0,@x[$b3])" + ); +} + +my $xframe = $win64 ? 0xa8 : 8; + +$code.=<<___ if ($flavour =~ /kernel/); +.globl ChaCha20_avx2 +___ +$code.=<<___; +.type ChaCha20_avx2,\@function,5 +.align 32 +ChaCha20_avx2: +.cfi_startproc +.LChaCha20_8x: + mov %rsp,%r10 # frame register +.cfi_def_cfa_register %r10 + sub \$0x280+$xframe,%rsp + and \$-32,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0xa8(%r10) + movaps %xmm7,-0x98(%r10) + movaps %xmm8,-0x88(%r10) + movaps %xmm9,-0x78(%r10) + movaps %xmm10,-0x68(%r10) + movaps %xmm11,-0x58(%r10) + movaps %xmm12,-0x48(%r10) + movaps %xmm13,-0x38(%r10) + movaps %xmm14,-0x28(%r10) + movaps %xmm15,-0x18(%r10) +.Lavx2_body: +___ +$code.=<<___; + vzeroupper + + ################ stack layout + # +0x00 SIMD equivalent of @x[8-12] + # ... + # +0x80 constant copy of key[0-2] smashed by lanes + # ... + # +0x200 SIMD counters (with nonce smashed by lanes) + # ... + # +0x280 + + vbroadcasti128 .Lsigma(%rip),$xa3 # key[0] + vbroadcasti128 ($key),$xb3 # key[1] + vbroadcasti128 16($key),$xt3 # key[2] + vbroadcasti128 ($counter),$xd3 # key[3] + lea 0x100(%rsp),%rcx # size optimization + lea 0x200(%rsp),%rax # size optimization + lea .Lrot16(%rip),%r9 + lea .Lrot24(%rip),%r11 + + vpshufd \$0x00,$xa3,$xa0 # smash key by lanes... + vpshufd \$0x55,$xa3,$xa1 + vmovdqa $xa0,0x80-0x100(%rcx) # ... and offload + vpshufd \$0xaa,$xa3,$xa2 + vmovdqa $xa1,0xa0-0x100(%rcx) + vpshufd \$0xff,$xa3,$xa3 + vmovdqa $xa2,0xc0-0x100(%rcx) + vmovdqa $xa3,0xe0-0x100(%rcx) + + vpshufd \$0x00,$xb3,$xb0 + vpshufd \$0x55,$xb3,$xb1 + vmovdqa $xb0,0x100-0x100(%rcx) + vpshufd \$0xaa,$xb3,$xb2 + vmovdqa $xb1,0x120-0x100(%rcx) + vpshufd \$0xff,$xb3,$xb3 + vmovdqa $xb2,0x140-0x100(%rcx) + vmovdqa $xb3,0x160-0x100(%rcx) + + vpshufd \$0x00,$xt3,$xt0 # "xc0" + vpshufd \$0x55,$xt3,$xt1 # "xc1" + vmovdqa $xt0,0x180-0x200(%rax) + vpshufd \$0xaa,$xt3,$xt2 # "xc2" + vmovdqa $xt1,0x1a0-0x200(%rax) + vpshufd \$0xff,$xt3,$xt3 # "xc3" + vmovdqa $xt2,0x1c0-0x200(%rax) + vmovdqa $xt3,0x1e0-0x200(%rax) + + vpshufd \$0x00,$xd3,$xd0 + vpshufd \$0x55,$xd3,$xd1 + vpaddd .Lincy(%rip),$xd0,$xd0 # don't save counters yet + vpshufd \$0xaa,$xd3,$xd2 + vmovdqa $xd1,0x220-0x200(%rax) + vpshufd \$0xff,$xd3,$xd3 + vmovdqa $xd2,0x240-0x200(%rax) + vmovdqa $xd3,0x260-0x200(%rax) + + jmp .Loop_enter8x + +.align 32 +.Loop_outer8x: + vmovdqa 0x80-0x100(%rcx),$xa0 # re-load smashed key + vmovdqa 0xa0-0x100(%rcx),$xa1 + vmovdqa 0xc0-0x100(%rcx),$xa2 + vmovdqa 0xe0-0x100(%rcx),$xa3 + vmovdqa 0x100-0x100(%rcx),$xb0 + vmovdqa 0x120-0x100(%rcx),$xb1 + vmovdqa 0x140-0x100(%rcx),$xb2 + vmovdqa 0x160-0x100(%rcx),$xb3 + vmovdqa 0x180-0x200(%rax),$xt0 # "xc0" + vmovdqa 0x1a0-0x200(%rax),$xt1 # "xc1" + vmovdqa 0x1c0-0x200(%rax),$xt2 # "xc2" + vmovdqa 0x1e0-0x200(%rax),$xt3 # "xc3" + vmovdqa 0x200-0x200(%rax),$xd0 + vmovdqa 0x220-0x200(%rax),$xd1 + vmovdqa 0x240-0x200(%rax),$xd2 + vmovdqa 0x260-0x200(%rax),$xd3 + vpaddd .Leight(%rip),$xd0,$xd0 # next SIMD counters + +.Loop_enter8x: + vmovdqa $xt2,0x40(%rsp) # SIMD equivalent of "@x[10]" + vmovdqa $xt3,0x60(%rsp) # SIMD equivalent of "@x[11]" + vbroadcasti128 (%r9),$xt3 + vmovdqa $xd0,0x200-0x200(%rax) # save SIMD counters + mov \$10,%eax + jmp .Loop8x + +.align 32 +.Loop8x: +___ + foreach (&AVX2_lane_ROUND(0, 4, 8,12)) { eval; } + foreach (&AVX2_lane_ROUND(0, 5,10,15)) { eval; } +$code.=<<___; + dec %eax + jnz .Loop8x + + lea 0x200(%rsp),%rax # size optimization + vpaddd 0x80-0x100(%rcx),$xa0,$xa0 # accumulate key + vpaddd 0xa0-0x100(%rcx),$xa1,$xa1 + vpaddd 0xc0-0x100(%rcx),$xa2,$xa2 + vpaddd 0xe0-0x100(%rcx),$xa3,$xa3 + + vpunpckldq $xa1,$xa0,$xt2 # "de-interlace" data + vpunpckldq $xa3,$xa2,$xt3 + vpunpckhdq $xa1,$xa0,$xa0 + vpunpckhdq $xa3,$xa2,$xa2 + vpunpcklqdq $xt3,$xt2,$xa1 # "a0" + vpunpckhqdq $xt3,$xt2,$xt2 # "a1" + vpunpcklqdq $xa2,$xa0,$xa3 # "a2" + vpunpckhqdq $xa2,$xa0,$xa0 # "a3" +___ + ($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2); +$code.=<<___; + vpaddd 0x100-0x100(%rcx),$xb0,$xb0 + vpaddd 0x120-0x100(%rcx),$xb1,$xb1 + vpaddd 0x140-0x100(%rcx),$xb2,$xb2 + vpaddd 0x160-0x100(%rcx),$xb3,$xb3 + + vpunpckldq $xb1,$xb0,$xt2 + vpunpckldq $xb3,$xb2,$xt3 + vpunpckhdq $xb1,$xb0,$xb0 + vpunpckhdq $xb3,$xb2,$xb2 + vpunpcklqdq $xt3,$xt2,$xb1 # "b0" + vpunpckhqdq $xt3,$xt2,$xt2 # "b1" + vpunpcklqdq $xb2,$xb0,$xb3 # "b2" + vpunpckhqdq $xb2,$xb0,$xb0 # "b3" +___ + ($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2); +$code.=<<___; + vperm2i128 \$0x20,$xb0,$xa0,$xt3 # "de-interlace" further + vperm2i128 \$0x31,$xb0,$xa0,$xb0 + vperm2i128 \$0x20,$xb1,$xa1,$xa0 + vperm2i128 \$0x31,$xb1,$xa1,$xb1 + vperm2i128 \$0x20,$xb2,$xa2,$xa1 + vperm2i128 \$0x31,$xb2,$xa2,$xb2 + vperm2i128 \$0x20,$xb3,$xa3,$xa2 + vperm2i128 \$0x31,$xb3,$xa3,$xb3 +___ + ($xa0,$xa1,$xa2,$xa3,$xt3)=($xt3,$xa0,$xa1,$xa2,$xa3); + my ($xc0,$xc1,$xc2,$xc3)=($xt0,$xt1,$xa0,$xa1); +$code.=<<___; + vmovdqa $xa0,0x00(%rsp) # offload $xaN + vmovdqa $xa1,0x20(%rsp) + vmovdqa 0x40(%rsp),$xc2 # $xa0 + vmovdqa 0x60(%rsp),$xc3 # $xa1 + + vpaddd 0x180-0x200(%rax),$xc0,$xc0 + vpaddd 0x1a0-0x200(%rax),$xc1,$xc1 + vpaddd 0x1c0-0x200(%rax),$xc2,$xc2 + vpaddd 0x1e0-0x200(%rax),$xc3,$xc3 + + vpunpckldq $xc1,$xc0,$xt2 + vpunpckldq $xc3,$xc2,$xt3 + vpunpckhdq $xc1,$xc0,$xc0 + vpunpckhdq $xc3,$xc2,$xc2 + vpunpcklqdq $xt3,$xt2,$xc1 # "c0" + vpunpckhqdq $xt3,$xt2,$xt2 # "c1" + vpunpcklqdq $xc2,$xc0,$xc3 # "c2" + vpunpckhqdq $xc2,$xc0,$xc0 # "c3" +___ + ($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2); +$code.=<<___; + vpaddd 0x200-0x200(%rax),$xd0,$xd0 + vpaddd 0x220-0x200(%rax),$xd1,$xd1 + vpaddd 0x240-0x200(%rax),$xd2,$xd2 + vpaddd 0x260-0x200(%rax),$xd3,$xd3 + + vpunpckldq $xd1,$xd0,$xt2 + vpunpckldq $xd3,$xd2,$xt3 + vpunpckhdq $xd1,$xd0,$xd0 + vpunpckhdq $xd3,$xd2,$xd2 + vpunpcklqdq $xt3,$xt2,$xd1 # "d0" + vpunpckhqdq $xt3,$xt2,$xt2 # "d1" + vpunpcklqdq $xd2,$xd0,$xd3 # "d2" + vpunpckhqdq $xd2,$xd0,$xd0 # "d3" +___ + ($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2); +$code.=<<___; + vperm2i128 \$0x20,$xd0,$xc0,$xt3 # "de-interlace" further + vperm2i128 \$0x31,$xd0,$xc0,$xd0 + vperm2i128 \$0x20,$xd1,$xc1,$xc0 + vperm2i128 \$0x31,$xd1,$xc1,$xd1 + vperm2i128 \$0x20,$xd2,$xc2,$xc1 + vperm2i128 \$0x31,$xd2,$xc2,$xd2 + vperm2i128 \$0x20,$xd3,$xc3,$xc2 + vperm2i128 \$0x31,$xd3,$xc3,$xd3 +___ + ($xc0,$xc1,$xc2,$xc3,$xt3)=($xt3,$xc0,$xc1,$xc2,$xc3); + ($xb0,$xb1,$xb2,$xb3,$xc0,$xc1,$xc2,$xc3)= + ($xc0,$xc1,$xc2,$xc3,$xb0,$xb1,$xb2,$xb3); + ($xa0,$xa1)=($xt2,$xt3); +$code.=<<___; + vmovdqa 0x00(%rsp),$xa0 # $xaN was offloaded, remember? + vmovdqa 0x20(%rsp),$xa1 + + cmp \$64*8,$len + jb .Ltail8x + + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + lea 0x80($inp),$inp # size optimization + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + lea 0x80($out),$out # size optimization + + vpxor 0x00($inp),$xa1,$xa1 + vpxor 0x20($inp),$xb1,$xb1 + vpxor 0x40($inp),$xc1,$xc1 + vpxor 0x60($inp),$xd1,$xd1 + lea 0x80($inp),$inp # size optimization + vmovdqu $xa1,0x00($out) + vmovdqu $xb1,0x20($out) + vmovdqu $xc1,0x40($out) + vmovdqu $xd1,0x60($out) + lea 0x80($out),$out # size optimization + + vpxor 0x00($inp),$xa2,$xa2 + vpxor 0x20($inp),$xb2,$xb2 + vpxor 0x40($inp),$xc2,$xc2 + vpxor 0x60($inp),$xd2,$xd2 + lea 0x80($inp),$inp # size optimization + vmovdqu $xa2,0x00($out) + vmovdqu $xb2,0x20($out) + vmovdqu $xc2,0x40($out) + vmovdqu $xd2,0x60($out) + lea 0x80($out),$out # size optimization + + vpxor 0x00($inp),$xa3,$xa3 + vpxor 0x20($inp),$xb3,$xb3 + vpxor 0x40($inp),$xc3,$xc3 + vpxor 0x60($inp),$xd3,$xd3 + lea 0x80($inp),$inp # size optimization + vmovdqu $xa3,0x00($out) + vmovdqu $xb3,0x20($out) + vmovdqu $xc3,0x40($out) + vmovdqu $xd3,0x60($out) + lea 0x80($out),$out # size optimization + + sub \$64*8,$len + jnz .Loop_outer8x + + jmp .Ldone8x + +.Ltail8x: + cmp \$448,$len + jae .L448_or_more8x + cmp \$384,$len + jae .L384_or_more8x + cmp \$320,$len + jae .L320_or_more8x + cmp \$256,$len + jae .L256_or_more8x + cmp \$192,$len + jae .L192_or_more8x + cmp \$128,$len + jae .L128_or_more8x + cmp \$64,$len + jae .L64_or_more8x + + xor %r9,%r9 + vmovdqa $xa0,0x00(%rsp) + vmovdqa $xb0,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L64_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + je .Ldone8x + + lea 0x40($inp),$inp # inp+=64*1 + xor %r9,%r9 + vmovdqa $xc0,0x00(%rsp) + lea 0x40($out),$out # out+=64*1 + sub \$64,$len # len-=64*1 + vmovdqa $xd0,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L128_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + je .Ldone8x + + lea 0x80($inp),$inp # inp+=64*2 + xor %r9,%r9 + vmovdqa $xa1,0x00(%rsp) + lea 0x80($out),$out # out+=64*2 + sub \$128,$len # len-=64*2 + vmovdqa $xb1,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L192_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + vpxor 0x80($inp),$xa1,$xa1 + vpxor 0xa0($inp),$xb1,$xb1 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + vmovdqu $xa1,0x80($out) + vmovdqu $xb1,0xa0($out) + je .Ldone8x + + lea 0xc0($inp),$inp # inp+=64*3 + xor %r9,%r9 + vmovdqa $xc1,0x00(%rsp) + lea 0xc0($out),$out # out+=64*3 + sub \$192,$len # len-=64*3 + vmovdqa $xd1,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L256_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + vpxor 0x80($inp),$xa1,$xa1 + vpxor 0xa0($inp),$xb1,$xb1 + vpxor 0xc0($inp),$xc1,$xc1 + vpxor 0xe0($inp),$xd1,$xd1 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + vmovdqu $xa1,0x80($out) + vmovdqu $xb1,0xa0($out) + vmovdqu $xc1,0xc0($out) + vmovdqu $xd1,0xe0($out) + je .Ldone8x + + lea 0x100($inp),$inp # inp+=64*4 + xor %r9,%r9 + vmovdqa $xa2,0x00(%rsp) + lea 0x100($out),$out # out+=64*4 + sub \$256,$len # len-=64*4 + vmovdqa $xb2,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L320_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + vpxor 0x80($inp),$xa1,$xa1 + vpxor 0xa0($inp),$xb1,$xb1 + vpxor 0xc0($inp),$xc1,$xc1 + vpxor 0xe0($inp),$xd1,$xd1 + vpxor 0x100($inp),$xa2,$xa2 + vpxor 0x120($inp),$xb2,$xb2 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + vmovdqu $xa1,0x80($out) + vmovdqu $xb1,0xa0($out) + vmovdqu $xc1,0xc0($out) + vmovdqu $xd1,0xe0($out) + vmovdqu $xa2,0x100($out) + vmovdqu $xb2,0x120($out) + je .Ldone8x + + lea 0x140($inp),$inp # inp+=64*5 + xor %r9,%r9 + vmovdqa $xc2,0x00(%rsp) + lea 0x140($out),$out # out+=64*5 + sub \$320,$len # len-=64*5 + vmovdqa $xd2,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L384_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + vpxor 0x80($inp),$xa1,$xa1 + vpxor 0xa0($inp),$xb1,$xb1 + vpxor 0xc0($inp),$xc1,$xc1 + vpxor 0xe0($inp),$xd1,$xd1 + vpxor 0x100($inp),$xa2,$xa2 + vpxor 0x120($inp),$xb2,$xb2 + vpxor 0x140($inp),$xc2,$xc2 + vpxor 0x160($inp),$xd2,$xd2 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + vmovdqu $xa1,0x80($out) + vmovdqu $xb1,0xa0($out) + vmovdqu $xc1,0xc0($out) + vmovdqu $xd1,0xe0($out) + vmovdqu $xa2,0x100($out) + vmovdqu $xb2,0x120($out) + vmovdqu $xc2,0x140($out) + vmovdqu $xd2,0x160($out) + je .Ldone8x + + lea 0x180($inp),$inp # inp+=64*6 + xor %r9,%r9 + vmovdqa $xa3,0x00(%rsp) + lea 0x180($out),$out # out+=64*6 + sub \$384,$len # len-=64*6 + vmovdqa $xb3,0x20(%rsp) + jmp .Loop_tail8x + +.align 32 +.L448_or_more8x: + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + vpxor 0x80($inp),$xa1,$xa1 + vpxor 0xa0($inp),$xb1,$xb1 + vpxor 0xc0($inp),$xc1,$xc1 + vpxor 0xe0($inp),$xd1,$xd1 + vpxor 0x100($inp),$xa2,$xa2 + vpxor 0x120($inp),$xb2,$xb2 + vpxor 0x140($inp),$xc2,$xc2 + vpxor 0x160($inp),$xd2,$xd2 + vpxor 0x180($inp),$xa3,$xa3 + vpxor 0x1a0($inp),$xb3,$xb3 + vmovdqu $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + vmovdqu $xa1,0x80($out) + vmovdqu $xb1,0xa0($out) + vmovdqu $xc1,0xc0($out) + vmovdqu $xd1,0xe0($out) + vmovdqu $xa2,0x100($out) + vmovdqu $xb2,0x120($out) + vmovdqu $xc2,0x140($out) + vmovdqu $xd2,0x160($out) + vmovdqu $xa3,0x180($out) + vmovdqu $xb3,0x1a0($out) + je .Ldone8x + + lea 0x1c0($inp),$inp # inp+=64*7 + xor %r9,%r9 + vmovdqa $xc3,0x00(%rsp) + lea 0x1c0($out),$out # out+=64*7 + sub \$448,$len # len-=64*7 + vmovdqa $xd3,0x20(%rsp) + +.Loop_tail8x: + movzb ($inp,%r9),%eax + movzb (%rsp,%r9),%ecx + lea 1(%r9),%r9 + xor %ecx,%eax + mov %al,-1($out,%r9) + dec $len + jnz .Loop_tail8x + +.Ldone8x: + vzeroall +___ +$code.=<<___ if ($win64); + movaps -0xa8(%r10),%xmm6 + movaps -0x98(%r10),%xmm7 + movaps -0x88(%r10),%xmm8 + movaps -0x78(%r10),%xmm9 + movaps -0x68(%r10),%xmm10 + movaps -0x58(%r10),%xmm11 + movaps -0x48(%r10),%xmm12 + movaps -0x38(%r10),%xmm13 + movaps -0x28(%r10),%xmm14 + movaps -0x18(%r10),%xmm15 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.Lavx2_epilogue: + ret +.cfi_endproc +.size ChaCha20_avx2,.-ChaCha20_avx2 +___ +} + +######################################################################## +# AVX512 code paths +if ($avx>2) { +# This one handles shorter inputs... + +my ($a,$b,$c,$d, $a_,$b_,$c_,$d_,$fourz) = map("%zmm$_",(0..3,16..20)); +my ($t0,$t1,$t2,$t3) = map("%xmm$_",(4..7)); + +sub vpxord() # size optimization +{ my $opcode = "vpxor"; # adhere to vpxor when possible + + foreach (@_) { + if (/%([zy])mm([0-9]+)/ && ($1 eq "z" || $2>=16)) { + $opcode = "vpxord"; + last; + } + } + + $code .= "\t$opcode\t".join(',',reverse @_)."\n"; +} + +sub AVX512ROUND { # critical path is 14 "SIMD ticks" per round + &vpaddd ($a,$a,$b); + &vpxord ($d,$d,$a); + &vprold ($d,$d,16); + + &vpaddd ($c,$c,$d); + &vpxord ($b,$b,$c); + &vprold ($b,$b,12); + + &vpaddd ($a,$a,$b); + &vpxord ($d,$d,$a); + &vprold ($d,$d,8); + + &vpaddd ($c,$c,$d); + &vpxord ($b,$b,$c); + &vprold ($b,$b,7); +} + +my $xframe = $win64 ? 32+8 : 8; + +$code.=<<___ if ($flavour =~ /kernel/); +.globl ChaCha20_avx512 +___ +$code.=<<___; +.type ChaCha20_avx512,\@function,5 +.align 32 +ChaCha20_avx512: +.cfi_startproc +.LChaCha20_avx512: + mov %rsp,%r10 # frame pointer +.cfi_def_cfa_register %r10 + cmp \$512,$len + ja .LChaCha20_16x + + sub \$64+$xframe,%rsp + and \$-16,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0x28(%r10) + movaps %xmm7,-0x18(%r10) +.Lavx512_body: +___ +$code.=<<___; + vbroadcasti32x4 .Lsigma(%rip),$a + vbroadcasti32x4 ($key),$b_ + vbroadcasti32x4 16($key),$c_ + vbroadcasti32x4 ($counter),$d_ + + vmovdqa32 $a,$a_ + vmovdqa32 .Lfourz(%rip),$fourz + vpaddd .Lzeroz(%rip),$d_,$d + jmp .Loop_outer_avx512 + +.align 32 +.Loop_outer_avx512: + vmovdqa32 $b_,$b + vmovdqa32 $c_,$c + vmovdqa32 $d,$d_ + mov \$10,$counter # reuse $counter + jmp .Loop_avx512 + +.align 32 +.Loop_avx512: +___ + &AVX512ROUND(); + &vpshufd ($c,$c,0b01001110); + &vpshufd ($b,$b,0b00111001); + &vpshufd ($d,$d,0b10010011); + + &AVX512ROUND(); + &vpshufd ($c,$c,0b01001110); + &vpshufd ($b,$b,0b10010011); + &vpshufd ($d,$d,0b00111001); + + &dec ($counter); + &jnz (".Loop_avx512"); + +$code.=<<___; + vpaddd $a_,$a,$a + vpaddd $b_,$b,$b + vpaddd $c_,$c,$c + vpaddd $d_,$d,$d + + sub \$64,$len + jb .Ltail64_avx512 + + vpxor 0x00($inp),%x#$a,$t0 # xor with input + vpxor 0x10($inp),%x#$b,$t1 + vpxor 0x20($inp),%x#$c,$t2 + vpxor 0x30($inp),%x#$d,$t3 + lea 0x40($inp),$inp # inp+=64 + + vmovdqu $t0,0x00($out) # write output + vmovdqu $t1,0x10($out) + vmovdqu $t2,0x20($out) + vmovdqu $t3,0x30($out) + lea 0x40($out),$out # out+=64 + + jz .Ldone_avx512 + + vextracti32x4 \$1,$a,$t0 + vextracti32x4 \$1,$b,$t1 + vextracti32x4 \$1,$c,$t2 + vextracti32x4 \$1,$d,$t3 + + sub \$64,$len + jb .Ltail_avx512 + + vpxor 0x00($inp),$t0,$t0 # xor with input + vpxor 0x10($inp),$t1,$t1 + vpxor 0x20($inp),$t2,$t2 + vpxor 0x30($inp),$t3,$t3 + lea 0x40($inp),$inp # inp+=64 + + vmovdqu $t0,0x00($out) # write output + vmovdqu $t1,0x10($out) + vmovdqu $t2,0x20($out) + vmovdqu $t3,0x30($out) + lea 0x40($out),$out # out+=64 + + jz .Ldone_avx512 + + vextracti32x4 \$2,$a,$t0 + vextracti32x4 \$2,$b,$t1 + vextracti32x4 \$2,$c,$t2 + vextracti32x4 \$2,$d,$t3 + + sub \$64,$len + jb .Ltail_avx512 + + vpxor 0x00($inp),$t0,$t0 # xor with input + vpxor 0x10($inp),$t1,$t1 + vpxor 0x20($inp),$t2,$t2 + vpxor 0x30($inp),$t3,$t3 + lea 0x40($inp),$inp # inp+=64 + + vmovdqu $t0,0x00($out) # write output + vmovdqu $t1,0x10($out) + vmovdqu $t2,0x20($out) + vmovdqu $t3,0x30($out) + lea 0x40($out),$out # out+=64 + + jz .Ldone_avx512 + + vextracti32x4 \$3,$a,$t0 + vextracti32x4 \$3,$b,$t1 + vextracti32x4 \$3,$c,$t2 + vextracti32x4 \$3,$d,$t3 + + sub \$64,$len + jb .Ltail_avx512 + + vmovdqa32 $a_,$a + vpaddd $fourz,$d_,$d + + vpxor 0x00($inp),$t0,$t0 # xor with input + vpxor 0x10($inp),$t1,$t1 + vpxor 0x20($inp),$t2,$t2 + vpxor 0x30($inp),$t3,$t3 + lea 0x40($inp),$inp # inp+=64 + + vmovdqu $t0,0x00($out) # write output + vmovdqu $t1,0x10($out) + vmovdqu $t2,0x20($out) + vmovdqu $t3,0x30($out) + lea 0x40($out),$out # out+=64 + + jnz .Loop_outer_avx512 + + jmp .Ldone_avx512 + +.align 16 +.Ltail64_avx512: + vmovdqa %x#$a,0x00(%rsp) + vmovdqa %x#$b,0x10(%rsp) + vmovdqa %x#$c,0x20(%rsp) + vmovdqa %x#$d,0x30(%rsp) + add \$64,$len + jmp .Loop_tail_avx512 + +.align 16 +.Ltail_avx512: + vmovdqa $t0,0x00(%rsp) + vmovdqa $t1,0x10(%rsp) + vmovdqa $t2,0x20(%rsp) + vmovdqa $t3,0x30(%rsp) + add \$64,$len + +.Loop_tail_avx512: + movzb ($inp,$counter),%eax + movzb (%rsp,$counter),%ecx + lea 1($counter),$counter + xor %ecx,%eax + mov %al,-1($out,$counter) + dec $len + jnz .Loop_tail_avx512 + + vmovdqu32 $a_,0x00(%rsp) + +.Ldone_avx512: + vzeroall +___ +$code.=<<___ if ($win64); + movaps -0x28(%r10),%xmm6 + movaps -0x18(%r10),%xmm7 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.Lavx512_epilogue: + ret +.cfi_endproc +.size ChaCha20_avx512,.-ChaCha20_avx512 +___ + +map(s/%z/%y/, $a,$b,$c,$d, $a_,$b_,$c_,$d_,$fourz); + +$code.=<<___ if ($flavour =~ /kernel/); +.globl ChaCha20_avx512vl +___ +$code.=<<___; +.type ChaCha20_avx512vl,\@function,5 +.align 32 +ChaCha20_avx512vl: +.cfi_startproc +.LChaCha20_avx512vl: + mov %rsp,%r10 # frame pointer +.cfi_def_cfa_register %r10 + cmp \$128,$len + ja .LChaCha20_8xvl + + sub \$64+$xframe,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0x28(%r10) + movaps %xmm7,-0x18(%r10) +.Lavx512vl_body: +___ +$code.=<<___; + vbroadcasti32x4 .Lsigma(%rip),$a + vbroadcasti32x4 ($key),$b_ + vbroadcasti32x4 16($key),$c_ + vbroadcasti32x4 ($counter),$d_ + + vmovdqa32 $a,$a_ + vmovdqa32 .Ltwoy(%rip),$fourz + vpaddd .Lzeroz(%rip),$d_,$d + jmp .Loop_outer_avx512vl + +.align 32 +.Loop_outer_avx512vl: + vmovdqa32 $b_,$b + vmovdqa32 $c_,$c + vmovdqa32 $d,$d_ + mov \$10,$counter # reuse $counter + jmp .Loop_avx512vl + +.align 32 +.Loop_avx512vl: +___ + &AVX512ROUND(); + &vpshufd ($c,$c,0b01001110); + &vpshufd ($b,$b,0b00111001); + &vpshufd ($d,$d,0b10010011); + + &AVX512ROUND(); + &vpshufd ($c,$c,0b01001110); + &vpshufd ($b,$b,0b10010011); + &vpshufd ($d,$d,0b00111001); + + &sub ($counter,1); + &jnz (".Loop_avx512vl"); + +$code.=<<___; + vpaddd $a_,$a,$a + vpaddd $b_,$b,$b + vpaddd $c_,$c,$c + vpaddd $d_,$d,$d + + sub \$64,$len + jb .Ltail64_avx512vl + + vpxor 0x00($inp),%x#$a,$t0 # xor with input + vpxor 0x10($inp),%x#$b,$t1 + vpxor 0x20($inp),%x#$c,$t2 + vpxor 0x30($inp),%x#$d,$t3 + lea 0x40($inp),$inp # inp+=64 + + vmovdqu $t0,0x00($out) # write output + vmovdqu $t1,0x10($out) + vmovdqu $t2,0x20($out) + vmovdqu $t3,0x30($out) + lea 0x40($out),$out # out+=64 + + jz .Ldone_avx512vl + + vextracti128 \$1,$a,$t0 + vextracti128 \$1,$b,$t1 + vextracti128 \$1,$c,$t2 + vextracti128 \$1,$d,$t3 + + sub \$64,$len + jb .Ltail_avx512vl + + vmovdqa32 $a_,$a + vpaddd $fourz,$d_,$d + + vpxor 0x00($inp),$t0,$t0 # xor with input + vpxor 0x10($inp),$t1,$t1 + vpxor 0x20($inp),$t2,$t2 + vpxor 0x30($inp),$t3,$t3 + lea 0x40($inp),$inp # inp+=64 + + vmovdqu $t0,0x00($out) # write output + vmovdqu $t1,0x10($out) + vmovdqu $t2,0x20($out) + vmovdqu $t3,0x30($out) + lea 0x40($out),$out # out+=64 + + jnz .Loop_outer_avx512vl + + jmp .Ldone_avx512vl + +.align 16 +.Ltail64_avx512vl: + vmovdqa %x#$a,0x00(%rsp) + vmovdqa %x#$b,0x10(%rsp) + vmovdqa %x#$c,0x20(%rsp) + vmovdqa %x#$d,0x30(%rsp) + add \$64,$len + jmp .Loop_tail_avx512vl + +.align 16 +.Ltail_avx512vl: + vmovdqa $t0,0x00(%rsp) + vmovdqa $t1,0x10(%rsp) + vmovdqa $t2,0x20(%rsp) + vmovdqa $t3,0x30(%rsp) + add \$64,$len + +.Loop_tail_avx512vl: + movzb ($inp,$counter),%eax + movzb (%rsp,$counter),%ecx + lea 1($counter),$counter + xor %ecx,%eax + mov %al,-1($out,$counter) + dec $len + jnz .Loop_tail_avx512vl + + vmovdqu32 $a_,0x00(%rsp) + vmovdqu32 $a_,0x20(%rsp) + +.Ldone_avx512vl: + vzeroall +___ +$code.=<<___ if ($win64); + movaps -0x28(%r10),%xmm6 + movaps -0x18(%r10),%xmm7 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.Lavx512vl_epilogue: + ret +.cfi_endproc +.size ChaCha20_avx512vl,.-ChaCha20_avx512vl +___ +} +if ($avx>2) { +# This one handles longer inputs... + +my ($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3)=map("%zmm$_",(0..15)); +my @xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3); +my @key=map("%zmm$_",(16..31)); +my ($xt0,$xt1,$xt2,$xt3)=@key[0..3]; + +sub AVX512_lane_ROUND { +my ($a0,$b0,$c0,$d0)=@_; +my ($a1,$b1,$c1,$d1)=map(($_&~3)+(($_+1)&3),($a0,$b0,$c0,$d0)); +my ($a2,$b2,$c2,$d2)=map(($_&~3)+(($_+1)&3),($a1,$b1,$c1,$d1)); +my ($a3,$b3,$c3,$d3)=map(($_&~3)+(($_+1)&3),($a2,$b2,$c2,$d2)); +my @x=map("\"$_\"",@xx); + + ( + "&vpaddd (@x[$a0],@x[$a0],@x[$b0])", # Q1 + "&vpaddd (@x[$a1],@x[$a1],@x[$b1])", # Q2 + "&vpaddd (@x[$a2],@x[$a2],@x[$b2])", # Q3 + "&vpaddd (@x[$a3],@x[$a3],@x[$b3])", # Q4 + "&vpxord (@x[$d0],@x[$d0],@x[$a0])", + "&vpxord (@x[$d1],@x[$d1],@x[$a1])", + "&vpxord (@x[$d2],@x[$d2],@x[$a2])", + "&vpxord (@x[$d3],@x[$d3],@x[$a3])", + "&vprold (@x[$d0],@x[$d0],16)", + "&vprold (@x[$d1],@x[$d1],16)", + "&vprold (@x[$d2],@x[$d2],16)", + "&vprold (@x[$d3],@x[$d3],16)", + + "&vpaddd (@x[$c0],@x[$c0],@x[$d0])", + "&vpaddd (@x[$c1],@x[$c1],@x[$d1])", + "&vpaddd (@x[$c2],@x[$c2],@x[$d2])", + "&vpaddd (@x[$c3],@x[$c3],@x[$d3])", + "&vpxord (@x[$b0],@x[$b0],@x[$c0])", + "&vpxord (@x[$b1],@x[$b1],@x[$c1])", + "&vpxord (@x[$b2],@x[$b2],@x[$c2])", + "&vpxord (@x[$b3],@x[$b3],@x[$c3])", + "&vprold (@x[$b0],@x[$b0],12)", + "&vprold (@x[$b1],@x[$b1],12)", + "&vprold (@x[$b2],@x[$b2],12)", + "&vprold (@x[$b3],@x[$b3],12)", + + "&vpaddd (@x[$a0],@x[$a0],@x[$b0])", + "&vpaddd (@x[$a1],@x[$a1],@x[$b1])", + "&vpaddd (@x[$a2],@x[$a2],@x[$b2])", + "&vpaddd (@x[$a3],@x[$a3],@x[$b3])", + "&vpxord (@x[$d0],@x[$d0],@x[$a0])", + "&vpxord (@x[$d1],@x[$d1],@x[$a1])", + "&vpxord (@x[$d2],@x[$d2],@x[$a2])", + "&vpxord (@x[$d3],@x[$d3],@x[$a3])", + "&vprold (@x[$d0],@x[$d0],8)", + "&vprold (@x[$d1],@x[$d1],8)", + "&vprold (@x[$d2],@x[$d2],8)", + "&vprold (@x[$d3],@x[$d3],8)", + + "&vpaddd (@x[$c0],@x[$c0],@x[$d0])", + "&vpaddd (@x[$c1],@x[$c1],@x[$d1])", + "&vpaddd (@x[$c2],@x[$c2],@x[$d2])", + "&vpaddd (@x[$c3],@x[$c3],@x[$d3])", + "&vpxord (@x[$b0],@x[$b0],@x[$c0])", + "&vpxord (@x[$b1],@x[$b1],@x[$c1])", + "&vpxord (@x[$b2],@x[$b2],@x[$c2])", + "&vpxord (@x[$b3],@x[$b3],@x[$c3])", + "&vprold (@x[$b0],@x[$b0],7)", + "&vprold (@x[$b1],@x[$b1],7)", + "&vprold (@x[$b2],@x[$b2],7)", + "&vprold (@x[$b3],@x[$b3],7)" + ); +} + +my $xframe = $win64 ? 0xa8 : 8; + +$code.=<<___; +.type ChaCha20_16x,\@function,5 +.align 32 +ChaCha20_16x: +.cfi_startproc +.LChaCha20_16x: + mov %rsp,%r10 # frame register +.cfi_def_cfa_register %r10 + sub \$64+$xframe,%rsp + and \$-64,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0xa8(%r10) + movaps %xmm7,-0x98(%r10) + movaps %xmm8,-0x88(%r10) + movaps %xmm9,-0x78(%r10) + movaps %xmm10,-0x68(%r10) + movaps %xmm11,-0x58(%r10) + movaps %xmm12,-0x48(%r10) + movaps %xmm13,-0x38(%r10) + movaps %xmm14,-0x28(%r10) + movaps %xmm15,-0x18(%r10) +.L16x_body: +___ +$code.=<<___; + vzeroupper + + lea .Lsigma(%rip),%r9 + vbroadcasti32x4 (%r9),$xa3 # key[0] + vbroadcasti32x4 ($key),$xb3 # key[1] + vbroadcasti32x4 16($key),$xc3 # key[2] + vbroadcasti32x4 ($counter),$xd3 # key[3] + + vpshufd \$0x00,$xa3,$xa0 # smash key by lanes... + vpshufd \$0x55,$xa3,$xa1 + vpshufd \$0xaa,$xa3,$xa2 + vpshufd \$0xff,$xa3,$xa3 + vmovdqa64 $xa0,@key[0] + vmovdqa64 $xa1,@key[1] + vmovdqa64 $xa2,@key[2] + vmovdqa64 $xa3,@key[3] + + vpshufd \$0x00,$xb3,$xb0 + vpshufd \$0x55,$xb3,$xb1 + vpshufd \$0xaa,$xb3,$xb2 + vpshufd \$0xff,$xb3,$xb3 + vmovdqa64 $xb0,@key[4] + vmovdqa64 $xb1,@key[5] + vmovdqa64 $xb2,@key[6] + vmovdqa64 $xb3,@key[7] + + vpshufd \$0x00,$xc3,$xc0 + vpshufd \$0x55,$xc3,$xc1 + vpshufd \$0xaa,$xc3,$xc2 + vpshufd \$0xff,$xc3,$xc3 + vmovdqa64 $xc0,@key[8] + vmovdqa64 $xc1,@key[9] + vmovdqa64 $xc2,@key[10] + vmovdqa64 $xc3,@key[11] + + vpshufd \$0x00,$xd3,$xd0 + vpshufd \$0x55,$xd3,$xd1 + vpshufd \$0xaa,$xd3,$xd2 + vpshufd \$0xff,$xd3,$xd3 + vpaddd .Lincz(%rip),$xd0,$xd0 # don't save counters yet + vmovdqa64 $xd0,@key[12] + vmovdqa64 $xd1,@key[13] + vmovdqa64 $xd2,@key[14] + vmovdqa64 $xd3,@key[15] + + mov \$10,%eax + jmp .Loop16x + +.align 32 +.Loop_outer16x: + vpbroadcastd 0(%r9),$xa0 # reload key + vpbroadcastd 4(%r9),$xa1 + vpbroadcastd 8(%r9),$xa2 + vpbroadcastd 12(%r9),$xa3 + vpaddd .Lsixteen(%rip),@key[12],@key[12] # next SIMD counters + vmovdqa64 @key[4],$xb0 + vmovdqa64 @key[5],$xb1 + vmovdqa64 @key[6],$xb2 + vmovdqa64 @key[7],$xb3 + vmovdqa64 @key[8],$xc0 + vmovdqa64 @key[9],$xc1 + vmovdqa64 @key[10],$xc2 + vmovdqa64 @key[11],$xc3 + vmovdqa64 @key[12],$xd0 + vmovdqa64 @key[13],$xd1 + vmovdqa64 @key[14],$xd2 + vmovdqa64 @key[15],$xd3 + + vmovdqa64 $xa0,@key[0] + vmovdqa64 $xa1,@key[1] + vmovdqa64 $xa2,@key[2] + vmovdqa64 $xa3,@key[3] + + mov \$10,%eax + jmp .Loop16x + +.align 32 +.Loop16x: +___ + foreach (&AVX512_lane_ROUND(0, 4, 8,12)) { eval; } + foreach (&AVX512_lane_ROUND(0, 5,10,15)) { eval; } +$code.=<<___; + dec %eax + jnz .Loop16x + + vpaddd @key[0],$xa0,$xa0 # accumulate key + vpaddd @key[1],$xa1,$xa1 + vpaddd @key[2],$xa2,$xa2 + vpaddd @key[3],$xa3,$xa3 + + vpunpckldq $xa1,$xa0,$xt2 # "de-interlace" data + vpunpckldq $xa3,$xa2,$xt3 + vpunpckhdq $xa1,$xa0,$xa0 + vpunpckhdq $xa3,$xa2,$xa2 + vpunpcklqdq $xt3,$xt2,$xa1 # "a0" + vpunpckhqdq $xt3,$xt2,$xt2 # "a1" + vpunpcklqdq $xa2,$xa0,$xa3 # "a2" + vpunpckhqdq $xa2,$xa0,$xa0 # "a3" +___ + ($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2); +$code.=<<___; + vpaddd @key[4],$xb0,$xb0 + vpaddd @key[5],$xb1,$xb1 + vpaddd @key[6],$xb2,$xb2 + vpaddd @key[7],$xb3,$xb3 + + vpunpckldq $xb1,$xb0,$xt2 + vpunpckldq $xb3,$xb2,$xt3 + vpunpckhdq $xb1,$xb0,$xb0 + vpunpckhdq $xb3,$xb2,$xb2 + vpunpcklqdq $xt3,$xt2,$xb1 # "b0" + vpunpckhqdq $xt3,$xt2,$xt2 # "b1" + vpunpcklqdq $xb2,$xb0,$xb3 # "b2" + vpunpckhqdq $xb2,$xb0,$xb0 # "b3" +___ + ($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2); +$code.=<<___; + vshufi32x4 \$0x44,$xb0,$xa0,$xt3 # "de-interlace" further + vshufi32x4 \$0xee,$xb0,$xa0,$xb0 + vshufi32x4 \$0x44,$xb1,$xa1,$xa0 + vshufi32x4 \$0xee,$xb1,$xa1,$xb1 + vshufi32x4 \$0x44,$xb2,$xa2,$xa1 + vshufi32x4 \$0xee,$xb2,$xa2,$xb2 + vshufi32x4 \$0x44,$xb3,$xa3,$xa2 + vshufi32x4 \$0xee,$xb3,$xa3,$xb3 +___ + ($xa0,$xa1,$xa2,$xa3,$xt3)=($xt3,$xa0,$xa1,$xa2,$xa3); +$code.=<<___; + vpaddd @key[8],$xc0,$xc0 + vpaddd @key[9],$xc1,$xc1 + vpaddd @key[10],$xc2,$xc2 + vpaddd @key[11],$xc3,$xc3 + + vpunpckldq $xc1,$xc0,$xt2 + vpunpckldq $xc3,$xc2,$xt3 + vpunpckhdq $xc1,$xc0,$xc0 + vpunpckhdq $xc3,$xc2,$xc2 + vpunpcklqdq $xt3,$xt2,$xc1 # "c0" + vpunpckhqdq $xt3,$xt2,$xt2 # "c1" + vpunpcklqdq $xc2,$xc0,$xc3 # "c2" + vpunpckhqdq $xc2,$xc0,$xc0 # "c3" +___ + ($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2); +$code.=<<___; + vpaddd @key[12],$xd0,$xd0 + vpaddd @key[13],$xd1,$xd1 + vpaddd @key[14],$xd2,$xd2 + vpaddd @key[15],$xd3,$xd3 + + vpunpckldq $xd1,$xd0,$xt2 + vpunpckldq $xd3,$xd2,$xt3 + vpunpckhdq $xd1,$xd0,$xd0 + vpunpckhdq $xd3,$xd2,$xd2 + vpunpcklqdq $xt3,$xt2,$xd1 # "d0" + vpunpckhqdq $xt3,$xt2,$xt2 # "d1" + vpunpcklqdq $xd2,$xd0,$xd3 # "d2" + vpunpckhqdq $xd2,$xd0,$xd0 # "d3" +___ + ($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2); +$code.=<<___; + vshufi32x4 \$0x44,$xd0,$xc0,$xt3 # "de-interlace" further + vshufi32x4 \$0xee,$xd0,$xc0,$xd0 + vshufi32x4 \$0x44,$xd1,$xc1,$xc0 + vshufi32x4 \$0xee,$xd1,$xc1,$xd1 + vshufi32x4 \$0x44,$xd2,$xc2,$xc1 + vshufi32x4 \$0xee,$xd2,$xc2,$xd2 + vshufi32x4 \$0x44,$xd3,$xc3,$xc2 + vshufi32x4 \$0xee,$xd3,$xc3,$xd3 +___ + ($xc0,$xc1,$xc2,$xc3,$xt3)=($xt3,$xc0,$xc1,$xc2,$xc3); +$code.=<<___; + vshufi32x4 \$0x88,$xc0,$xa0,$xt0 # "de-interlace" further + vshufi32x4 \$0xdd,$xc0,$xa0,$xa0 + vshufi32x4 \$0x88,$xd0,$xb0,$xc0 + vshufi32x4 \$0xdd,$xd0,$xb0,$xd0 + vshufi32x4 \$0x88,$xc1,$xa1,$xt1 + vshufi32x4 \$0xdd,$xc1,$xa1,$xa1 + vshufi32x4 \$0x88,$xd1,$xb1,$xc1 + vshufi32x4 \$0xdd,$xd1,$xb1,$xd1 + vshufi32x4 \$0x88,$xc2,$xa2,$xt2 + vshufi32x4 \$0xdd,$xc2,$xa2,$xa2 + vshufi32x4 \$0x88,$xd2,$xb2,$xc2 + vshufi32x4 \$0xdd,$xd2,$xb2,$xd2 + vshufi32x4 \$0x88,$xc3,$xa3,$xt3 + vshufi32x4 \$0xdd,$xc3,$xa3,$xa3 + vshufi32x4 \$0x88,$xd3,$xb3,$xc3 + vshufi32x4 \$0xdd,$xd3,$xb3,$xd3 +___ + ($xa0,$xa1,$xa2,$xa3,$xb0,$xb1,$xb2,$xb3)= + ($xt0,$xt1,$xt2,$xt3,$xa0,$xa1,$xa2,$xa3); + + ($xa0,$xb0,$xc0,$xd0, $xa1,$xb1,$xc1,$xd1, + $xa2,$xb2,$xc2,$xd2, $xa3,$xb3,$xc3,$xd3) = + ($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3); +$code.=<<___; + cmp \$64*16,$len + jb .Ltail16x + + vpxord 0x00($inp),$xa0,$xa0 # xor with input + vpxord 0x40($inp),$xb0,$xb0 + vpxord 0x80($inp),$xc0,$xc0 + vpxord 0xc0($inp),$xd0,$xd0 + vmovdqu32 $xa0,0x00($out) + vmovdqu32 $xb0,0x40($out) + vmovdqu32 $xc0,0x80($out) + vmovdqu32 $xd0,0xc0($out) + + vpxord 0x100($inp),$xa1,$xa1 + vpxord 0x140($inp),$xb1,$xb1 + vpxord 0x180($inp),$xc1,$xc1 + vpxord 0x1c0($inp),$xd1,$xd1 + vmovdqu32 $xa1,0x100($out) + vmovdqu32 $xb1,0x140($out) + vmovdqu32 $xc1,0x180($out) + vmovdqu32 $xd1,0x1c0($out) + + vpxord 0x200($inp),$xa2,$xa2 + vpxord 0x240($inp),$xb2,$xb2 + vpxord 0x280($inp),$xc2,$xc2 + vpxord 0x2c0($inp),$xd2,$xd2 + vmovdqu32 $xa2,0x200($out) + vmovdqu32 $xb2,0x240($out) + vmovdqu32 $xc2,0x280($out) + vmovdqu32 $xd2,0x2c0($out) + + vpxord 0x300($inp),$xa3,$xa3 + vpxord 0x340($inp),$xb3,$xb3 + vpxord 0x380($inp),$xc3,$xc3 + vpxord 0x3c0($inp),$xd3,$xd3 + lea 0x400($inp),$inp + vmovdqu32 $xa3,0x300($out) + vmovdqu32 $xb3,0x340($out) + vmovdqu32 $xc3,0x380($out) + vmovdqu32 $xd3,0x3c0($out) + lea 0x400($out),$out + + sub \$64*16,$len + jnz .Loop_outer16x + + jmp .Ldone16x + +.align 32 +.Ltail16x: + xor %r9,%r9 + sub $inp,$out + cmp \$64*1,$len + jb .Less_than_64_16x + vpxord ($inp),$xa0,$xa0 # xor with input + vmovdqu32 $xa0,($out,$inp) + je .Ldone16x + vmovdqa32 $xb0,$xa0 + lea 64($inp),$inp + + cmp \$64*2,$len + jb .Less_than_64_16x + vpxord ($inp),$xb0,$xb0 + vmovdqu32 $xb0,($out,$inp) + je .Ldone16x + vmovdqa32 $xc0,$xa0 + lea 64($inp),$inp + + cmp \$64*3,$len + jb .Less_than_64_16x + vpxord ($inp),$xc0,$xc0 + vmovdqu32 $xc0,($out,$inp) + je .Ldone16x + vmovdqa32 $xd0,$xa0 + lea 64($inp),$inp + + cmp \$64*4,$len + jb .Less_than_64_16x + vpxord ($inp),$xd0,$xd0 + vmovdqu32 $xd0,($out,$inp) + je .Ldone16x + vmovdqa32 $xa1,$xa0 + lea 64($inp),$inp + + cmp \$64*5,$len + jb .Less_than_64_16x + vpxord ($inp),$xa1,$xa1 + vmovdqu32 $xa1,($out,$inp) + je .Ldone16x + vmovdqa32 $xb1,$xa0 + lea 64($inp),$inp + + cmp \$64*6,$len + jb .Less_than_64_16x + vpxord ($inp),$xb1,$xb1 + vmovdqu32 $xb1,($out,$inp) + je .Ldone16x + vmovdqa32 $xc1,$xa0 + lea 64($inp),$inp + + cmp \$64*7,$len + jb .Less_than_64_16x + vpxord ($inp),$xc1,$xc1 + vmovdqu32 $xc1,($out,$inp) + je .Ldone16x + vmovdqa32 $xd1,$xa0 + lea 64($inp),$inp + + cmp \$64*8,$len + jb .Less_than_64_16x + vpxord ($inp),$xd1,$xd1 + vmovdqu32 $xd1,($out,$inp) + je .Ldone16x + vmovdqa32 $xa2,$xa0 + lea 64($inp),$inp + + cmp \$64*9,$len + jb .Less_than_64_16x + vpxord ($inp),$xa2,$xa2 + vmovdqu32 $xa2,($out,$inp) + je .Ldone16x + vmovdqa32 $xb2,$xa0 + lea 64($inp),$inp + + cmp \$64*10,$len + jb .Less_than_64_16x + vpxord ($inp),$xb2,$xb2 + vmovdqu32 $xb2,($out,$inp) + je .Ldone16x + vmovdqa32 $xc2,$xa0 + lea 64($inp),$inp + + cmp \$64*11,$len + jb .Less_than_64_16x + vpxord ($inp),$xc2,$xc2 + vmovdqu32 $xc2,($out,$inp) + je .Ldone16x + vmovdqa32 $xd2,$xa0 + lea 64($inp),$inp + + cmp \$64*12,$len + jb .Less_than_64_16x + vpxord ($inp),$xd2,$xd2 + vmovdqu32 $xd2,($out,$inp) + je .Ldone16x + vmovdqa32 $xa3,$xa0 + lea 64($inp),$inp + + cmp \$64*13,$len + jb .Less_than_64_16x + vpxord ($inp),$xa3,$xa3 + vmovdqu32 $xa3,($out,$inp) + je .Ldone16x + vmovdqa32 $xb3,$xa0 + lea 64($inp),$inp + + cmp \$64*14,$len + jb .Less_than_64_16x + vpxord ($inp),$xb3,$xb3 + vmovdqu32 $xb3,($out,$inp) + je .Ldone16x + vmovdqa32 $xc3,$xa0 + lea 64($inp),$inp + + cmp \$64*15,$len + jb .Less_than_64_16x + vpxord ($inp),$xc3,$xc3 + vmovdqu32 $xc3,($out,$inp) + je .Ldone16x + vmovdqa32 $xd3,$xa0 + lea 64($inp),$inp + +.Less_than_64_16x: + vmovdqa32 $xa0,0x00(%rsp) + lea ($out,$inp),$out + and \$63,$len + +.Loop_tail16x: + movzb ($inp,%r9),%eax + movzb (%rsp,%r9),%ecx + lea 1(%r9),%r9 + xor %ecx,%eax + mov %al,-1($out,%r9) + dec $len + jnz .Loop_tail16x + + vpxord $xa0,$xa0,$xa0 + vmovdqa32 $xa0,0(%rsp) + +.Ldone16x: + vzeroall +___ +$code.=<<___ if ($win64); + movaps -0xa8(%r10),%xmm6 + movaps -0x98(%r10),%xmm7 + movaps -0x88(%r10),%xmm8 + movaps -0x78(%r10),%xmm9 + movaps -0x68(%r10),%xmm10 + movaps -0x58(%r10),%xmm11 + movaps -0x48(%r10),%xmm12 + movaps -0x38(%r10),%xmm13 + movaps -0x28(%r10),%xmm14 + movaps -0x18(%r10),%xmm15 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.L16x_epilogue: + ret +.cfi_endproc +.size ChaCha20_16x,.-ChaCha20_16x +___ + +# switch to %ymm domain +($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3)=map("%ymm$_",(0..15)); +@xx=($xa0,$xa1,$xa2,$xa3, $xb0,$xb1,$xb2,$xb3, + $xc0,$xc1,$xc2,$xc3, $xd0,$xd1,$xd2,$xd3); +@key=map("%ymm$_",(16..31)); +($xt0,$xt1,$xt2,$xt3)=@key[0..3]; + +$code.=<<___; +.type ChaCha20_8xvl,\@function,5 +.align 32 +ChaCha20_8xvl: +.cfi_startproc +.LChaCha20_8xvl: + mov %rsp,%r10 # frame register +.cfi_def_cfa_register %r10 + sub \$64+$xframe,%rsp + and \$-64,%rsp +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0xa8(%r10) + movaps %xmm7,-0x98(%r10) + movaps %xmm8,-0x88(%r10) + movaps %xmm9,-0x78(%r10) + movaps %xmm10,-0x68(%r10) + movaps %xmm11,-0x58(%r10) + movaps %xmm12,-0x48(%r10) + movaps %xmm13,-0x38(%r10) + movaps %xmm14,-0x28(%r10) + movaps %xmm15,-0x18(%r10) +.L8xvl_body: +___ +$code.=<<___; + vzeroupper + + lea .Lsigma(%rip),%r9 + vbroadcasti128 (%r9),$xa3 # key[0] + vbroadcasti128 ($key),$xb3 # key[1] + vbroadcasti128 16($key),$xc3 # key[2] + vbroadcasti128 ($counter),$xd3 # key[3] + + vpshufd \$0x00,$xa3,$xa0 # smash key by lanes... + vpshufd \$0x55,$xa3,$xa1 + vpshufd \$0xaa,$xa3,$xa2 + vpshufd \$0xff,$xa3,$xa3 + vmovdqa64 $xa0,@key[0] + vmovdqa64 $xa1,@key[1] + vmovdqa64 $xa2,@key[2] + vmovdqa64 $xa3,@key[3] + + vpshufd \$0x00,$xb3,$xb0 + vpshufd \$0x55,$xb3,$xb1 + vpshufd \$0xaa,$xb3,$xb2 + vpshufd \$0xff,$xb3,$xb3 + vmovdqa64 $xb0,@key[4] + vmovdqa64 $xb1,@key[5] + vmovdqa64 $xb2,@key[6] + vmovdqa64 $xb3,@key[7] + + vpshufd \$0x00,$xc3,$xc0 + vpshufd \$0x55,$xc3,$xc1 + vpshufd \$0xaa,$xc3,$xc2 + vpshufd \$0xff,$xc3,$xc3 + vmovdqa64 $xc0,@key[8] + vmovdqa64 $xc1,@key[9] + vmovdqa64 $xc2,@key[10] + vmovdqa64 $xc3,@key[11] + + vpshufd \$0x00,$xd3,$xd0 + vpshufd \$0x55,$xd3,$xd1 + vpshufd \$0xaa,$xd3,$xd2 + vpshufd \$0xff,$xd3,$xd3 + vpaddd .Lincy(%rip),$xd0,$xd0 # don't save counters yet + vmovdqa64 $xd0,@key[12] + vmovdqa64 $xd1,@key[13] + vmovdqa64 $xd2,@key[14] + vmovdqa64 $xd3,@key[15] + + mov \$10,%eax + jmp .Loop8xvl + +.align 32 +.Loop_outer8xvl: + #vpbroadcastd 0(%r9),$xa0 # reload key + #vpbroadcastd 4(%r9),$xa1 + vpbroadcastd 8(%r9),$xa2 + vpbroadcastd 12(%r9),$xa3 + vpaddd .Leight(%rip),@key[12],@key[12] # next SIMD counters + vmovdqa64 @key[4],$xb0 + vmovdqa64 @key[5],$xb1 + vmovdqa64 @key[6],$xb2 + vmovdqa64 @key[7],$xb3 + vmovdqa64 @key[8],$xc0 + vmovdqa64 @key[9],$xc1 + vmovdqa64 @key[10],$xc2 + vmovdqa64 @key[11],$xc3 + vmovdqa64 @key[12],$xd0 + vmovdqa64 @key[13],$xd1 + vmovdqa64 @key[14],$xd2 + vmovdqa64 @key[15],$xd3 + + vmovdqa64 $xa0,@key[0] + vmovdqa64 $xa1,@key[1] + vmovdqa64 $xa2,@key[2] + vmovdqa64 $xa3,@key[3] + + mov \$10,%eax + jmp .Loop8xvl + +.align 32 +.Loop8xvl: +___ + foreach (&AVX512_lane_ROUND(0, 4, 8,12)) { eval; } + foreach (&AVX512_lane_ROUND(0, 5,10,15)) { eval; } +$code.=<<___; + dec %eax + jnz .Loop8xvl + + vpaddd @key[0],$xa0,$xa0 # accumulate key + vpaddd @key[1],$xa1,$xa1 + vpaddd @key[2],$xa2,$xa2 + vpaddd @key[3],$xa3,$xa3 + + vpunpckldq $xa1,$xa0,$xt2 # "de-interlace" data + vpunpckldq $xa3,$xa2,$xt3 + vpunpckhdq $xa1,$xa0,$xa0 + vpunpckhdq $xa3,$xa2,$xa2 + vpunpcklqdq $xt3,$xt2,$xa1 # "a0" + vpunpckhqdq $xt3,$xt2,$xt2 # "a1" + vpunpcklqdq $xa2,$xa0,$xa3 # "a2" + vpunpckhqdq $xa2,$xa0,$xa0 # "a3" +___ + ($xa0,$xa1,$xa2,$xa3,$xt2)=($xa1,$xt2,$xa3,$xa0,$xa2); +$code.=<<___; + vpaddd @key[4],$xb0,$xb0 + vpaddd @key[5],$xb1,$xb1 + vpaddd @key[6],$xb2,$xb2 + vpaddd @key[7],$xb3,$xb3 + + vpunpckldq $xb1,$xb0,$xt2 + vpunpckldq $xb3,$xb2,$xt3 + vpunpckhdq $xb1,$xb0,$xb0 + vpunpckhdq $xb3,$xb2,$xb2 + vpunpcklqdq $xt3,$xt2,$xb1 # "b0" + vpunpckhqdq $xt3,$xt2,$xt2 # "b1" + vpunpcklqdq $xb2,$xb0,$xb3 # "b2" + vpunpckhqdq $xb2,$xb0,$xb0 # "b3" +___ + ($xb0,$xb1,$xb2,$xb3,$xt2)=($xb1,$xt2,$xb3,$xb0,$xb2); +$code.=<<___; + vshufi32x4 \$0,$xb0,$xa0,$xt3 # "de-interlace" further + vshufi32x4 \$3,$xb0,$xa0,$xb0 + vshufi32x4 \$0,$xb1,$xa1,$xa0 + vshufi32x4 \$3,$xb1,$xa1,$xb1 + vshufi32x4 \$0,$xb2,$xa2,$xa1 + vshufi32x4 \$3,$xb2,$xa2,$xb2 + vshufi32x4 \$0,$xb3,$xa3,$xa2 + vshufi32x4 \$3,$xb3,$xa3,$xb3 +___ + ($xa0,$xa1,$xa2,$xa3,$xt3)=($xt3,$xa0,$xa1,$xa2,$xa3); +$code.=<<___; + vpaddd @key[8],$xc0,$xc0 + vpaddd @key[9],$xc1,$xc1 + vpaddd @key[10],$xc2,$xc2 + vpaddd @key[11],$xc3,$xc3 + + vpunpckldq $xc1,$xc0,$xt2 + vpunpckldq $xc3,$xc2,$xt3 + vpunpckhdq $xc1,$xc0,$xc0 + vpunpckhdq $xc3,$xc2,$xc2 + vpunpcklqdq $xt3,$xt2,$xc1 # "c0" + vpunpckhqdq $xt3,$xt2,$xt2 # "c1" + vpunpcklqdq $xc2,$xc0,$xc3 # "c2" + vpunpckhqdq $xc2,$xc0,$xc0 # "c3" +___ + ($xc0,$xc1,$xc2,$xc3,$xt2)=($xc1,$xt2,$xc3,$xc0,$xc2); +$code.=<<___; + vpaddd @key[12],$xd0,$xd0 + vpaddd @key[13],$xd1,$xd1 + vpaddd @key[14],$xd2,$xd2 + vpaddd @key[15],$xd3,$xd3 + + vpunpckldq $xd1,$xd0,$xt2 + vpunpckldq $xd3,$xd2,$xt3 + vpunpckhdq $xd1,$xd0,$xd0 + vpunpckhdq $xd3,$xd2,$xd2 + vpunpcklqdq $xt3,$xt2,$xd1 # "d0" + vpunpckhqdq $xt3,$xt2,$xt2 # "d1" + vpunpcklqdq $xd2,$xd0,$xd3 # "d2" + vpunpckhqdq $xd2,$xd0,$xd0 # "d3" +___ + ($xd0,$xd1,$xd2,$xd3,$xt2)=($xd1,$xt2,$xd3,$xd0,$xd2); +$code.=<<___; + vperm2i128 \$0x20,$xd0,$xc0,$xt3 # "de-interlace" further + vperm2i128 \$0x31,$xd0,$xc0,$xd0 + vperm2i128 \$0x20,$xd1,$xc1,$xc0 + vperm2i128 \$0x31,$xd1,$xc1,$xd1 + vperm2i128 \$0x20,$xd2,$xc2,$xc1 + vperm2i128 \$0x31,$xd2,$xc2,$xd2 + vperm2i128 \$0x20,$xd3,$xc3,$xc2 + vperm2i128 \$0x31,$xd3,$xc3,$xd3 +___ + ($xc0,$xc1,$xc2,$xc3,$xt3)=($xt3,$xc0,$xc1,$xc2,$xc3); + ($xb0,$xb1,$xb2,$xb3,$xc0,$xc1,$xc2,$xc3)= + ($xc0,$xc1,$xc2,$xc3,$xb0,$xb1,$xb2,$xb3); +$code.=<<___; + cmp \$64*8,$len + jb .Ltail8xvl + + mov \$0x80,%eax # size optimization + vpxord 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vpxor 0x40($inp),$xc0,$xc0 + vpxor 0x60($inp),$xd0,$xd0 + lea ($inp,%rax),$inp # size optimization + vmovdqu32 $xa0,0x00($out) + vmovdqu $xb0,0x20($out) + vmovdqu $xc0,0x40($out) + vmovdqu $xd0,0x60($out) + lea ($out,%rax),$out # size optimization + + vpxor 0x00($inp),$xa1,$xa1 + vpxor 0x20($inp),$xb1,$xb1 + vpxor 0x40($inp),$xc1,$xc1 + vpxor 0x60($inp),$xd1,$xd1 + lea ($inp,%rax),$inp # size optimization + vmovdqu $xa1,0x00($out) + vmovdqu $xb1,0x20($out) + vmovdqu $xc1,0x40($out) + vmovdqu $xd1,0x60($out) + lea ($out,%rax),$out # size optimization + + vpxord 0x00($inp),$xa2,$xa2 + vpxor 0x20($inp),$xb2,$xb2 + vpxor 0x40($inp),$xc2,$xc2 + vpxor 0x60($inp),$xd2,$xd2 + lea ($inp,%rax),$inp # size optimization + vmovdqu32 $xa2,0x00($out) + vmovdqu $xb2,0x20($out) + vmovdqu $xc2,0x40($out) + vmovdqu $xd2,0x60($out) + lea ($out,%rax),$out # size optimization + + vpxor 0x00($inp),$xa3,$xa3 + vpxor 0x20($inp),$xb3,$xb3 + vpxor 0x40($inp),$xc3,$xc3 + vpxor 0x60($inp),$xd3,$xd3 + lea ($inp,%rax),$inp # size optimization + vmovdqu $xa3,0x00($out) + vmovdqu $xb3,0x20($out) + vmovdqu $xc3,0x40($out) + vmovdqu $xd3,0x60($out) + lea ($out,%rax),$out # size optimization + + vpbroadcastd 0(%r9),%ymm0 # reload key + vpbroadcastd 4(%r9),%ymm1 + + sub \$64*8,$len + jnz .Loop_outer8xvl + + jmp .Ldone8xvl + +.align 32 +.Ltail8xvl: + vmovdqa64 $xa0,%ymm8 # size optimization +___ +$xa0 = "%ymm8"; +$code.=<<___; + xor %r9,%r9 + sub $inp,$out + cmp \$64*1,$len + jb .Less_than_64_8xvl + vpxor 0x00($inp),$xa0,$xa0 # xor with input + vpxor 0x20($inp),$xb0,$xb0 + vmovdqu $xa0,0x00($out,$inp) + vmovdqu $xb0,0x20($out,$inp) + je .Ldone8xvl + vmovdqa $xc0,$xa0 + vmovdqa $xd0,$xb0 + lea 64($inp),$inp + + cmp \$64*2,$len + jb .Less_than_64_8xvl + vpxor 0x00($inp),$xc0,$xc0 + vpxor 0x20($inp),$xd0,$xd0 + vmovdqu $xc0,0x00($out,$inp) + vmovdqu $xd0,0x20($out,$inp) + je .Ldone8xvl + vmovdqa $xa1,$xa0 + vmovdqa $xb1,$xb0 + lea 64($inp),$inp + + cmp \$64*3,$len + jb .Less_than_64_8xvl + vpxor 0x00($inp),$xa1,$xa1 + vpxor 0x20($inp),$xb1,$xb1 + vmovdqu $xa1,0x00($out,$inp) + vmovdqu $xb1,0x20($out,$inp) + je .Ldone8xvl + vmovdqa $xc1,$xa0 + vmovdqa $xd1,$xb0 + lea 64($inp),$inp + + cmp \$64*4,$len + jb .Less_than_64_8xvl + vpxor 0x00($inp),$xc1,$xc1 + vpxor 0x20($inp),$xd1,$xd1 + vmovdqu $xc1,0x00($out,$inp) + vmovdqu $xd1,0x20($out,$inp) + je .Ldone8xvl + vmovdqa32 $xa2,$xa0 + vmovdqa $xb2,$xb0 + lea 64($inp),$inp + + cmp \$64*5,$len + jb .Less_than_64_8xvl + vpxord 0x00($inp),$xa2,$xa2 + vpxor 0x20($inp),$xb2,$xb2 + vmovdqu32 $xa2,0x00($out,$inp) + vmovdqu $xb2,0x20($out,$inp) + je .Ldone8xvl + vmovdqa $xc2,$xa0 + vmovdqa $xd2,$xb0 + lea 64($inp),$inp + + cmp \$64*6,$len + jb .Less_than_64_8xvl + vpxor 0x00($inp),$xc2,$xc2 + vpxor 0x20($inp),$xd2,$xd2 + vmovdqu $xc2,0x00($out,$inp) + vmovdqu $xd2,0x20($out,$inp) + je .Ldone8xvl + vmovdqa $xa3,$xa0 + vmovdqa $xb3,$xb0 + lea 64($inp),$inp + + cmp \$64*7,$len + jb .Less_than_64_8xvl + vpxor 0x00($inp),$xa3,$xa3 + vpxor 0x20($inp),$xb3,$xb3 + vmovdqu $xa3,0x00($out,$inp) + vmovdqu $xb3,0x20($out,$inp) + je .Ldone8xvl + vmovdqa $xc3,$xa0 + vmovdqa $xd3,$xb0 + lea 64($inp),$inp + +.Less_than_64_8xvl: + vmovdqa $xa0,0x00(%rsp) + vmovdqa $xb0,0x20(%rsp) + lea ($out,$inp),$out + and \$63,$len + +.Loop_tail8xvl: + movzb ($inp,%r9),%eax + movzb (%rsp,%r9),%ecx + lea 1(%r9),%r9 + xor %ecx,%eax + mov %al,-1($out,%r9) + dec $len + jnz .Loop_tail8xvl + + vpxor $xa0,$xa0,$xa0 + vmovdqa $xa0,0x00(%rsp) + vmovdqa $xa0,0x20(%rsp) + +.Ldone8xvl: + vzeroall +___ +$code.=<<___ if ($win64); + movaps -0xa8(%r10),%xmm6 + movaps -0x98(%r10),%xmm7 + movaps -0x88(%r10),%xmm8 + movaps -0x78(%r10),%xmm9 + movaps -0x68(%r10),%xmm10 + movaps -0x58(%r10),%xmm11 + movaps -0x48(%r10),%xmm12 + movaps -0x38(%r10),%xmm13 + movaps -0x28(%r10),%xmm14 + movaps -0x18(%r10),%xmm15 +___ +$code.=<<___; + lea (%r10),%rsp +.cfi_def_cfa_register %rsp +.L8xvl_epilogue: + ret +.cfi_endproc +.size ChaCha20_8xvl,.-ChaCha20_8xvl +___ +} + +# EXCEPTION_DISPOSITION handler (EXCEPTION_RECORD *rec,ULONG64 frame, +# CONTEXT *context,DISPATCHER_CONTEXT *disp) +if ($win64) { +$rec="%rcx"; +$frame="%rdx"; +$context="%r8"; +$disp="%r9"; + +$code.=<<___; +.extern __imp_RtlVirtualUnwind +.type se_handler,\@abi-omnipotent +.align 16 +se_handler: + push %rsi + push %rdi + push %rbx + push %rbp + push %r12 + push %r13 + push %r14 + push %r15 + pushfq + sub \$64,%rsp + + mov 120($context),%rax # pull context->Rax + mov 248($context),%rbx # pull context->Rip + + mov 8($disp),%rsi # disp->ImageBase + mov 56($disp),%r11 # disp->HandlerData + + lea .Lctr32_body(%rip),%r10 + cmp %r10,%rbx # context->Rip<.Lprologue + jb .Lcommon_seh_tail + + mov 152($context),%rax # pull context->Rsp + + lea .Lno_data(%rip),%r10 # epilogue label + cmp %r10,%rbx # context->Rip>=.Lepilogue + jae .Lcommon_seh_tail + + lea 64+24+48(%rax),%rax + + mov -8(%rax),%rbx + mov -16(%rax),%rbp + mov -24(%rax),%r12 + mov -32(%rax),%r13 + mov -40(%rax),%r14 + mov -48(%rax),%r15 + mov %rbx,144($context) # restore context->Rbx + mov %rbp,160($context) # restore context->Rbp + mov %r12,216($context) # restore context->R12 + mov %r13,224($context) # restore context->R13 + mov %r14,232($context) # restore context->R14 + mov %r15,240($context) # restore context->R14 + +.Lcommon_seh_tail: + mov 8(%rax),%rdi + mov 16(%rax),%rsi + mov %rax,152($context) # restore context->Rsp + mov %rsi,168($context) # restore context->Rsi + mov %rdi,176($context) # restore context->Rdi + + mov 40($disp),%rdi # disp->ContextRecord + mov $context,%rsi # context + mov \$154,%ecx # sizeof(CONTEXT) + .long 0xa548f3fc # cld; rep movsq + + mov $disp,%rsi + xor %rcx,%rcx # arg1, UNW_FLAG_NHANDLER + mov 8(%rsi),%rdx # arg2, disp->ImageBase + mov 0(%rsi),%r8 # arg3, disp->ControlPc + mov 16(%rsi),%r9 # arg4, disp->FunctionEntry + mov 40(%rsi),%r10 # disp->ContextRecord + lea 56(%rsi),%r11 # &disp->HandlerData + lea 24(%rsi),%r12 # &disp->EstablisherFrame + mov %r10,32(%rsp) # arg5 + mov %r11,40(%rsp) # arg6 + mov %r12,48(%rsp) # arg7 + mov %rcx,56(%rsp) # arg8, (NULL) + call *__imp_RtlVirtualUnwind(%rip) + + mov \$1,%eax # ExceptionContinueSearch + add \$64,%rsp + popfq + pop %r15 + pop %r14 + pop %r13 + pop %r12 + pop %rbp + pop %rbx + pop %rdi + pop %rsi + ret +.size se_handler,.-se_handler + +.type simd_handler,\@abi-omnipotent +.align 16 +simd_handler: + push %rsi + push %rdi + push %rbx + push %rbp + push %r12 + push %r13 + push %r14 + push %r15 + pushfq + sub \$64,%rsp + + mov 120($context),%rax # pull context->Rax + mov 248($context),%rbx # pull context->Rip + + mov 8($disp),%rsi # disp->ImageBase + mov 56($disp),%r11 # disp->HandlerData + + mov 0(%r11),%r10d # HandlerData[0] + lea (%rsi,%r10),%r10 # prologue label + cmp %r10,%rbx # context->RipR10 + + mov 4(%r11),%r10d # HandlerData[1] + mov 8(%r11),%ecx # HandlerData[2] + lea (%rsi,%r10),%r10 # epilogue label + cmp %r10,%rbx # context->Rip>=epilogue label + jae .Lcommon_seh_tail + + neg %rcx + lea -8(%rax,%rcx),%rsi + lea 512($context),%rdi # &context.Xmm6 + neg %ecx + shr \$3,%ecx + .long 0xa548f3fc # cld; rep movsq + + jmp .Lcommon_seh_tail +.size simd_handler,.-simd_handler + +.section .pdata +.align 4 + .rva .LSEH_begin_ChaCha20_ctr32 + .rva .LSEH_end_ChaCha20_ctr32 + .rva .LSEH_info_ChaCha20_ctr32 + + .rva .LSEH_begin_ChaCha20_ssse3 + .rva .LSEH_end_ChaCha20_ssse3 + .rva .LSEH_info_ChaCha20_ssse3 + + .rva .LSEH_begin_ChaCha20_128 + .rva .LSEH_end_ChaCha20_128 + .rva .LSEH_info_ChaCha20_128 + + .rva .LSEH_begin_ChaCha20_4x + .rva .LSEH_end_ChaCha20_4x + .rva .LSEH_info_ChaCha20_4x +___ +$code.=<<___ if ($avx); + .rva .LSEH_begin_ChaCha20_4xop + .rva .LSEH_end_ChaCha20_4xop + .rva .LSEH_info_ChaCha20_4xop +___ +$code.=<<___ if ($avx>1); + .rva .LSEH_begin_ChaCha20_avx2 + .rva .LSEH_end_ChaCha20_avx2 + .rva .LSEH_info_ChaCha20_avx2 +___ +$code.=<<___ if ($avx>2); + .rva .LSEH_begin_ChaCha20_avx512 + .rva .LSEH_end_ChaCha20_avx512 + .rva .LSEH_info_ChaCha20_avx512 + + .rva .LSEH_begin_ChaCha20_avx512vl + .rva .LSEH_end_ChaCha20_avx512vl + .rva .LSEH_info_ChaCha20_avx512vl + + .rva .LSEH_begin_ChaCha20_16x + .rva .LSEH_end_ChaCha20_16x + .rva .LSEH_info_ChaCha20_16x + + .rva .LSEH_begin_ChaCha20_8xvl + .rva .LSEH_end_ChaCha20_8xvl + .rva .LSEH_info_ChaCha20_8xvl +___ +$code.=<<___; +.section .xdata +.align 8 +.LSEH_info_ChaCha20_ctr32: + .byte 9,0,0,0 + .rva se_handler + +.LSEH_info_ChaCha20_ssse3: + .byte 9,0,0,0 + .rva simd_handler + .rva .Lssse3_body,.Lssse3_epilogue + .long 0x20,0 + +.LSEH_info_ChaCha20_128: + .byte 9,0,0,0 + .rva simd_handler + .rva .L128_body,.L128_epilogue + .long 0x60,0 + +.LSEH_info_ChaCha20_4x: + .byte 9,0,0,0 + .rva simd_handler + .rva .L4x_body,.L4x_epilogue + .long 0xa0,0 +___ +$code.=<<___ if ($avx); +.LSEH_info_ChaCha20_4xop: + .byte 9,0,0,0 + .rva simd_handler + .rva .L4xop_body,.L4xop_epilogue # HandlerData[] + .long 0xa0,0 +___ +$code.=<<___ if ($avx>1); +.LSEH_info_ChaCha20_avx2: + .byte 9,0,0,0 + .rva simd_handler + .rva .Lavx2_body,.Lavx2_epilogue # HandlerData[] + .long 0xa0,0 +___ +$code.=<<___ if ($avx>2); +.LSEH_info_ChaCha20_avx512: + .byte 9,0,0,0 + .rva simd_handler + .rva .Lavx512_body,.Lavx512_epilogue # HandlerData[] + .long 0x20,0 + +.LSEH_info_ChaCha20_avx512vl: + .byte 9,0,0,0 + .rva simd_handler + .rva .Lavx512vl_body,.Lavx512vl_epilogue # HandlerData[] + .long 0x20,0 + +.LSEH_info_ChaCha20_16x: + .byte 9,0,0,0 + .rva simd_handler + .rva .L16x_body,.L16x_epilogue # HandlerData[] + .long 0xa0,0 + +.LSEH_info_ChaCha20_8xvl: + .byte 9,0,0,0 + .rva simd_handler + .rva .L8xvl_body,.L8xvl_epilogue # HandlerData[] + .long 0xa0,0 +___ +} + +foreach (split("\n",$code)) { + s/\`([^\`]*)\`/eval $1/ge; + + s/%x#%[yz]/%x/g; # "down-shift" + + print $_,"\n"; +} + +close STDOUT; diff --git a/cbits/asm/generate.sh b/cbits/asm/generate.sh new file mode 100644 --- /dev/null +++ b/cbits/asm/generate.sh @@ -0,0 +1,153 @@ +#!/bin/sh +# +# Regenerate the assembly checked in beside this script. +# +# The .pl files come from the CRYPTOGAMS distribution, unmodified: +# +# https://github.com/dot-asm/cryptogams +# x86_64/aesni-gcm-x86_64.pl x86_64/chacha-x86_64.pl +# x86_64/poly1305-x86_64.pl x86_64/sha512-x86_64.pl +# x86_64/keccak1600-x86_64.pl x86_64/x86_64-xlate.pl +# arm/chacha-armv8.pl arm/poly1305-armv8.pl +# arm/sha1-armv8.pl arm/sha512-armv8.pl +# arm/keccak1600-armv8.pl arm/arm-xlate.pl +# arm/arm_arch.h +# +# The .pl files are the generator, not the product: each one emits +# assembly for a given "flavour", which is the calling convention and the +# object format together. The output is checked in so that building +# crypton needs no perl. +# +# Two things are done to the output here. The entry points are renamed: +# a program that links both crypton and OpenSSL would otherwise have two +# definitions of, say, aesni_gcm_encrypt, and the linker is entitled to +# refuse that. The same goes for OPENSSL_armcap_P, which the ChaCha +# module reads to find out whether the processor has NEON, and which +# crypton defines for itself in cbits/crypton_chacha.c. And the ELF +# output of the x86-64 module is given the note that says the code does +# not want an executable stack, which the generator leaves to the caller. +# +# Run this on a GNU/Linux host. The mingw64 flavour asks the compiler +# what __USER_LABEL_PREFIX__ is for its target, and a compiler for a +# platform that decorates symbols -- Apple's, for one -- answers for +# itself rather than for Windows, which would leave every entry point in +# that file with a leading underscore that nothing looks for. +# +# Usage: cd cbits/asm && ./generate.sh + +set -e + +# The x86-64 generators choose what to emit from the version of the +# assembler they are told about, so they are told one, rather than left to +# ask whatever compiler happens to be here: the checked-in files should not +# depend on the host that produced them. 2.24 predates AVX-512, which is +# the point -- the Poly1305 module has paths for it, and this does not take +# them, no machine here being able to run them, and a path nothing has +# executed not being worth the few per cent it might be worth. It leaves +# both modules with everything through AVX2. +cat > tmp-cc <<'SHIM' +#!/bin/sh +case "$*" in +*-Wa,-v*) echo "GNU assembler version 2.24" ;; +esac +exit 0 +SHIM +chmod +x tmp-cc +CC=./tmp-cc +export CC + +for flavour in elf macosx mingw64; do + perl aesni-gcm-x86_64.pl $flavour tmp-$flavour.S + sed -e 's/aesni_gcm_/crypton_gcm_asm_/g' \ + -e 's/aesni_ctr32_/crypton_gcm_asm_ctr32_/g' \ + tmp-$flavour.S > aesni-gcm-x86_64-$flavour.S + + perl poly1305-x86_64.pl $flavour tmp-$flavour.S + sed -e 's/poly1305_/crypton_poly1305_asm_/g' \ + -e 's/xor128_/crypton_xor128_/g' \ + -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \ + tmp-$flavour.S > poly1305-x86_64-$flavour.S + + perl chacha-x86_64.pl $flavour tmp-$flavour.S + sed -e 's/ChaCha20_/crypton_chacha20_asm_/g' \ + -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \ + tmp-$flavour.S > chacha-x86_64-$flavour.S + + # as on AArch64, this generator emits SHA-512 or SHA-256 according + # to the name it is given, and both are wanted here + perl sha512-x86_64.pl $flavour tmp-$flavour.S + sed -e 's/sha256_block_/crypton_sha256_asm_block_/g' \ + -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \ + tmp-$flavour.S > sha256-x86_64-$flavour.S + + perl keccak1600-x86_64.pl $flavour tmp-k-$flavour.S + sed -e 's/SHA3_absorb/crypton_keccak_asm_absorb/g' \ + -e 's/SHA3_squeeze/crypton_keccak_asm_squeeze/g' \ + -e 's/KeccakF1600/crypton_keccak_asm_f1600/g' \ + tmp-k-$flavour.S > keccak1600-x86_64-$flavour.S + + perl sha512-x86_64.pl $flavour tmp-512-$flavour.S + sed -e 's/sha512_block_/crypton_sha512_asm_block_/g' \ + -e 's/OPENSSL_ia32cap_P/crypton_ia32cap_P/g' \ + tmp-512-$flavour.S > sha512-x86_64-$flavour.S + rm -f tmp-$flavour.S tmp-512-$flavour.S tmp-k-$flavour.S +done + +for f in aesni-gcm-x86_64-elf.S poly1305-x86_64-elf.S chacha-x86_64-elf.S \ + sha256-x86_64-elf.S sha512-x86_64-elf.S keccak1600-x86_64-elf.S; do + cat >> $f <<-NOTE + + .section .note.GNU-stack,"",@progbits + NOTE +done + +unset CC +rm -f tmp-cc + +for flavour in linux64 ios64; do + perl chacha-armv8.pl $flavour tmp-$flavour.S + sed -e 's/ChaCha20_/crypton_chacha20_asm_/g' \ + -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \ + tmp-$flavour.S > chacha-armv8-$flavour.S + + perl poly1305-armv8.pl $flavour tmp-$flavour.S + sed -e 's/poly1305_/crypton_poly1305_asm_/g' \ + -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \ + tmp-$flavour.S > poly1305-armv8-$flavour.S + + # the same generator emits SHA-512 or SHA-256 according to the name + # it is given, and only the SHA-256 one is wanted here + perl sha512-armv8.pl $flavour tmp-$flavour.S + sed -e 's/sha256_block_/crypton_sha256_asm_block_/g' \ + -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \ + tmp-$flavour.S > sha256-armv8-$flavour.S + + perl sha1-armv8.pl $flavour tmp-$flavour.S + sed -e 's/sha1_block_/crypton_sha1_asm_block_/g' \ + -e 's/OPENSSL_armcap_P/crypton_armcap_P/g' \ + tmp-$flavour.S > sha1-armv8-$flavour.S + + perl keccak1600-armv8.pl $flavour tmp-$flavour.S + sed -e 's/SHA3_absorb/crypton_keccak_asm_absorb/g' \ + -e 's/SHA3_squeeze/crypton_keccak_asm_squeeze/g' \ + tmp-$flavour.S > keccak1600-armv8-$flavour.S + rm -f tmp-$flavour.S +done + +cat >> chacha-armv8-linux64.S <<'NOTE' + +.section .note.GNU-stack,"",%progbits +NOTE + +cat >> poly1305-armv8-linux64.S <<'NOTE' + +.section .note.GNU-stack,"",%progbits +NOTE + +for f in sha1-armv8-linux64.S sha256-armv8-linux64.S \ + keccak1600-armv8-linux64.S; do + cat >> $f <<-NOTE + + .section .note.GNU-stack,"",%progbits + NOTE +done diff --git a/cbits/asm/keccak1600-armv8-ios64.S b/cbits/asm/keccak1600-armv8-ios64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-armv8-ios64.S @@ -0,0 +1,841 @@ +.text + +.align 8 // strategic alignment and padding that allows to use + // address value as loop termination condition... +.quad 0,0,0,0,0,0,0,0 + +iotas: +.quad 0x0000000000000001 +.quad 0x0000000000008082 +.quad 0x800000000000808a +.quad 0x8000000080008000 +.quad 0x000000000000808b +.quad 0x0000000080000001 +.quad 0x8000000080008081 +.quad 0x8000000000008009 +.quad 0x000000000000008a +.quad 0x0000000000000088 +.quad 0x0000000080008009 +.quad 0x000000008000000a +Liotas12: +.quad 0x000000008000808b +.quad 0x800000000000008b +.quad 0x8000000000008089 +.quad 0x8000000000008003 +.quad 0x8000000000008002 +.quad 0x8000000000000080 +.quad 0x000000000000800a +.quad 0x800000008000000a +.quad 0x8000000080008081 +.quad 0x8000000000008080 +.quad 0x0000000080000001 +.quad 0x8000000080008008 + + +.align 5 +KeccakF1600_int: +.long 0xd503233f // paciasp + stp x28,x30,[sp,#16] // stack is pre-allocated + b Loop +.align 4 +Loop: + ////////////////////////////////////////// Theta + eor x26,x0,x5 + stp x4,x9,[sp,#0] // offload pair... + eor x27,x1,x6 + eor x28,x2,x7 + eor x30,x3,x8 + eor x4,x4,x9 + eor x26,x26,x10 + eor x27,x27,x11 + eor x28,x28,x12 + eor x30,x30,x13 + eor x4,x4,x14 + eor x26,x26,x15 + eor x27,x27,x16 + eor x28,x28,x17 + eor x30,x30,x25 + eor x4,x4,x19 + eor x26,x26,x20 + eor x28,x28,x22 + eor x27,x27,x21 + eor x30,x30,x23 + eor x4,x4,x24 + + eor x9,x26,x28,ror#63 + + eor x1,x1,x9 + eor x6,x6,x9 + eor x11,x11,x9 + eor x16,x16,x9 + eor x21,x21,x9 + + eor x9,x27,x30,ror#63 + eor x28,x28,x4,ror#63 + eor x30,x30,x26,ror#63 + eor x4,x4,x27,ror#63 + + eor x27, x2,x9 // mov x27,x2 + eor x7,x7,x9 + eor x12,x12,x9 + eor x17,x17,x9 + eor x22,x22,x9 + + eor x0,x0,x4 + eor x5,x5,x4 + eor x10,x10,x4 + eor x15,x15,x4 + eor x20,x20,x4 + ldp x4,x9,[sp,#0] // re-load offloaded data + eor x26, x3,x28 // mov x26,x3 + eor x8,x8,x28 + eor x13,x13,x28 + eor x25,x25,x28 + eor x23,x23,x28 + + eor x28, x4,x30 // mov x28,x4 + eor x9,x9,x30 + eor x14,x14,x30 + eor x19,x19,x30 + eor x24,x24,x30 + + ////////////////////////////////////////// Rho+Pi + mov x30,x1 + ror x1,x6,#64-44 + //mov x27,x2 + ror x2,x12,#64-43 + //mov x26,x3 + ror x3,x25,#64-21 // ? + //mov x28,x4 + ror x4,x24,#64-14 // ? + + ror x6,x9,#64-20 // ? + ror x12,x13,#64-25 // ? + ror x25,x17,#64-15 + ror x24,x21,#64-2 // ? + + ror x9,x22,#64-61 + ror x13,x19,#64-8 + ror x17,x11,#64-10 + ror x21,x8,#64-55 + + ror x22,x14,#64-39 + ror x19,x23,#64-56 + ror x11,x7,#64-6 // ? + ror x8,x16,#64-45 + + ror x14,x20,#64-18 + ror x23,x15,#64-41 + ror x7,x10,#64-3 + ror x16,x5,#64-36 // ? + + ror x5,x26,#64-28 // ? + ror x10,x30,#64-1 + ror x15,x28,#64-27 // ? + ror x20,x27,#64-62 // ? + + ////////////////////////////////////////// Chi+Iota + bic x26,x2,x1 + bic x27,x3,x2 + bic x28,x0,x4 + bic x30,x1,x0 + eor x0,x0,x26 + bic x26,x4,x3 + eor x1,x1,x27 + ldr x27,[sp,#16] + eor x3,x3,x28 + eor x4,x4,x30 + eor x2,x2,x26 + ldr x30,[x27],#8 // Iota[i++] + + bic x26,x7,x6 + tst x27,#255 // are we done? + str x27,[sp,#16] + bic x27,x8,x7 + bic x28,x5,x9 + eor x0,x0,x30 // A[0][0] ^= Iota + bic x30,x6,x5 + eor x5,x5,x26 + bic x26,x9,x8 + eor x6,x6,x27 + eor x8,x8,x28 + eor x9,x9,x30 + eor x7,x7,x26 + + bic x26,x12,x11 + bic x27,x13,x12 + bic x28,x10,x14 + bic x30,x11,x10 + eor x10,x10,x26 + bic x26,x14,x13 + eor x11,x11,x27 + eor x13,x13,x28 + eor x14,x14,x30 + eor x12,x12,x26 + + bic x26,x17,x16 + bic x27,x25,x17 + bic x28,x15,x19 + bic x30,x16,x15 + eor x15,x15,x26 + bic x26,x19,x25 + eor x16,x16,x27 + eor x25,x25,x28 + eor x19,x19,x30 + eor x17,x17,x26 + + bic x26,x22,x21 + bic x27,x23,x22 + bic x28,x20,x24 + bic x30,x21,x20 + eor x20,x20,x26 + bic x26,x24,x23 + eor x21,x21,x27 + eor x23,x23,x28 + eor x24,x24,x30 + eor x22,x22,x26 + + bne Loop + + ldr x30,[sp,#16+__SIZEOF_POINTER__] +.long 0xd50323bf // autiasp + ret + + + +.align 5 +KeccakF1600: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-16*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#16+4*__SIZEOF_POINTER__ + + str x0,[sp,#16+2*__SIZEOF_POINTER__] // offload argument + mov x26,x0 + ldp x0,x1,[x0,#16*0] + ldp x2,x3,[x26,#16*1] + ldp x4,x5,[x26,#16*2] + ldp x6,x7,[x26,#16*3] + ldp x8,x9,[x26,#16*4] + ldp x10,x11,[x26,#16*5] + ldp x12,x13,[x26,#16*6] + ldp x14,x15,[x26,#16*7] + ldp x16,x17,[x26,#16*8] + ldp x25,x19,[x26,#16*9] + ldp x20,x21,[x26,#16*10] + ldp x22,x23,[x26,#16*11] + ldr x24,[x26,#16*12] + + adr x28,iotas + bl KeccakF1600_int + + ldr x26,[sp,#16+2*__SIZEOF_POINTER__] + stp x0,x1,[x26,#16*0] + stp x2,x3,[x26,#16*1] + stp x4,x5,[x26,#16*2] + stp x6,x7,[x26,#16*3] + stp x8,x9,[x26,#16*4] + stp x10,x11,[x26,#16*5] + stp x12,x13,[x26,#16*6] + stp x14,x15,[x26,#16*7] + stp x16,x17,[x26,#16*8] + stp x25,x19,[x26,#16*9] + stp x20,x21,[x26,#16*10] + stp x22,x23,[x26,#16*11] + str x24,[x26,#16*12] + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#16+4*__SIZEOF_POINTER__ + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#16*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + + +.globl _crypton_keccak_asm_absorb + +.align 5 +_crypton_keccak_asm_absorb: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-16*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#16+4*__SIZEOF_POINTER__+16 + + stp x0,x1,[sp,#16+2*__SIZEOF_POINTER__] // offload arguments + stp x2,x3,[sp,#16+4*__SIZEOF_POINTER__] + + mov x26,x0 // uint64_t A[5][5] + mov x27,x1 // const void *inp + mov x28,x2 // size_t len + mov x30,x3 // size_t bsz + ldp x0,x1,[x26,#16*0] + ldp x2,x3,[x26,#16*1] + ldp x4,x5,[x26,#16*2] + ldp x6,x7,[x26,#16*3] + ldp x8,x9,[x26,#16*4] + ldp x10,x11,[x26,#16*5] + ldp x12,x13,[x26,#16*6] + ldp x14,x15,[x26,#16*7] + ldp x16,x17,[x26,#16*8] + ldp x25,x19,[x26,#16*9] + ldp x20,x21,[x26,#16*10] + ldp x22,x23,[x26,#16*11] + ldr x24,[x26,#16*12] + b Loop_absorb + +.align 4 +Loop_absorb: + subs x26,x28,x30 // len - bsz + blo Labsorbed + + str x26,[sp,#16+4*__SIZEOF_POINTER__] // save len - bsz + cmp x30,#104 + ldr x26,[x27,#0] // A[0][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x0,x0,x26 + ldr x26,[x27,#8] // A[0][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x1,x1,x26 + ldr x26,[x27,#16] // A[0][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x2,x2,x26 + ldr x26,[x27,#24] // A[0][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x3,x3,x26 + ldr x26,[x27,#32] // A[0][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x4,x4,x26 + ldr x26,[x27,#40] // A[1][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x5,x5,x26 + ldr x26,[x27,#48] // A[1][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x6,x6,x26 + ldr x26,[x27,#56] // A[1][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x7,x7,x26 + ldr x26,[x27,#64] // A[1][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x8,x8,x26 + blo Lprocess_block + + ldr x26,[x27,#72] // A[1][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x9,x9,x26 + ldr x26,[x27,#80] // A[2][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x10,x10,x26 + ldr x26,[x27,#88] // A[2][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x11,x11,x26 + ldr x26,[x27,#96] // A[2][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x12,x12,x26 + beq Lprocess_block + + cmp x30,#144 + ldr x26,[x27,#104] // A[2][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x13,x13,x26 + ldr x26,[x27,#112] // A[2][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x14,x14,x26 + ldr x26,[x27,#120] // A[3][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x15,x15,x26 + ldr x26,[x27,#128] // A[3][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x16,x16,x26 + blo Lprocess_block + + ldr x26,[x27,#136] // A[3][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x17,x17,x26 + beq Lprocess_block + + ldr x26,[x27,#144] // A[3][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x25,x25,x26 + ldr x26,[x27,#152] // A[3][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x19,x19,x26 + ldr x26,[x27,#160] // A[4][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x20,x20,x26 + +Lprocess_block: + add x27,x27,x30 + str x27,[sp,#16+3*__SIZEOF_POINTER__] // save inp + + adr x28,iotas + bl KeccakF1600_int + + ldr x27,[sp,#16+3*__SIZEOF_POINTER__] // restore arguments + ldp x28,x30,[sp,#16+4*__SIZEOF_POINTER__] + b Loop_absorb + +.align 4 +Labsorbed: + ldr x27,[sp,#16+2*__SIZEOF_POINTER__] + stp x0,x1,[x27,#16*0] + stp x2,x3,[x27,#16*1] + stp x4,x5,[x27,#16*2] + stp x6,x7,[x27,#16*3] + stp x8,x9,[x27,#16*4] + stp x10,x11,[x27,#16*5] + stp x12,x13,[x27,#16*6] + stp x14,x15,[x27,#16*7] + stp x16,x17,[x27,#16*8] + stp x25,x19,[x27,#16*9] + stp x20,x21,[x27,#16*10] + stp x22,x23,[x27,#16*11] + str x24,[x27,#16*12] + + mov x0,x28 // return value + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#16+4*__SIZEOF_POINTER__+16 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#16*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + +.globl _crypton_keccak_asm_squeeze + +.align 5 +_crypton_keccak_asm_squeeze: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-6*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + + mov x19,x0 // put aside arguments + mov x20,x1 + mov x21,x2 + mov x22,x3 + +Loop_squeeze: + ldr x4,[x0],#8 + cmp x21,#8 + blo Lsqueeze_tail +#ifdef __AARCH64EB__ + rev x4,x4 +#endif + str x4,[x20],#8 + subs x21,x21,#8 + beq Lsqueeze_done + + subs x3,x3,#8 + bhi Loop_squeeze + + mov x0,x19 + bl KeccakF1600 + mov x0,x19 + mov x3,x22 + b Loop_squeeze + +.align 4 +Lsqueeze_tail: + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq Lsqueeze_done + strb w4,[x20],#1 + +Lsqueeze_done: + ldp x19,x20,[sp,#2*__SIZEOF_POINTER__] + ldp x21,x22,[sp,#4*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#6*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + + +.align 5 +KeccakF1600_ce: +Loop_ce: + ////////////////////////////////////////////////// Theta +.long 0xce0f2a99 //eor3 v25.16b,v20.16b,v15.16b,v10.16b +.long 0xce102eba //eor3 v26.16b,v21.16b,v16.16b,v11.16b +.long 0xce1132db //eor3 v27.16b,v22.16b,v17.16b,v12.16b +.long 0xce1236fc //eor3 v28.16b,v23.16b,v18.16b,v13.16b +.long 0xce133b1d //eor3 v29.16b,v24.16b,v19.16b,v14.16b +.long 0xce050339 //eor3 v25.16b,v25.16b, v5.16b,v0.16b +.long 0xce06075a //eor3 v26.16b,v26.16b, v6.16b,v1.16b +.long 0xce070b7b //eor3 v27.16b,v27.16b, v7.16b,v2.16b +.long 0xce080f9c //eor3 v28.16b,v28.16b, v8.16b,v3.16b +.long 0xce0913bd //eor3 v29.16b,v29.16b, v9.16b,v4.16b + +.long 0xce7b8f3e //rax1 v30.2d,v25.2d,v27.2d // D[1] +.long 0xce7c8f5f //rax1 v31.2d,v26.2d,v28.2d // D[2] +.long 0xce7d8f7b //rax1 v27.2d,v27.2d,v29.2d // D[3] +.long 0xce798f9c //rax1 v28.2d,v28.2d,v25.2d // D[4] +.long 0xce7a8fbd //rax1 v29.2d,v29.2d,v26.2d // D[0] + + ////////////////////////////////////////////////// Theta+Rho+Pi +.long 0xce9efc39 //xar v25.2d, v1.2d,v30.2d,#64-1 // C[0]=A[2][0] + +.long 0xce9e50c1 //xar v1.2d,v6.2d,v30.2d,#64-44 +.long 0xce9cb126 //xar v6.2d,v9.2d,v28.2d,#64-20 +.long 0xce9f0ec9 //xar v9.2d,v22.2d,v31.2d,#64-61 +.long 0xce9c65d6 //xar v22.2d,v14.2d,v28.2d,#64-39 +.long 0xce9dba8e //xar v14.2d,v20.2d,v29.2d,#64-18 + +.long 0xce9f085a //xar v26.2d, v2.2d,v31.2d,#64-62 // C[1]=A[4][0] + +.long 0xce9f5582 //xar v2.2d,v12.2d,v31.2d,#64-43 +.long 0xce9b9dac //xar v12.2d,v13.2d,v27.2d,#64-25 +.long 0xce9ce26d //xar v13.2d,v19.2d,v28.2d,#64-8 +.long 0xce9b22f3 //xar v19.2d,v23.2d,v27.2d,#64-56 +.long 0xce9d5df7 //xar v23.2d,v15.2d,v29.2d,#64-41 + +.long 0xce9c948f //xar v15.2d,v4.2d,v28.2d,#64-27 + +.long 0xce9ccb1c //xar v28.2d, v24.2d,v28.2d,#64-14 // D[4]=A[0][4] +.long 0xce9efab8 //xar v24.2d,v21.2d,v30.2d,#64-2 +.long 0xce9b2508 //xar v8.2d,v8.2d,v27.2d,#64-55 // A[1][3]=A[4][1] +.long 0xce9e4e04 //xar v4.2d,v16.2d,v30.2d,#64-45 // A[0][4]=A[1][3] +.long 0xce9d70b0 //xar v16.2d,v5.2d,v29.2d,#64-36 + +.long 0xce9b9065 //xar v5.2d,v3.2d,v27.2d,#64-28 + + eor v0.16b,v0.16b,v29.16b + +.long 0xce9bae5b //xar v27.2d, v18.2d,v27.2d,#64-21 // D[3]=A[0][3] +.long 0xce9fc623 //xar v3.2d,v17.2d,v31.2d,#64-15 // A[0][3]=A[3][3] +.long 0xce9ed97e //xar v30.2d, v11.2d,v30.2d,#64-10 // D[1]=A[3][2] +.long 0xce9fe8ff //xar v31.2d, v7.2d,v31.2d,#64-6 // D[2]=A[2][1] +.long 0xce9df55d //xar v29.2d, v10.2d,v29.2d,#64-3 // D[0]=A[1][2] + + ////////////////////////////////////////////////// Chi+Iota +.long 0xce362354 //bcax v20.16b,v26.16b, v22.16b,v8.16b // A[1][3]=A[4][1] +.long 0xce375915 //bcax v21.16b,v8.16b,v23.16b,v22.16b // A[1][3]=A[4][1] +.long 0xce385ed6 //bcax v22.16b,v22.16b,v24.16b,v23.16b +.long 0xce3a62f7 //bcax v23.16b,v23.16b,v26.16b, v24.16b +.long 0xce286b18 //bcax v24.16b,v24.16b,v8.16b,v26.16b // A[1][3]=A[4][1] + + ld1r {v26.2d},[x10],#8 + +.long 0xce330fd1 //bcax v17.16b,v30.16b, v19.16b,v3.16b // A[0][3]=A[3][3] +.long 0xce2f4c72 //bcax v18.16b,v3.16b,v15.16b,v19.16b // A[0][3]=A[3][3] +.long 0xce303e73 //bcax v19.16b,v19.16b,v16.16b,v15.16b +.long 0xce3e41ef //bcax v15.16b,v15.16b,v30.16b, v16.16b +.long 0xce237a10 //bcax v16.16b,v16.16b,v3.16b,v30.16b // A[0][3]=A[3][3] + +.long 0xce2c7f2a //bcax v10.16b,v25.16b, v12.16b,v31.16b +.long 0xce2d33eb //bcax v11.16b,v31.16b, v13.16b,v12.16b +.long 0xce2e358c //bcax v12.16b,v12.16b,v14.16b,v13.16b +.long 0xce3939ad //bcax v13.16b,v13.16b,v25.16b, v14.16b +.long 0xce3f65ce //bcax v14.16b,v14.16b,v31.16b, v25.16b + +.long 0xce2913a7 //bcax v7.16b,v29.16b, v9.16b,v4.16b // A[0][4]=A[1][3] +.long 0xce252488 //bcax v8.16b,v4.16b,v5.16b,v9.16b // A[0][4]=A[1][3] +.long 0xce261529 //bcax v9.16b,v9.16b,v6.16b,v5.16b +.long 0xce3d18a5 //bcax v5.16b,v5.16b,v29.16b, v6.16b +.long 0xce2474c6 //bcax v6.16b,v6.16b,v4.16b,v29.16b // A[0][4]=A[1][3] + +.long 0xce207363 //bcax v3.16b,v27.16b, v0.16b,v28.16b +.long 0xce210384 //bcax v4.16b,v28.16b, v1.16b,v0.16b +.long 0xce220400 //bcax v0.16b,v0.16b,v2.16b,v1.16b +.long 0xce3b0821 //bcax v1.16b,v1.16b,v27.16b, v2.16b +.long 0xce3c6c42 //bcax v2.16b,v2.16b,v28.16b, v27.16b + + eor v0.16b,v0.16b,v26.16b + + tst x10,#255 + bne Loop_ce + + ret + + + +.align 5 +KeccakF1600_cext: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]! + add x29,sp,#0 + stp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // per ABI requirement + stp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldp d0,d1,[x0,#8*0] + ldp d2,d3,[x0,#8*2] + ldp d4,d5,[x0,#8*4] + ldp d6,d7,[x0,#8*6] + ldp d8,d9,[x0,#8*8] + ldp d10,d11,[x0,#8*10] + ldp d12,d13,[x0,#8*12] + ldp d14,d15,[x0,#8*14] + ldp d16,d17,[x0,#8*16] + ldp d18,d19,[x0,#8*18] + ldp d20,d21,[x0,#8*20] + ldp d22,d23,[x0,#8*22] + ldr d24,[x0,#8*24] + adr x10,iotas + bl KeccakF1600_ce + ldr x30,[sp,#__SIZEOF_POINTER__] + stp d0,d1,[x0,#8*0] + stp d2,d3,[x0,#8*2] + stp d4,d5,[x0,#8*4] + stp d6,d7,[x0,#8*6] + stp d8,d9,[x0,#8*8] + stp d10,d11,[x0,#8*10] + stp d12,d13,[x0,#8*12] + stp d14,d15,[x0,#8*14] + stp d16,d17,[x0,#8*16] + stp d18,d19,[x0,#8*18] + stp d20,d21,[x0,#8*20] + stp d22,d23,[x0,#8*22] + str d24,[x0,#8*24] + + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldr x29,[sp],#2*__SIZEOF_POINTER__+64 +.long 0xd50323bf // autiasp + ret + +.globl _crypton_keccak_asm_absorb_cext + +.align 5 +_crypton_keccak_asm_absorb_cext: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]! + add x29,sp,#0 + stp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // per ABI requirement + stp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldp d0,d1,[x0,#8*0] + ldp d2,d3,[x0,#8*2] + ldp d4,d5,[x0,#8*4] + ldp d6,d7,[x0,#8*6] + ldp d8,d9,[x0,#8*8] + ldp d10,d11,[x0,#8*10] + ldp d12,d13,[x0,#8*12] + ldp d14,d15,[x0,#8*14] + ldp d16,d17,[x0,#8*16] + ldp d18,d19,[x0,#8*18] + ldp d20,d21,[x0,#8*20] + ldp d22,d23,[x0,#8*22] + ldr d24,[x0,#8*24] + b Loop_absorb_ce + +.align 4 +Loop_absorb_ce: + subs x2,x2,x3 // len - bsz + blo Labsorbed_ce + + cmp x3,#104 + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v0.16b,v0.16b,v27.16b + eor v1.16b,v1.16b,v28.16b + eor v2.16b,v2.16b,v29.16b + eor v3.16b,v3.16b,v30.16b + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v4.16b,v4.16b,v27.16b + eor v5.16b,v5.16b,v28.16b + eor v6.16b,v6.16b,v29.16b + eor v7.16b,v7.16b,v30.16b + ld1 {v31.8b},[x1],#8 // A[1][4] ^= *inp++ + eor v8.16b,v8.16b,v31.16b + blo Lprocess_block_ce + + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v9.16b,v9.16b,v27.16b + eor v10.16b,v10.16b,v28.16b + eor v11.16b,v11.16b,v29.16b + eor v12.16b,v12.16b,v30.16b + beq Lprocess_block_ce + + cmp x3,#144 + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v13.16b,v13.16b,v27.16b + eor v14.16b,v14.16b,v28.16b + eor v15.16b,v15.16b,v29.16b + eor v16.16b,v16.16b,v30.16b + blo Lprocess_block_ce + + ld1 {v31.8b},[x1],#8 // A[3][3] ^= *inp++ + eor v17.16b,v17.16b,v31.16b + beq Lprocess_block_ce + + ld1 {v28.8b,v29.8b,v30.8b},[x1],#24 + eor v18.16b,v18.16b,v28.16b + eor v19.16b,v19.16b,v29.16b + eor v20.16b,v20.16b,v30.16b + +Lprocess_block_ce: + adr x10,iotas + bl KeccakF1600_ce + + b Loop_absorb_ce + +.align 4 +Labsorbed_ce: + stp d0,d1,[x0,#8*0] + stp d2,d3,[x0,#8*2] + stp d4,d5,[x0,#8*4] + stp d6,d7,[x0,#8*6] + stp d8,d9,[x0,#8*8] + stp d10,d11,[x0,#8*10] + stp d12,d13,[x0,#8*12] + stp d14,d15,[x0,#8*14] + stp d16,d17,[x0,#8*16] + stp d18,d19,[x0,#8*18] + stp d20,d21,[x0,#8*20] + stp d22,d23,[x0,#8*22] + str d24,[x0,#8*24] + add x0,x2,x3 // return value + + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldp x29,x30,[sp],#2*__SIZEOF_POINTER__+64 +.long 0xd50323bf // autiasp + ret + +.globl _crypton_keccak_asm_squeeze_cext + +.align 5 +_crypton_keccak_asm_squeeze_cext: +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__]! + add x29,sp,#0 + mov x9,x0 + mov x10,x3 + +Loop_squeeze_ce: + ldr x4,[x9],#8 + cmp x2,#8 + blo Lsqueeze_tail_ce +#ifdef __AARCH64EB__ + rev x4,x4 +#endif + str x4,[x1],#8 + beq Lsqueeze_done_ce + + sub x2,x2,#8 + subs x10,x10,#8 + bhi Loop_squeeze_ce + + bl KeccakF1600_cext + ldr x30,[sp,#__SIZEOF_POINTER__] + mov x9,x0 + mov x10,x3 + b Loop_squeeze_ce + +.align 4 +Lsqueeze_tail_ce: + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq Lsqueeze_done_ce + strb w4,[x1],#1 + +Lsqueeze_done_ce: + ldr x29,[sp],#2*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + +.byte 75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 diff --git a/cbits/asm/keccak1600-armv8-linux64.S b/cbits/asm/keccak1600-armv8-linux64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-armv8-linux64.S @@ -0,0 +1,843 @@ +.text + +.align 8 // strategic alignment and padding that allows to use + // address value as loop termination condition... +.quad 0,0,0,0,0,0,0,0 +.type iotas,%object +iotas: +.quad 0x0000000000000001 +.quad 0x0000000000008082 +.quad 0x800000000000808a +.quad 0x8000000080008000 +.quad 0x000000000000808b +.quad 0x0000000080000001 +.quad 0x8000000080008081 +.quad 0x8000000000008009 +.quad 0x000000000000008a +.quad 0x0000000000000088 +.quad 0x0000000080008009 +.quad 0x000000008000000a +.Liotas12: +.quad 0x000000008000808b +.quad 0x800000000000008b +.quad 0x8000000000008089 +.quad 0x8000000000008003 +.quad 0x8000000000008002 +.quad 0x8000000000000080 +.quad 0x000000000000800a +.quad 0x800000008000000a +.quad 0x8000000080008081 +.quad 0x8000000000008080 +.quad 0x0000000080000001 +.quad 0x8000000080008008 +.size iotas,.-iotas +.type KeccakF1600_int,%function +.align 5 +KeccakF1600_int: +.inst 0xd503233f // paciasp + stp x28,x30,[sp,#16] // stack is pre-allocated + b .Loop +.align 4 +.Loop: + ////////////////////////////////////////// Theta + eor x26,x0,x5 + stp x4,x9,[sp,#0] // offload pair... + eor x27,x1,x6 + eor x28,x2,x7 + eor x30,x3,x8 + eor x4,x4,x9 + eor x26,x26,x10 + eor x27,x27,x11 + eor x28,x28,x12 + eor x30,x30,x13 + eor x4,x4,x14 + eor x26,x26,x15 + eor x27,x27,x16 + eor x28,x28,x17 + eor x30,x30,x25 + eor x4,x4,x19 + eor x26,x26,x20 + eor x28,x28,x22 + eor x27,x27,x21 + eor x30,x30,x23 + eor x4,x4,x24 + + eor x9,x26,x28,ror#63 + + eor x1,x1,x9 + eor x6,x6,x9 + eor x11,x11,x9 + eor x16,x16,x9 + eor x21,x21,x9 + + eor x9,x27,x30,ror#63 + eor x28,x28,x4,ror#63 + eor x30,x30,x26,ror#63 + eor x4,x4,x27,ror#63 + + eor x27, x2,x9 // mov x27,x2 + eor x7,x7,x9 + eor x12,x12,x9 + eor x17,x17,x9 + eor x22,x22,x9 + + eor x0,x0,x4 + eor x5,x5,x4 + eor x10,x10,x4 + eor x15,x15,x4 + eor x20,x20,x4 + ldp x4,x9,[sp,#0] // re-load offloaded data + eor x26, x3,x28 // mov x26,x3 + eor x8,x8,x28 + eor x13,x13,x28 + eor x25,x25,x28 + eor x23,x23,x28 + + eor x28, x4,x30 // mov x28,x4 + eor x9,x9,x30 + eor x14,x14,x30 + eor x19,x19,x30 + eor x24,x24,x30 + + ////////////////////////////////////////// Rho+Pi + mov x30,x1 + ror x1,x6,#64-44 + //mov x27,x2 + ror x2,x12,#64-43 + //mov x26,x3 + ror x3,x25,#64-21 // ? + //mov x28,x4 + ror x4,x24,#64-14 // ? + + ror x6,x9,#64-20 // ? + ror x12,x13,#64-25 // ? + ror x25,x17,#64-15 + ror x24,x21,#64-2 // ? + + ror x9,x22,#64-61 + ror x13,x19,#64-8 + ror x17,x11,#64-10 + ror x21,x8,#64-55 + + ror x22,x14,#64-39 + ror x19,x23,#64-56 + ror x11,x7,#64-6 // ? + ror x8,x16,#64-45 + + ror x14,x20,#64-18 + ror x23,x15,#64-41 + ror x7,x10,#64-3 + ror x16,x5,#64-36 // ? + + ror x5,x26,#64-28 // ? + ror x10,x30,#64-1 + ror x15,x28,#64-27 // ? + ror x20,x27,#64-62 // ? + + ////////////////////////////////////////// Chi+Iota + bic x26,x2,x1 + bic x27,x3,x2 + bic x28,x0,x4 + bic x30,x1,x0 + eor x0,x0,x26 + bic x26,x4,x3 + eor x1,x1,x27 + ldr x27,[sp,#16] + eor x3,x3,x28 + eor x4,x4,x30 + eor x2,x2,x26 + ldr x30,[x27],#8 // Iota[i++] + + bic x26,x7,x6 + tst x27,#255 // are we done? + str x27,[sp,#16] + bic x27,x8,x7 + bic x28,x5,x9 + eor x0,x0,x30 // A[0][0] ^= Iota + bic x30,x6,x5 + eor x5,x5,x26 + bic x26,x9,x8 + eor x6,x6,x27 + eor x8,x8,x28 + eor x9,x9,x30 + eor x7,x7,x26 + + bic x26,x12,x11 + bic x27,x13,x12 + bic x28,x10,x14 + bic x30,x11,x10 + eor x10,x10,x26 + bic x26,x14,x13 + eor x11,x11,x27 + eor x13,x13,x28 + eor x14,x14,x30 + eor x12,x12,x26 + + bic x26,x17,x16 + bic x27,x25,x17 + bic x28,x15,x19 + bic x30,x16,x15 + eor x15,x15,x26 + bic x26,x19,x25 + eor x16,x16,x27 + eor x25,x25,x28 + eor x19,x19,x30 + eor x17,x17,x26 + + bic x26,x22,x21 + bic x27,x23,x22 + bic x28,x20,x24 + bic x30,x21,x20 + eor x20,x20,x26 + bic x26,x24,x23 + eor x21,x21,x27 + eor x23,x23,x28 + eor x24,x24,x30 + eor x22,x22,x26 + + bne .Loop + + ldr x30,[sp,#16+__SIZEOF_POINTER__] +.inst 0xd50323bf // autiasp + ret +.size KeccakF1600_int,.-KeccakF1600_int + +.type KeccakF1600,%function +.align 5 +KeccakF1600: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-16*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#16+4*__SIZEOF_POINTER__ + + str x0,[sp,#16+2*__SIZEOF_POINTER__] // offload argument + mov x26,x0 + ldp x0,x1,[x0,#16*0] + ldp x2,x3,[x26,#16*1] + ldp x4,x5,[x26,#16*2] + ldp x6,x7,[x26,#16*3] + ldp x8,x9,[x26,#16*4] + ldp x10,x11,[x26,#16*5] + ldp x12,x13,[x26,#16*6] + ldp x14,x15,[x26,#16*7] + ldp x16,x17,[x26,#16*8] + ldp x25,x19,[x26,#16*9] + ldp x20,x21,[x26,#16*10] + ldp x22,x23,[x26,#16*11] + ldr x24,[x26,#16*12] + + adr x28,iotas + bl KeccakF1600_int + + ldr x26,[sp,#16+2*__SIZEOF_POINTER__] + stp x0,x1,[x26,#16*0] + stp x2,x3,[x26,#16*1] + stp x4,x5,[x26,#16*2] + stp x6,x7,[x26,#16*3] + stp x8,x9,[x26,#16*4] + stp x10,x11,[x26,#16*5] + stp x12,x13,[x26,#16*6] + stp x14,x15,[x26,#16*7] + stp x16,x17,[x26,#16*8] + stp x25,x19,[x26,#16*9] + stp x20,x21,[x26,#16*10] + stp x22,x23,[x26,#16*11] + str x24,[x26,#16*12] + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#16+4*__SIZEOF_POINTER__ + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#16*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size KeccakF1600,.-KeccakF1600 + +.globl crypton_keccak_asm_absorb +.type crypton_keccak_asm_absorb,%function +.align 5 +crypton_keccak_asm_absorb: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-16*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#16+4*__SIZEOF_POINTER__+16 + + stp x0,x1,[sp,#16+2*__SIZEOF_POINTER__] // offload arguments + stp x2,x3,[sp,#16+4*__SIZEOF_POINTER__] + + mov x26,x0 // uint64_t A[5][5] + mov x27,x1 // const void *inp + mov x28,x2 // size_t len + mov x30,x3 // size_t bsz + ldp x0,x1,[x26,#16*0] + ldp x2,x3,[x26,#16*1] + ldp x4,x5,[x26,#16*2] + ldp x6,x7,[x26,#16*3] + ldp x8,x9,[x26,#16*4] + ldp x10,x11,[x26,#16*5] + ldp x12,x13,[x26,#16*6] + ldp x14,x15,[x26,#16*7] + ldp x16,x17,[x26,#16*8] + ldp x25,x19,[x26,#16*9] + ldp x20,x21,[x26,#16*10] + ldp x22,x23,[x26,#16*11] + ldr x24,[x26,#16*12] + b .Loop_absorb + +.align 4 +.Loop_absorb: + subs x26,x28,x30 // len - bsz + blo .Labsorbed + + str x26,[sp,#16+4*__SIZEOF_POINTER__] // save len - bsz + cmp x30,#104 + ldr x26,[x27,#0] // A[0][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x0,x0,x26 + ldr x26,[x27,#8] // A[0][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x1,x1,x26 + ldr x26,[x27,#16] // A[0][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x2,x2,x26 + ldr x26,[x27,#24] // A[0][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x3,x3,x26 + ldr x26,[x27,#32] // A[0][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x4,x4,x26 + ldr x26,[x27,#40] // A[1][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x5,x5,x26 + ldr x26,[x27,#48] // A[1][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x6,x6,x26 + ldr x26,[x27,#56] // A[1][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x7,x7,x26 + ldr x26,[x27,#64] // A[1][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x8,x8,x26 + blo .Lprocess_block + + ldr x26,[x27,#72] // A[1][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x9,x9,x26 + ldr x26,[x27,#80] // A[2][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x10,x10,x26 + ldr x26,[x27,#88] // A[2][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x11,x11,x26 + ldr x26,[x27,#96] // A[2][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x12,x12,x26 + beq .Lprocess_block + + cmp x30,#144 + ldr x26,[x27,#104] // A[2][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x13,x13,x26 + ldr x26,[x27,#112] // A[2][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x14,x14,x26 + ldr x26,[x27,#120] // A[3][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x15,x15,x26 + ldr x26,[x27,#128] // A[3][1] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x16,x16,x26 + blo .Lprocess_block + + ldr x26,[x27,#136] // A[3][2] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x17,x17,x26 + beq .Lprocess_block + + ldr x26,[x27,#144] // A[3][3] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x25,x25,x26 + ldr x26,[x27,#152] // A[3][4] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x19,x19,x26 + ldr x26,[x27,#160] // A[4][0] ^= *inp++ +#ifdef __AARCH64EB__ + rev x26,x26 +#endif + eor x20,x20,x26 + +.Lprocess_block: + add x27,x27,x30 + str x27,[sp,#16+3*__SIZEOF_POINTER__] // save inp + + adr x28,iotas + bl KeccakF1600_int + + ldr x27,[sp,#16+3*__SIZEOF_POINTER__] // restore arguments + ldp x28,x30,[sp,#16+4*__SIZEOF_POINTER__] + b .Loop_absorb + +.align 4 +.Labsorbed: + ldr x27,[sp,#16+2*__SIZEOF_POINTER__] + stp x0,x1,[x27,#16*0] + stp x2,x3,[x27,#16*1] + stp x4,x5,[x27,#16*2] + stp x6,x7,[x27,#16*3] + stp x8,x9,[x27,#16*4] + stp x10,x11,[x27,#16*5] + stp x12,x13,[x27,#16*6] + stp x14,x15,[x27,#16*7] + stp x16,x17,[x27,#16*8] + stp x25,x19,[x27,#16*9] + stp x20,x21,[x27,#16*10] + stp x22,x23,[x27,#16*11] + str x24,[x27,#16*12] + + mov x0,x28 // return value + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#16+4*__SIZEOF_POINTER__+16 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#16*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size crypton_keccak_asm_absorb,.-crypton_keccak_asm_absorb +.globl crypton_keccak_asm_squeeze +.type crypton_keccak_asm_squeeze,%function +.align 5 +crypton_keccak_asm_squeeze: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-6*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + + mov x19,x0 // put aside arguments + mov x20,x1 + mov x21,x2 + mov x22,x3 + +.Loop_squeeze: + ldr x4,[x0],#8 + cmp x21,#8 + blo .Lsqueeze_tail +#ifdef __AARCH64EB__ + rev x4,x4 +#endif + str x4,[x20],#8 + subs x21,x21,#8 + beq .Lsqueeze_done + + subs x3,x3,#8 + bhi .Loop_squeeze + + mov x0,x19 + bl KeccakF1600 + mov x0,x19 + mov x3,x22 + b .Loop_squeeze + +.align 4 +.Lsqueeze_tail: + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq .Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq .Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq .Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq .Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq .Lsqueeze_done + strb w4,[x20],#1 + lsr x4,x4,#8 + subs x21,x21,#1 + beq .Lsqueeze_done + strb w4,[x20],#1 + +.Lsqueeze_done: + ldp x19,x20,[sp,#2*__SIZEOF_POINTER__] + ldp x21,x22,[sp,#4*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#6*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size crypton_keccak_asm_squeeze,.-crypton_keccak_asm_squeeze +.type KeccakF1600_ce,%function +.align 5 +KeccakF1600_ce: +.Loop_ce: + ////////////////////////////////////////////////// Theta +.inst 0xce0f2a99 //eor3 v25.16b,v20.16b,v15.16b,v10.16b +.inst 0xce102eba //eor3 v26.16b,v21.16b,v16.16b,v11.16b +.inst 0xce1132db //eor3 v27.16b,v22.16b,v17.16b,v12.16b +.inst 0xce1236fc //eor3 v28.16b,v23.16b,v18.16b,v13.16b +.inst 0xce133b1d //eor3 v29.16b,v24.16b,v19.16b,v14.16b +.inst 0xce050339 //eor3 v25.16b,v25.16b, v5.16b,v0.16b +.inst 0xce06075a //eor3 v26.16b,v26.16b, v6.16b,v1.16b +.inst 0xce070b7b //eor3 v27.16b,v27.16b, v7.16b,v2.16b +.inst 0xce080f9c //eor3 v28.16b,v28.16b, v8.16b,v3.16b +.inst 0xce0913bd //eor3 v29.16b,v29.16b, v9.16b,v4.16b + +.inst 0xce7b8f3e //rax1 v30.2d,v25.2d,v27.2d // D[1] +.inst 0xce7c8f5f //rax1 v31.2d,v26.2d,v28.2d // D[2] +.inst 0xce7d8f7b //rax1 v27.2d,v27.2d,v29.2d // D[3] +.inst 0xce798f9c //rax1 v28.2d,v28.2d,v25.2d // D[4] +.inst 0xce7a8fbd //rax1 v29.2d,v29.2d,v26.2d // D[0] + + ////////////////////////////////////////////////// Theta+Rho+Pi +.inst 0xce9efc39 //xar v25.2d, v1.2d,v30.2d,#64-1 // C[0]=A[2][0] + +.inst 0xce9e50c1 //xar v1.2d,v6.2d,v30.2d,#64-44 +.inst 0xce9cb126 //xar v6.2d,v9.2d,v28.2d,#64-20 +.inst 0xce9f0ec9 //xar v9.2d,v22.2d,v31.2d,#64-61 +.inst 0xce9c65d6 //xar v22.2d,v14.2d,v28.2d,#64-39 +.inst 0xce9dba8e //xar v14.2d,v20.2d,v29.2d,#64-18 + +.inst 0xce9f085a //xar v26.2d, v2.2d,v31.2d,#64-62 // C[1]=A[4][0] + +.inst 0xce9f5582 //xar v2.2d,v12.2d,v31.2d,#64-43 +.inst 0xce9b9dac //xar v12.2d,v13.2d,v27.2d,#64-25 +.inst 0xce9ce26d //xar v13.2d,v19.2d,v28.2d,#64-8 +.inst 0xce9b22f3 //xar v19.2d,v23.2d,v27.2d,#64-56 +.inst 0xce9d5df7 //xar v23.2d,v15.2d,v29.2d,#64-41 + +.inst 0xce9c948f //xar v15.2d,v4.2d,v28.2d,#64-27 + +.inst 0xce9ccb1c //xar v28.2d, v24.2d,v28.2d,#64-14 // D[4]=A[0][4] +.inst 0xce9efab8 //xar v24.2d,v21.2d,v30.2d,#64-2 +.inst 0xce9b2508 //xar v8.2d,v8.2d,v27.2d,#64-55 // A[1][3]=A[4][1] +.inst 0xce9e4e04 //xar v4.2d,v16.2d,v30.2d,#64-45 // A[0][4]=A[1][3] +.inst 0xce9d70b0 //xar v16.2d,v5.2d,v29.2d,#64-36 + +.inst 0xce9b9065 //xar v5.2d,v3.2d,v27.2d,#64-28 + + eor v0.16b,v0.16b,v29.16b + +.inst 0xce9bae5b //xar v27.2d, v18.2d,v27.2d,#64-21 // D[3]=A[0][3] +.inst 0xce9fc623 //xar v3.2d,v17.2d,v31.2d,#64-15 // A[0][3]=A[3][3] +.inst 0xce9ed97e //xar v30.2d, v11.2d,v30.2d,#64-10 // D[1]=A[3][2] +.inst 0xce9fe8ff //xar v31.2d, v7.2d,v31.2d,#64-6 // D[2]=A[2][1] +.inst 0xce9df55d //xar v29.2d, v10.2d,v29.2d,#64-3 // D[0]=A[1][2] + + ////////////////////////////////////////////////// Chi+Iota +.inst 0xce362354 //bcax v20.16b,v26.16b, v22.16b,v8.16b // A[1][3]=A[4][1] +.inst 0xce375915 //bcax v21.16b,v8.16b,v23.16b,v22.16b // A[1][3]=A[4][1] +.inst 0xce385ed6 //bcax v22.16b,v22.16b,v24.16b,v23.16b +.inst 0xce3a62f7 //bcax v23.16b,v23.16b,v26.16b, v24.16b +.inst 0xce286b18 //bcax v24.16b,v24.16b,v8.16b,v26.16b // A[1][3]=A[4][1] + + ld1r {v26.2d},[x10],#8 + +.inst 0xce330fd1 //bcax v17.16b,v30.16b, v19.16b,v3.16b // A[0][3]=A[3][3] +.inst 0xce2f4c72 //bcax v18.16b,v3.16b,v15.16b,v19.16b // A[0][3]=A[3][3] +.inst 0xce303e73 //bcax v19.16b,v19.16b,v16.16b,v15.16b +.inst 0xce3e41ef //bcax v15.16b,v15.16b,v30.16b, v16.16b +.inst 0xce237a10 //bcax v16.16b,v16.16b,v3.16b,v30.16b // A[0][3]=A[3][3] + +.inst 0xce2c7f2a //bcax v10.16b,v25.16b, v12.16b,v31.16b +.inst 0xce2d33eb //bcax v11.16b,v31.16b, v13.16b,v12.16b +.inst 0xce2e358c //bcax v12.16b,v12.16b,v14.16b,v13.16b +.inst 0xce3939ad //bcax v13.16b,v13.16b,v25.16b, v14.16b +.inst 0xce3f65ce //bcax v14.16b,v14.16b,v31.16b, v25.16b + +.inst 0xce2913a7 //bcax v7.16b,v29.16b, v9.16b,v4.16b // A[0][4]=A[1][3] +.inst 0xce252488 //bcax v8.16b,v4.16b,v5.16b,v9.16b // A[0][4]=A[1][3] +.inst 0xce261529 //bcax v9.16b,v9.16b,v6.16b,v5.16b +.inst 0xce3d18a5 //bcax v5.16b,v5.16b,v29.16b, v6.16b +.inst 0xce2474c6 //bcax v6.16b,v6.16b,v4.16b,v29.16b // A[0][4]=A[1][3] + +.inst 0xce207363 //bcax v3.16b,v27.16b, v0.16b,v28.16b +.inst 0xce210384 //bcax v4.16b,v28.16b, v1.16b,v0.16b +.inst 0xce220400 //bcax v0.16b,v0.16b,v2.16b,v1.16b +.inst 0xce3b0821 //bcax v1.16b,v1.16b,v27.16b, v2.16b +.inst 0xce3c6c42 //bcax v2.16b,v2.16b,v28.16b, v27.16b + + eor v0.16b,v0.16b,v26.16b + + tst x10,#255 + bne .Loop_ce + + ret +.size KeccakF1600_ce,.-KeccakF1600_ce + +.type KeccakF1600_cext,%function +.align 5 +KeccakF1600_cext: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]! + add x29,sp,#0 + stp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // per ABI requirement + stp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldp d0,d1,[x0,#8*0] + ldp d2,d3,[x0,#8*2] + ldp d4,d5,[x0,#8*4] + ldp d6,d7,[x0,#8*6] + ldp d8,d9,[x0,#8*8] + ldp d10,d11,[x0,#8*10] + ldp d12,d13,[x0,#8*12] + ldp d14,d15,[x0,#8*14] + ldp d16,d17,[x0,#8*16] + ldp d18,d19,[x0,#8*18] + ldp d20,d21,[x0,#8*20] + ldp d22,d23,[x0,#8*22] + ldr d24,[x0,#8*24] + adr x10,iotas + bl KeccakF1600_ce + ldr x30,[sp,#__SIZEOF_POINTER__] + stp d0,d1,[x0,#8*0] + stp d2,d3,[x0,#8*2] + stp d4,d5,[x0,#8*4] + stp d6,d7,[x0,#8*6] + stp d8,d9,[x0,#8*8] + stp d10,d11,[x0,#8*10] + stp d12,d13,[x0,#8*12] + stp d14,d15,[x0,#8*14] + stp d16,d17,[x0,#8*16] + stp d18,d19,[x0,#8*18] + stp d20,d21,[x0,#8*20] + stp d22,d23,[x0,#8*22] + str d24,[x0,#8*24] + + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldr x29,[sp],#2*__SIZEOF_POINTER__+64 +.inst 0xd50323bf // autiasp + ret +.size KeccakF1600_cext,.-KeccakF1600_cext +.globl crypton_keccak_asm_absorb_cext +.type crypton_keccak_asm_absorb_cext,%function +.align 5 +crypton_keccak_asm_absorb_cext: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]! + add x29,sp,#0 + stp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // per ABI requirement + stp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldp d0,d1,[x0,#8*0] + ldp d2,d3,[x0,#8*2] + ldp d4,d5,[x0,#8*4] + ldp d6,d7,[x0,#8*6] + ldp d8,d9,[x0,#8*8] + ldp d10,d11,[x0,#8*10] + ldp d12,d13,[x0,#8*12] + ldp d14,d15,[x0,#8*14] + ldp d16,d17,[x0,#8*16] + ldp d18,d19,[x0,#8*18] + ldp d20,d21,[x0,#8*20] + ldp d22,d23,[x0,#8*22] + ldr d24,[x0,#8*24] + b .Loop_absorb_ce + +.align 4 +.Loop_absorb_ce: + subs x2,x2,x3 // len - bsz + blo .Labsorbed_ce + + cmp x3,#104 + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v0.16b,v0.16b,v27.16b + eor v1.16b,v1.16b,v28.16b + eor v2.16b,v2.16b,v29.16b + eor v3.16b,v3.16b,v30.16b + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v4.16b,v4.16b,v27.16b + eor v5.16b,v5.16b,v28.16b + eor v6.16b,v6.16b,v29.16b + eor v7.16b,v7.16b,v30.16b + ld1 {v31.8b},[x1],#8 // A[1][4] ^= *inp++ + eor v8.16b,v8.16b,v31.16b + blo .Lprocess_block_ce + + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v9.16b,v9.16b,v27.16b + eor v10.16b,v10.16b,v28.16b + eor v11.16b,v11.16b,v29.16b + eor v12.16b,v12.16b,v30.16b + beq .Lprocess_block_ce + + cmp x3,#144 + ld1 {v27.8b,v28.8b,v29.8b,v30.8b},[x1],#32 + eor v13.16b,v13.16b,v27.16b + eor v14.16b,v14.16b,v28.16b + eor v15.16b,v15.16b,v29.16b + eor v16.16b,v16.16b,v30.16b + blo .Lprocess_block_ce + + ld1 {v31.8b},[x1],#8 // A[3][3] ^= *inp++ + eor v17.16b,v17.16b,v31.16b + beq .Lprocess_block_ce + + ld1 {v28.8b,v29.8b,v30.8b},[x1],#24 + eor v18.16b,v18.16b,v28.16b + eor v19.16b,v19.16b,v29.16b + eor v20.16b,v20.16b,v30.16b + +.Lprocess_block_ce: + adr x10,iotas + bl KeccakF1600_ce + + b .Loop_absorb_ce + +.align 4 +.Labsorbed_ce: + stp d0,d1,[x0,#8*0] + stp d2,d3,[x0,#8*2] + stp d4,d5,[x0,#8*4] + stp d6,d7,[x0,#8*6] + stp d8,d9,[x0,#8*8] + stp d10,d11,[x0,#8*10] + stp d12,d13,[x0,#8*12] + stp d14,d15,[x0,#8*14] + stp d16,d17,[x0,#8*16] + stp d18,d19,[x0,#8*18] + stp d20,d21,[x0,#8*20] + stp d22,d23,[x0,#8*22] + str d24,[x0,#8*24] + add x0,x2,x3 // return value + + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + ldp x29,x30,[sp],#2*__SIZEOF_POINTER__+64 +.inst 0xd50323bf // autiasp + ret +.size crypton_keccak_asm_absorb_cext,.-crypton_keccak_asm_absorb_cext +.globl crypton_keccak_asm_squeeze_cext +.type crypton_keccak_asm_squeeze_cext,%function +.align 5 +crypton_keccak_asm_squeeze_cext: +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__]! + add x29,sp,#0 + mov x9,x0 + mov x10,x3 + +.Loop_squeeze_ce: + ldr x4,[x9],#8 + cmp x2,#8 + blo .Lsqueeze_tail_ce +#ifdef __AARCH64EB__ + rev x4,x4 +#endif + str x4,[x1],#8 + beq .Lsqueeze_done_ce + + sub x2,x2,#8 + subs x10,x10,#8 + bhi .Loop_squeeze_ce + + bl KeccakF1600_cext + ldr x30,[sp,#__SIZEOF_POINTER__] + mov x9,x0 + mov x10,x3 + b .Loop_squeeze_ce + +.align 4 +.Lsqueeze_tail_ce: + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq .Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq .Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq .Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq .Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq .Lsqueeze_done_ce + strb w4,[x1],#1 + lsr x4,x4,#8 + subs x2,x2,#1 + beq .Lsqueeze_done_ce + strb w4,[x1],#1 + +.Lsqueeze_done_ce: + ldr x29,[sp],#2*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size crypton_keccak_asm_squeeze_cext,.-crypton_keccak_asm_squeeze_cext +.byte 75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 + +.section .note.GNU-stack,"",%progbits diff --git a/cbits/asm/keccak1600-armv8.pl b/cbits/asm/keccak1600-armv8.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-armv8.pl @@ -0,0 +1,932 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. The module is, however, dual licensed under OpenSSL and +# CRYPTOGAMS licenses depending on where you obtain it. For further +# details see http://www.openssl.org/~appro/cryptogams/. +# ==================================================================== +# +# Keccak-1600 for ARMv8. +# +# June 2017. +# +# This is straightforward KECCAK_1X_ALT implementation. It makes no +# sense to attempt SIMD/NEON implementation for following reason. +# 64-bit lanes of vector registers can't be addressed as easily as in +# 32-bit mode. This means that 64-bit NEON is bound to be slower than +# 32-bit NEON, and this implementation is faster than 32-bit NEON on +# same processor. Even though it takes more scalar xor's and andn's, +# it gets compensated by availability of rotate. Not to forget that +# most processors achieve higher issue rate with scalar instructions. +# +# February 2018. +# +# Add hardware-assisted ARMv8.2 implementation. It's KECCAK_1X_ALT +# variant with register permutation/rotation twist that allows to +# eliminate copies to temporary registers. If you look closely you'll +# notice that it uses only one lane of vector registers. The new +# instructions effectively facilitate parallel hashing, which we don't +# support [yet?]. But lowest-level core procedure is prepared for it. +# The inner round is 67 [vector] instructions, so it's not actually +# obvious that it will provide performance improvement [in serial +# hash] as long as vector instructions issue rate is limited to 1 per +# cycle... +# +###################################################################### +# Numbers are cycles per processed byte. +# +# r=1088(*) +# +# Cortex-A53 13 +# Cortex-A57 12 +# Cortex-A76 7.9 +# Cortex-X2 6.1 (***) +# Cortex-X925 3.0 (**) +# X-Gene 14 +# Mongoose 10 +# Kryo 12 +# Snapdragon X 3.8 (**) +# Denver 7.8 +# Apple A7 7.2 +# Apple A10 6.1 +# Apple A12 4.4 +# Apple A14/M1 3.5 (**) +# ThunderX2 9.7 +# +# (*) Corresponds to SHA3-256. No improvement coefficients are listed +# because they vary too much from compiler to compiler. Newer +# compiler does much better and improvement varies from 5% on +# Cortex-A57 to 25% on Cortex-A53. While in comparison to older +# compiler this code is at least 2x faster... +# (**) The result is for hardware-assisted implementation below. +# (***) Hardware-assisted code is significantly slower, 11.3, +# apparently because the processor can issue just one SHA3 +# instruction per cycle. + +$flavour = shift; +$output = shift; + +if ($flavour && $flavour ne "void") { + $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; + ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or + ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or + die "can't locate arm-xlate.pl"; + + open STDOUT,"| \"$^X\" $xlate $flavour $output"; +} else { + open STDOUT,">$output"; +} + +my @rhotates = ([ 0, 1, 62, 28, 27 ], + [ 36, 44, 6, 55, 20 ], + [ 3, 10, 43, 25, 39 ], + [ 41, 45, 15, 21, 8 ], + [ 18, 2, 61, 56, 14 ]); + +my $sha3ops = ($flavour =~ /\+sha3/); + +$code.=<<___ if ($sha3ops); +.arch armv8.2-a+sha3 +___ +$code.=<<___; +.text + +.align 8 // strategic alignment and padding that allows to use + // address value as loop termination condition... + .quad 0,0,0,0,0,0,0,0 +.type iotas,%object +iotas: + .quad 0x0000000000000001 + .quad 0x0000000000008082 + .quad 0x800000000000808a + .quad 0x8000000080008000 + .quad 0x000000000000808b + .quad 0x0000000080000001 + .quad 0x8000000080008081 + .quad 0x8000000000008009 + .quad 0x000000000000008a + .quad 0x0000000000000088 + .quad 0x0000000080008009 + .quad 0x000000008000000a +.Liotas12: + .quad 0x000000008000808b + .quad 0x800000000000008b + .quad 0x8000000000008089 + .quad 0x8000000000008003 + .quad 0x8000000000008002 + .quad 0x8000000000000080 + .quad 0x000000000000800a + .quad 0x800000008000000a + .quad 0x8000000080008081 + .quad 0x8000000000008080 + .quad 0x0000000080000001 + .quad 0x8000000080008008 +.size iotas,.-iotas +___ + {{{ +my @A = map([ "x$_", "x".($_+1), "x".($_+2), "x".($_+3), "x".($_+4) ], + (0, 5, 10, 15, 20)); + $A[3][3] = "x25"; # x18 is reserved + +my @C = map("x$_", (26,27,28,30)); + +$code.=<<___; +.type KeccakF1600_int,%function +.align 5 +KeccakF1600_int: + .inst 0xd503233f // paciasp + stp c#$C[2],c30,[csp,#16] // stack is pre-allocated + b .Loop +.align 4 +.Loop: + ////////////////////////////////////////// Theta + eor $C[0],$A[0][0],$A[1][0] + stp $A[0][4],$A[1][4],[sp,#0] // offload pair... + eor $C[1],$A[0][1],$A[1][1] + eor $C[2],$A[0][2],$A[1][2] + eor $C[3],$A[0][3],$A[1][3] +___ + $C[4]=$A[0][4]; + $C[5]=$A[1][4]; +$code.=<<___; + eor $C[4],$A[0][4],$A[1][4] + eor $C[0],$C[0],$A[2][0] + eor $C[1],$C[1],$A[2][1] + eor $C[2],$C[2],$A[2][2] + eor $C[3],$C[3],$A[2][3] + eor $C[4],$C[4],$A[2][4] + eor $C[0],$C[0],$A[3][0] + eor $C[1],$C[1],$A[3][1] + eor $C[2],$C[2],$A[3][2] + eor $C[3],$C[3],$A[3][3] + eor $C[4],$C[4],$A[3][4] + eor $C[0],$C[0],$A[4][0] + eor $C[2],$C[2],$A[4][2] + eor $C[1],$C[1],$A[4][1] + eor $C[3],$C[3],$A[4][3] + eor $C[4],$C[4],$A[4][4] + + eor $C[5],$C[0],$C[2],ror#63 + + eor $A[0][1],$A[0][1],$C[5] + eor $A[1][1],$A[1][1],$C[5] + eor $A[2][1],$A[2][1],$C[5] + eor $A[3][1],$A[3][1],$C[5] + eor $A[4][1],$A[4][1],$C[5] + + eor $C[5],$C[1],$C[3],ror#63 + eor $C[2],$C[2],$C[4],ror#63 + eor $C[3],$C[3],$C[0],ror#63 + eor $C[4],$C[4],$C[1],ror#63 + + eor $C[1], $A[0][2],$C[5] // mov $C[1],$A[0][2] + eor $A[1][2],$A[1][2],$C[5] + eor $A[2][2],$A[2][2],$C[5] + eor $A[3][2],$A[3][2],$C[5] + eor $A[4][2],$A[4][2],$C[5] + + eor $A[0][0],$A[0][0],$C[4] + eor $A[1][0],$A[1][0],$C[4] + eor $A[2][0],$A[2][0],$C[4] + eor $A[3][0],$A[3][0],$C[4] + eor $A[4][0],$A[4][0],$C[4] +___ + $C[4]=undef; + $C[5]=undef; +$code.=<<___; + ldp $A[0][4],$A[1][4],[sp,#0] // re-load offloaded data + eor $C[0], $A[0][3],$C[2] // mov $C[0],$A[0][3] + eor $A[1][3],$A[1][3],$C[2] + eor $A[2][3],$A[2][3],$C[2] + eor $A[3][3],$A[3][3],$C[2] + eor $A[4][3],$A[4][3],$C[2] + + eor $C[2], $A[0][4],$C[3] // mov $C[2],$A[0][4] + eor $A[1][4],$A[1][4],$C[3] + eor $A[2][4],$A[2][4],$C[3] + eor $A[3][4],$A[3][4],$C[3] + eor $A[4][4],$A[4][4],$C[3] + + ////////////////////////////////////////// Rho+Pi + mov $C[3],$A[0][1] + ror $A[0][1],$A[1][1],#64-$rhotates[1][1] + //mov $C[1],$A[0][2] + ror $A[0][2],$A[2][2],#64-$rhotates[2][2] + //mov $C[0],$A[0][3] + ror $A[0][3],$A[3][3],#64-$rhotates[3][3] // ? + //mov $C[2],$A[0][4] + ror $A[0][4],$A[4][4],#64-$rhotates[4][4] // ? + + ror $A[1][1],$A[1][4],#64-$rhotates[1][4] // ? + ror $A[2][2],$A[2][3],#64-$rhotates[2][3] // ? + ror $A[3][3],$A[3][2],#64-$rhotates[3][2] + ror $A[4][4],$A[4][1],#64-$rhotates[4][1] // ? + + ror $A[1][4],$A[4][2],#64-$rhotates[4][2] + ror $A[2][3],$A[3][4],#64-$rhotates[3][4] + ror $A[3][2],$A[2][1],#64-$rhotates[2][1] + ror $A[4][1],$A[1][3],#64-$rhotates[1][3] + + ror $A[4][2],$A[2][4],#64-$rhotates[2][4] + ror $A[3][4],$A[4][3],#64-$rhotates[4][3] + ror $A[2][1],$A[1][2],#64-$rhotates[1][2] // ? + ror $A[1][3],$A[3][1],#64-$rhotates[3][1] + + ror $A[2][4],$A[4][0],#64-$rhotates[4][0] + ror $A[4][3],$A[3][0],#64-$rhotates[3][0] + ror $A[1][2],$A[2][0],#64-$rhotates[2][0] + ror $A[3][1],$A[1][0],#64-$rhotates[1][0] // ? + + ror $A[1][0],$C[0],#64-$rhotates[0][3] // ? + ror $A[2][0],$C[3],#64-$rhotates[0][1] + ror $A[3][0],$C[2],#64-$rhotates[0][4] // ? + ror $A[4][0],$C[1],#64-$rhotates[0][2] // ? + + ////////////////////////////////////////// Chi+Iota + bic $C[0],$A[0][2],$A[0][1] + bic $C[1],$A[0][3],$A[0][2] + bic $C[2],$A[0][0],$A[0][4] + bic $C[3],$A[0][1],$A[0][0] + eor $A[0][0],$A[0][0],$C[0] + bic $C[0],$A[0][4],$A[0][3] + eor $A[0][1],$A[0][1],$C[1] + ldr c#$C[1],[csp,#16] + eor $A[0][3],$A[0][3],$C[2] + eor $A[0][4],$A[0][4],$C[3] + eor $A[0][2],$A[0][2],$C[0] + ldr $C[3],[$C[1]],#8 // Iota[i++] + + bic $C[0],$A[1][2],$A[1][1] + tst $C[1],#255 // are we done? + str c#$C[1],[csp,#16] + bic $C[1],$A[1][3],$A[1][2] + bic $C[2],$A[1][0],$A[1][4] + eor $A[0][0],$A[0][0],$C[3] // A[0][0] ^= Iota + bic $C[3],$A[1][1],$A[1][0] + eor $A[1][0],$A[1][0],$C[0] + bic $C[0],$A[1][4],$A[1][3] + eor $A[1][1],$A[1][1],$C[1] + eor $A[1][3],$A[1][3],$C[2] + eor $A[1][4],$A[1][4],$C[3] + eor $A[1][2],$A[1][2],$C[0] + + bic $C[0],$A[2][2],$A[2][1] + bic $C[1],$A[2][3],$A[2][2] + bic $C[2],$A[2][0],$A[2][4] + bic $C[3],$A[2][1],$A[2][0] + eor $A[2][0],$A[2][0],$C[0] + bic $C[0],$A[2][4],$A[2][3] + eor $A[2][1],$A[2][1],$C[1] + eor $A[2][3],$A[2][3],$C[2] + eor $A[2][4],$A[2][4],$C[3] + eor $A[2][2],$A[2][2],$C[0] + + bic $C[0],$A[3][2],$A[3][1] + bic $C[1],$A[3][3],$A[3][2] + bic $C[2],$A[3][0],$A[3][4] + bic $C[3],$A[3][1],$A[3][0] + eor $A[3][0],$A[3][0],$C[0] + bic $C[0],$A[3][4],$A[3][3] + eor $A[3][1],$A[3][1],$C[1] + eor $A[3][3],$A[3][3],$C[2] + eor $A[3][4],$A[3][4],$C[3] + eor $A[3][2],$A[3][2],$C[0] + + bic $C[0],$A[4][2],$A[4][1] + bic $C[1],$A[4][3],$A[4][2] + bic $C[2],$A[4][0],$A[4][4] + bic $C[3],$A[4][1],$A[4][0] + eor $A[4][0],$A[4][0],$C[0] + bic $C[0],$A[4][4],$A[4][3] + eor $A[4][1],$A[4][1],$C[1] + eor $A[4][3],$A[4][3],$C[2] + eor $A[4][4],$A[4][4],$C[3] + eor $A[4][2],$A[4][2],$C[0] + + bne .Loop + + ldr c30,[csp,#16+__SIZEOF_POINTER__] + .inst 0xd50323bf // autiasp + ret +.size KeccakF1600_int,.-KeccakF1600_int + +.type KeccakF1600,%function +.align 5 +KeccakF1600: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-16*__SIZEOF_POINTER__]! + add c29,csp,#0 + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + sub csp,csp,#16+4*__SIZEOF_POINTER__ + + str c0,[csp,#16+2*__SIZEOF_POINTER__] // offload argument + mov c#$C[0],c0 + ldp $A[0][0],$A[0][1],[x0,#16*0] + ldp $A[0][2],$A[0][3],[$C[0],#16*1] + ldp $A[0][4],$A[1][0],[$C[0],#16*2] + ldp $A[1][1],$A[1][2],[$C[0],#16*3] + ldp $A[1][3],$A[1][4],[$C[0],#16*4] + ldp $A[2][0],$A[2][1],[$C[0],#16*5] + ldp $A[2][2],$A[2][3],[$C[0],#16*6] + ldp $A[2][4],$A[3][0],[$C[0],#16*7] + ldp $A[3][1],$A[3][2],[$C[0],#16*8] + ldp $A[3][3],$A[3][4],[$C[0],#16*9] + ldp $A[4][0],$A[4][1],[$C[0],#16*10] + ldp $A[4][2],$A[4][3],[$C[0],#16*11] + ldr $A[4][4],[$C[0],#16*12] + + adr $C[2],iotas + bl KeccakF1600_int + + ldr c#$C[0],[csp,#16+2*__SIZEOF_POINTER__] + stp $A[0][0],$A[0][1],[$C[0],#16*0] + stp $A[0][2],$A[0][3],[$C[0],#16*1] + stp $A[0][4],$A[1][0],[$C[0],#16*2] + stp $A[1][1],$A[1][2],[$C[0],#16*3] + stp $A[1][3],$A[1][4],[$C[0],#16*4] + stp $A[2][0],$A[2][1],[$C[0],#16*5] + stp $A[2][2],$A[2][3],[$C[0],#16*6] + stp $A[2][4],$A[3][0],[$C[0],#16*7] + stp $A[3][1],$A[3][2],[$C[0],#16*8] + stp $A[3][3],$A[3][4],[$C[0],#16*9] + stp $A[4][0],$A[4][1],[$C[0],#16*10] + stp $A[4][2],$A[4][3],[$C[0],#16*11] + str $A[4][4],[$C[0],#16*12] + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#16+4*__SIZEOF_POINTER__ + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#16*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size KeccakF1600,.-KeccakF1600 + +.globl SHA3_absorb +.type SHA3_absorb,%function +.align 5 +SHA3_absorb: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-16*__SIZEOF_POINTER__]! + add c29,csp,#0 + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + sub csp,csp,#16+4*__SIZEOF_POINTER__+16 + + stp c0,c1,[csp,#16+2*__SIZEOF_POINTER__] // offload arguments + stp x2,x3,[csp,#16+4*__SIZEOF_POINTER__] + + mov c#$C[0],c0 // uint64_t A[5][5] + mov c#$C[1],c1 // const void *inp + mov $C[2],x2 // size_t len + mov $C[3],x3 // size_t bsz + ldp $A[0][0],$A[0][1],[$C[0],#16*0] + ldp $A[0][2],$A[0][3],[$C[0],#16*1] + ldp $A[0][4],$A[1][0],[$C[0],#16*2] + ldp $A[1][1],$A[1][2],[$C[0],#16*3] + ldp $A[1][3],$A[1][4],[$C[0],#16*4] + ldp $A[2][0],$A[2][1],[$C[0],#16*5] + ldp $A[2][2],$A[2][3],[$C[0],#16*6] + ldp $A[2][4],$A[3][0],[$C[0],#16*7] + ldp $A[3][1],$A[3][2],[$C[0],#16*8] + ldp $A[3][3],$A[3][4],[$C[0],#16*9] + ldp $A[4][0],$A[4][1],[$C[0],#16*10] + ldp $A[4][2],$A[4][3],[$C[0],#16*11] + ldr $A[4][4],[$C[0],#16*12] + b .Loop_absorb + +.align 4 +.Loop_absorb: + subs $C[0],$C[2],$C[3] // len - bsz + blo .Labsorbed + + str $C[0],[csp,#16+4*__SIZEOF_POINTER__] // save len - bsz + cmp $C[3],#104 +___ +sub load_n_xor { + my ($from,$to) = @_; + + for (my $i=$from; $i<=$to; $i++) { +$code.=<<___; + ldr $C[0],[$C[1],#`8*$i`] // A[`$i/5`][`$i%5`] ^= *inp++ +#ifdef __AARCH64EB__ + rev $C[0],$C[0] +#endif + eor $A[$i/5][$i%5],$A[$i/5][$i%5],$C[0] +___ + } +} +load_n_xor(0,8); +$code.=<<___; + blo .Lprocess_block + +___ +load_n_xor(9,12); +$code.=<<___; + beq .Lprocess_block + + cmp $C[3],#144 +___ +load_n_xor(13,16); +$code.=<<___; + blo .Lprocess_block + +___ +load_n_xor(17,17); +$code.=<<___; + beq .Lprocess_block + +___ +load_n_xor(18,20); +$code.=<<___; + +.Lprocess_block: + add c#$C[1],c#@C[1],@C[3] + str c#$C[1],[csp,#16+3*__SIZEOF_POINTER__] // save inp + + adr $C[2],iotas + bl KeccakF1600_int + + ldr c#$C[1],[csp,#16+3*__SIZEOF_POINTER__] // restore arguments + ldp $C[2],$C[3],[csp,#16+4*__SIZEOF_POINTER__] + b .Loop_absorb + +.align 4 +.Labsorbed: + ldr c#$C[1],[sp,#16+2*__SIZEOF_POINTER__] + stp $A[0][0],$A[0][1],[$C[1],#16*0] + stp $A[0][2],$A[0][3],[$C[1],#16*1] + stp $A[0][4],$A[1][0],[$C[1],#16*2] + stp $A[1][1],$A[1][2],[$C[1],#16*3] + stp $A[1][3],$A[1][4],[$C[1],#16*4] + stp $A[2][0],$A[2][1],[$C[1],#16*5] + stp $A[2][2],$A[2][3],[$C[1],#16*6] + stp $A[2][4],$A[3][0],[$C[1],#16*7] + stp $A[3][1],$A[3][2],[$C[1],#16*8] + stp $A[3][3],$A[3][4],[$C[1],#16*9] + stp $A[4][0],$A[4][1],[$C[1],#16*10] + stp $A[4][2],$A[4][3],[$C[1],#16*11] + str $A[4][4],[$C[1],#16*12] + + mov x0,$C[2] // return value + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#16+4*__SIZEOF_POINTER__+16 + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#16*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size SHA3_absorb,.-SHA3_absorb +___ +{ +my ($A_flat,$out,$len,$bsz) = map("x$_",(19..22)); +$code.=<<___; +.globl SHA3_squeeze +.type SHA3_squeeze,%function +.align 5 +SHA3_squeeze: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-6*__SIZEOF_POINTER__]! + add c29,csp,#0 + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + + cmov $A_flat,x0 // put aside arguments + cmov $out,x1 + mov $len,x2 + mov $bsz,x3 + +.Loop_squeeze: + ldr x4,[x0],#8 + cmp $len,#8 + blo .Lsqueeze_tail +#ifdef __AARCH64EB__ + rev x4,x4 +#endif + str x4,[$out],#8 + subs $len,$len,#8 + beq .Lsqueeze_done + + subs x3,x3,#8 + bhi .Loop_squeeze + + cmov x0,$A_flat + bl KeccakF1600 + cmov x0,$A_flat + mov x3,$bsz + b .Loop_squeeze + +.align 4 +.Lsqueeze_tail: + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done + strb w4,[$out],#1 + +.Lsqueeze_done: + ldp c19,c20,[csp,#2*__SIZEOF_POINTER__] + ldp c21,c22,[csp,#4*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#6*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size SHA3_squeeze,.-SHA3_squeeze +___ +} }}} + {{{ +my @A = map([ "v".$_.".16b", "v".($_+1).".16b", "v".($_+2).".16b", + "v".($_+3).".16b", "v".($_+4).".16b" ], + (0, 5, 10, 15, 20)); + +my @C = map("v$_.16b", (25..31)); +my @D = @C[4,5,6,2,3]; + +$code.=<<___; +.type KeccakF1600_ce,%function +.align 5 +KeccakF1600_ce: +.Loop_ce: + ////////////////////////////////////////////////// Theta + eor3 $C[0],$A[4][0],$A[3][0],$A[2][0] + eor3 $C[1],$A[4][1],$A[3][1],$A[2][1] + eor3 $C[2],$A[4][2],$A[3][2],$A[2][2] + eor3 $C[3],$A[4][3],$A[3][3],$A[2][3] + eor3 $C[4],$A[4][4],$A[3][4],$A[2][4] + eor3 $C[0],$C[0], $A[1][0],$A[0][0] + eor3 $C[1],$C[1], $A[1][1],$A[0][1] + eor3 $C[2],$C[2], $A[1][2],$A[0][2] + eor3 $C[3],$C[3], $A[1][3],$A[0][3] + eor3 $C[4],$C[4], $A[1][4],$A[0][4] + + rax1 $C[5],$C[0],$C[2] // D[1] + rax1 $C[6],$C[1],$C[3] // D[2] + rax1 $C[2],$C[2],$C[4] // D[3] + rax1 $C[3],$C[3],$C[0] // D[4] + rax1 $C[4],$C[4],$C[1] // D[0] + + ////////////////////////////////////////////////// Theta+Rho+Pi + xar $C[0], $A[0][1],$D[1],#64-$rhotates[0][1] // C[0]=A[2][0] + + xar $A[0][1],$A[1][1],$D[1],#64-$rhotates[1][1] + xar $A[1][1],$A[1][4],$D[4],#64-$rhotates[1][4] + xar $A[1][4],$A[4][2],$D[2],#64-$rhotates[4][2] + xar $A[4][2],$A[2][4],$D[4],#64-$rhotates[2][4] + xar $A[2][4],$A[4][0],$D[0],#64-$rhotates[4][0] + + xar $C[1], $A[0][2],$D[2],#64-$rhotates[0][2] // C[1]=A[4][0] + + xar $A[0][2],$A[2][2],$D[2],#64-$rhotates[2][2] + xar $A[2][2],$A[2][3],$D[3],#64-$rhotates[2][3] + xar $A[2][3],$A[3][4],$D[4],#64-$rhotates[3][4] + xar $A[3][4],$A[4][3],$D[3],#64-$rhotates[4][3] + xar $A[4][3],$A[3][0],$D[0],#64-$rhotates[3][0] + + xar $A[3][0],$A[0][4],$D[4],#64-$rhotates[0][4] + + xar $D[4], $A[4][4],$D[4],#64-$rhotates[4][4] // D[4]=A[0][4] + xar $A[4][4],$A[4][1],$D[1],#64-$rhotates[4][1] + xar $A[1][3],$A[1][3],$D[3],#64-$rhotates[1][3] // A[1][3]=A[4][1] + xar $A[0][4],$A[3][1],$D[1],#64-$rhotates[3][1] // A[0][4]=A[1][3] + xar $A[3][1],$A[1][0],$D[0],#64-$rhotates[1][0] + + xar $A[1][0],$A[0][3],$D[3],#64-$rhotates[0][3] + + eor $A[0][0],$A[0][0],$D[0] + + xar $D[3], $A[3][3],$D[3],#64-$rhotates[3][3] // D[3]=A[0][3] + xar $A[0][3],$A[3][2],$D[2],#64-$rhotates[3][2] // A[0][3]=A[3][3] + xar $D[1], $A[2][1],$D[1],#64-$rhotates[2][1] // D[1]=A[3][2] + xar $D[2], $A[1][2],$D[2],#64-$rhotates[1][2] // D[2]=A[2][1] + xar $D[0], $A[2][0],$D[0],#64-$rhotates[2][0] // D[0]=A[1][2] + + ////////////////////////////////////////////////// Chi+Iota + bcax $A[4][0],$C[1], $A[4][2],$A[1][3] // A[1][3]=A[4][1] + bcax $A[4][1],$A[1][3],$A[4][3],$A[4][2] // A[1][3]=A[4][1] + bcax $A[4][2],$A[4][2],$A[4][4],$A[4][3] + bcax $A[4][3],$A[4][3],$C[1], $A[4][4] + bcax $A[4][4],$A[4][4],$A[1][3],$C[1] // A[1][3]=A[4][1] + + ld1r {$C[1]},[x10],#8 + + bcax $A[3][2],$D[1], $A[3][4],$A[0][3] // A[0][3]=A[3][3] + bcax $A[3][3],$A[0][3],$A[3][0],$A[3][4] // A[0][3]=A[3][3] + bcax $A[3][4],$A[3][4],$A[3][1],$A[3][0] + bcax $A[3][0],$A[3][0],$D[1], $A[3][1] + bcax $A[3][1],$A[3][1],$A[0][3],$D[1] // A[0][3]=A[3][3] + + bcax $A[2][0],$C[0], $A[2][2],$D[2] + bcax $A[2][1],$D[2], $A[2][3],$A[2][2] + bcax $A[2][2],$A[2][2],$A[2][4],$A[2][3] + bcax $A[2][3],$A[2][3],$C[0], $A[2][4] + bcax $A[2][4],$A[2][4],$D[2], $C[0] + + bcax $A[1][2],$D[0], $A[1][4],$A[0][4] // A[0][4]=A[1][3] + bcax $A[1][3],$A[0][4],$A[1][0],$A[1][4] // A[0][4]=A[1][3] + bcax $A[1][4],$A[1][4],$A[1][1],$A[1][0] + bcax $A[1][0],$A[1][0],$D[0], $A[1][1] + bcax $A[1][1],$A[1][1],$A[0][4],$D[0] // A[0][4]=A[1][3] + + bcax $A[0][3],$D[3], $A[0][0],$D[4] + bcax $A[0][4],$D[4], $A[0][1],$A[0][0] + bcax $A[0][0],$A[0][0],$A[0][2],$A[0][1] + bcax $A[0][1],$A[0][1],$D[3], $A[0][2] + bcax $A[0][2],$A[0][2],$D[4], $D[3] + + eor $A[0][0],$A[0][0],$C[1] + + tst x10,#255 + bne .Loop_ce + + ret +.size KeccakF1600_ce,.-KeccakF1600_ce + +.type KeccakF1600_cext,%function +.align 5 +KeccakF1600_cext: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-2*__SIZEOF_POINTER__-64]! + add c29,csp,#0 + stp d8,d9,[csp,#2*__SIZEOF_POINTER__+0] // per ABI requirement + stp d10,d11,[csp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[csp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[csp,#2*__SIZEOF_POINTER__+48] +___ +for($i=0; $i<24; $i+=2) { # load A[5][5] +my $j=$i+1; +$code.=<<___; + ldp d$i,d$j,[x0,#8*$i] +___ +} +$code.=<<___; + ldr d24,[x0,#8*$i] + adr x10,iotas + bl KeccakF1600_ce + ldr c30,[csp,#__SIZEOF_POINTER__] +___ +for($i=0; $i<24; $i+=2) { # store A[5][5] +my $j=$i+1; +$code.=<<___; + stp d$i,d$j,[x0,#8*$i] +___ +} +$code.=<<___; + str d24,[x0,#8*$i] + + ldp d8,d9,[csp,#2*__SIZEOF_POINTER__+0] + ldp d10,d11,[csp,#2*__SIZEOF_POINTER__+16] + ldp d12,d13,[csp,#2*__SIZEOF_POINTER__+32] + ldp d14,d15,[csp,#2*__SIZEOF_POINTER__+48] + ldr c29,[csp],#2*__SIZEOF_POINTER__+64 + .inst 0xd50323bf // autiasp + ret +.size KeccakF1600_cext,.-KeccakF1600_cext +___ + +{ +my ($ctx,$inp,$len,$bsz) = map("x$_",(0..3)); + +$code.=<<___; +.globl SHA3_absorb_cext +.type SHA3_absorb_cext,%function +.align 5 +SHA3_absorb_cext: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-2*__SIZEOF_POINTER__-64]! + add c29,csp,#0 + stp d8,d9,[csp,#2*__SIZEOF_POINTER__+0] // per ABI requirement + stp d10,d11,[csp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[csp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[csp,#2*__SIZEOF_POINTER__+48] +___ +for($i=0; $i<24; $i+=2) { # load A[5][5] +my $j=$i+1; +$code.=<<___; + ldp d$i,d$j,[x0,#8*$i] +___ +} +$code.=<<___; + ldr d24,[x0,#8*$i] + b .Loop_absorb_ce + +.align 4 +.Loop_absorb_ce: + subs $len,$len,$bsz // len - bsz + blo .Labsorbed_ce + + cmp $bsz,#104 +___ +sub load_n_xor_ce { + my ($from,$to) = @_; + my $range = $to-$from+1; + + while ($range>=4) { +$code.=<<___; + ld1 {v27.8b-v30.8b},[$inp],#32 + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v27.16b + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v28.16b + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v29.16b + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v30.16b +___ + $range-=4; + } + while ($range>=3) { +$code.=<<___; + ld1 {v28.8b-v30.8b},[$inp],#24 + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v28.16b + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v29.16b + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v30.16b +___ + $range-=3; + } + while ($from<=$to) { +$code.=<<___; + ld1 {v31.8b},[$inp],#8 // A[`$from/5`][`$from%5`] ^= *inp++ + eor $A[$from/5][$from%5],$A[$from/5][$from++%5],v31.16b +___ + } +} +load_n_xor_ce(0,8); +$code.=<<___; + blo .Lprocess_block_ce + +___ +load_n_xor_ce(9,12); +$code.=<<___; + beq .Lprocess_block_ce + + cmp $bsz,#144 +___ +load_n_xor_ce(13,16); +$code.=<<___; + blo .Lprocess_block_ce + +___ +load_n_xor_ce(17,17); +$code.=<<___; + beq .Lprocess_block_ce + +___ +load_n_xor_ce(18,20); +$code.=<<___; + +.Lprocess_block_ce: + adr x10,iotas + bl KeccakF1600_ce + + b .Loop_absorb_ce + +.align 4 +.Labsorbed_ce: +___ +for($i=0; $i<24; $i+=2) { # store A[5][5] +my $j=$i+1; +$code.=<<___; + stp d$i,d$j,[x0,#8*$i] +___ +} +$code.=<<___; + str d24,[x0,#8*$i] + add x0,$len,$bsz // return value + + ldp d8,d9,[csp,#2*__SIZEOF_POINTER__+0] + ldp d10,d11,[csp,#2*__SIZEOF_POINTER__+16] + ldp d12,d13,[csp,#2*__SIZEOF_POINTER__+32] + ldp d14,d15,[csp,#2*__SIZEOF_POINTER__+48] + ldp c29,c30,[csp],#2*__SIZEOF_POINTER__+64 + .inst 0xd50323bf // autiasp + ret +.size SHA3_absorb_cext,.-SHA3_absorb_cext +___ +} +{ +my ($ctx,$out,$len,$bsz) = map("x$_",(0..3)); +$code.=<<___; +.globl SHA3_squeeze_cext +.type SHA3_squeeze_cext,%function +.align 5 +SHA3_squeeze_cext: + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-2*__SIZEOF_POINTER__]! + add c29,csp,#0 + cmov x9,$ctx + mov x10,$bsz + +.Loop_squeeze_ce: + ldr x4,[x9],#8 + cmp $len,#8 + blo .Lsqueeze_tail_ce +#ifdef __AARCH64EB__ + rev x4,x4 +#endif + str x4,[$out],#8 + beq .Lsqueeze_done_ce + + sub $len,$len,#8 + subs x10,x10,#8 + bhi .Loop_squeeze_ce + + bl KeccakF1600_cext + ldr c30,[csp,#__SIZEOF_POINTER__] + cmov x9,$ctx + mov x10,$bsz + b .Loop_squeeze_ce + +.align 4 +.Lsqueeze_tail_ce: + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done_ce + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done_ce + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done_ce + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done_ce + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done_ce + strb w4,[$out],#1 + lsr x4,x4,#8 + subs $len,$len,#1 + beq .Lsqueeze_done_ce + strb w4,[$out],#1 + +.Lsqueeze_done_ce: + ldr c29,[csp],#2*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size SHA3_squeeze_cext,.-SHA3_squeeze_cext +___ +} }}} +$code.=<<___; +.asciz "Keccak-1600 absorb and squeeze for ARMv8, CRYPTOGAMS by \@dot-asm" +___ + +{ my %opcode = ( + "rax1" => 0xce608c00, "eor3" => 0xce000000, + "bcax" => 0xce200000, "xar" => 0xce800000 ); + + sub unsha3 { + my ($mnemonic,$arg)=@_; + + $arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv#]([0-9\-]+))?)?/ + && + sprintf ".inst\t0x%08x\t//%s %s", + $opcode{$mnemonic}|$1|($2<<5)|($3<<16)|(eval($4)<<10), + $mnemonic,$arg; + } +} + +foreach(split("\n",$code)) { + use integer; + + s/\`([^\`]*)\`/eval($1)/ge; + + m/\b(ld1r|rax1|xar)\b/ and s/\.16b/.2d/g; + $sha3ops or s/\b(eor3|rax1|xar|bcax)\s+(v.*)/unsha3($1,$2)/ge; + s/([cw])#x([0-9]+)/$1$2/g; + + print $_,"\n"; +} + +close STDOUT; diff --git a/cbits/asm/keccak1600-x86_64-elf.S b/cbits/asm/keccak1600-x86_64-elf.S new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-x86_64-elf.S @@ -0,0 +1,538 @@ +.text + +.type __crypton_keccak_asm_f1600,@function +.align 32 +__crypton_keccak_asm_f1600: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + movq 60(%rdi),%rax + movq 68(%rdi),%rbx + movq 76(%rdi),%rcx + movq 84(%rdi),%rdx + movq 92(%rdi),%rbp + jmp .Loop + +.align 32 +.Loop: + movq -100(%rdi),%r8 + movq -52(%rdi),%r9 + movq -4(%rdi),%r10 + movq 44(%rdi),%r11 + + xorq -84(%rdi),%rcx + xorq -76(%rdi),%rdx + xorq %r8,%rax + xorq -92(%rdi),%rbx + xorq -44(%rdi),%rcx + xorq -60(%rdi),%rax + movq %rbp,%r12 + xorq -68(%rdi),%rbp + + xorq %r10,%rcx + xorq -20(%rdi),%rax + xorq -36(%rdi),%rdx + xorq %r9,%rbx + xorq -28(%rdi),%rbp + + xorq 36(%rdi),%rcx + xorq 20(%rdi),%rax + xorq 4(%rdi),%rdx + xorq -12(%rdi),%rbx + xorq 12(%rdi),%rbp + + movq %rcx,%r13 + rolq $1,%rcx + xorq %rax,%rcx + xorq %r11,%rdx + + rolq $1,%rax + xorq %rdx,%rax + xorq 28(%rdi),%rbx + + rolq $1,%rdx + xorq %rbx,%rdx + xorq 52(%rdi),%rbp + + rolq $1,%rbx + xorq %rbp,%rbx + + rolq $1,%rbp + xorq %r13,%rbp + xorq %rcx,%r9 + xorq %rdx,%r10 + rolq $44,%r9 + xorq %rbp,%r11 + xorq %rax,%r12 + rolq $43,%r10 + xorq %rbx,%r8 + movq %r9,%r13 + rolq $21,%r11 + orq %r10,%r9 + xorq %r8,%r9 + rolq $14,%r12 + + xorq (%r15),%r9 + leaq 8(%r15),%r15 + + movq %r12,%r14 + andq %r11,%r12 + movq %r9,-100(%rsi) + xorq %r10,%r12 + notq %r10 + movq %r12,-84(%rsi) + + orq %r11,%r10 + movq 76(%rdi),%r12 + xorq %r13,%r10 + movq %r10,-92(%rsi) + + andq %r8,%r13 + movq -28(%rdi),%r9 + xorq %r14,%r13 + movq -20(%rdi),%r10 + movq %r13,-68(%rsi) + + orq %r8,%r14 + movq -76(%rdi),%r8 + xorq %r11,%r14 + movq 28(%rdi),%r11 + movq %r14,-76(%rsi) + + + xorq %rbp,%r8 + xorq %rdx,%r12 + rolq $28,%r8 + xorq %rcx,%r11 + xorq %rax,%r9 + rolq $61,%r12 + rolq $45,%r11 + xorq %rbx,%r10 + rolq $20,%r9 + movq %r8,%r13 + orq %r12,%r8 + rolq $3,%r10 + + xorq %r11,%r8 + movq %r8,-36(%rsi) + + movq %r9,%r14 + andq %r13,%r9 + movq -92(%rdi),%r8 + xorq %r12,%r9 + notq %r12 + movq %r9,-28(%rsi) + + orq %r11,%r12 + movq -44(%rdi),%r9 + xorq %r10,%r12 + movq %r12,-44(%rsi) + + andq %r10,%r11 + movq 60(%rdi),%r12 + xorq %r14,%r11 + movq %r11,-52(%rsi) + + orq %r10,%r14 + movq 4(%rdi),%r10 + xorq %r13,%r14 + movq 52(%rdi),%r11 + movq %r14,-60(%rsi) + + + xorq %rbp,%r10 + xorq %rax,%r11 + rolq $25,%r10 + xorq %rdx,%r9 + rolq $8,%r11 + xorq %rbx,%r12 + rolq $6,%r9 + xorq %rcx,%r8 + rolq $18,%r12 + movq %r10,%r13 + andq %r11,%r10 + rolq $1,%r8 + + notq %r11 + xorq %r9,%r10 + movq %r10,-12(%rsi) + + movq %r12,%r14 + andq %r11,%r12 + movq -12(%rdi),%r10 + xorq %r13,%r12 + movq %r12,-4(%rsi) + + orq %r9,%r13 + movq 84(%rdi),%r12 + xorq %r8,%r13 + movq %r13,-20(%rsi) + + andq %r8,%r9 + xorq %r14,%r9 + movq %r9,12(%rsi) + + orq %r8,%r14 + movq -60(%rdi),%r9 + xorq %r11,%r14 + movq 36(%rdi),%r11 + movq %r14,4(%rsi) + + + movq -68(%rdi),%r8 + + xorq %rcx,%r10 + xorq %rdx,%r11 + rolq $10,%r10 + xorq %rbx,%r9 + rolq $15,%r11 + xorq %rbp,%r12 + rolq $36,%r9 + xorq %rax,%r8 + rolq $56,%r12 + movq %r10,%r13 + orq %r11,%r10 + rolq $27,%r8 + + notq %r11 + xorq %r9,%r10 + movq %r10,28(%rsi) + + movq %r12,%r14 + orq %r11,%r12 + xorq %r13,%r12 + movq %r12,36(%rsi) + + andq %r9,%r13 + xorq %r8,%r13 + movq %r13,20(%rsi) + + orq %r8,%r9 + xorq %r14,%r9 + movq %r9,52(%rsi) + + andq %r14,%r8 + xorq %r11,%r8 + movq %r8,44(%rsi) + + + xorq -84(%rdi),%rdx + xorq -36(%rdi),%rbp + rolq $62,%rdx + xorq 68(%rdi),%rcx + rolq $55,%rbp + xorq 12(%rdi),%rax + rolq $2,%rcx + xorq 20(%rdi),%rbx + xchgq %rsi,%rdi + rolq $39,%rax + rolq $41,%rbx + movq %rdx,%r13 + andq %rbp,%rdx + notq %rbp + xorq %rcx,%rdx + movq %rdx,92(%rdi) + + movq %rax,%r14 + andq %rbp,%rax + xorq %r13,%rax + movq %rax,60(%rdi) + + orq %rcx,%r13 + xorq %rbx,%r13 + movq %r13,84(%rdi) + + andq %rbx,%rcx + xorq %r14,%rcx + movq %rcx,76(%rdi) + + orq %r14,%rbx + xorq %rbp,%rbx + movq %rbx,68(%rdi) + + movq %rdx,%rbp + movq %r13,%rdx + + testq $255,%r15 + jnz .Loop + + leaq -192(%r15),%r15 + .byte 0xf3,0xc3 +.cfi_endproc +.size __crypton_keccak_asm_f1600,.-__crypton_keccak_asm_f1600 + +.globl crypton_keccak_asm_f1600 +.type crypton_keccak_asm_f1600,@function +.align 32 +crypton_keccak_asm_f1600: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + + leaq 100(%rdi),%rdi + subq $200,%rsp +.cfi_adjust_cfa_offset 200 + + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + + leaq iotas(%rip),%r15 + leaq 100(%rsp),%rsi + + call __crypton_keccak_asm_f1600 + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + leaq -100(%rdi),%rdi + + leaq 248(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq -48(%r11),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbp + movq -8(%r11),%rbx + leaq (%r11),%rsp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_keccak_asm_f1600,.-crypton_keccak_asm_f1600 +.globl crypton_keccak_asm_absorb +.type crypton_keccak_asm_absorb,@function +.align 32 +crypton_keccak_asm_absorb: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + + leaq 100(%rdi),%rdi + subq $232,%rsp +.cfi_adjust_cfa_offset 232 + + + movq %rsi,%r9 + leaq 100(%rsp),%rsi + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + leaq iotas(%rip),%r15 + + movq %rcx,216-100(%rsi) + +.Loop_absorb: + cmpq %rcx,%rdx + jc .Ldone_absorb + + shrq $3,%rcx + leaq -100(%rdi),%r8 + +.Lblock_absorb: + movq (%r9),%rax + leaq 8(%r9),%r9 + xorq (%r8),%rax + leaq 8(%r8),%r8 + subq $8,%rdx + movq %rax,-8(%r8) + subq $1,%rcx + jnz .Lblock_absorb + + movq %r9,200-100(%rsi) + movq %rdx,208-100(%rsi) + call __crypton_keccak_asm_f1600 + movq 200-100(%rsi),%r9 + movq 208-100(%rsi),%rdx + movq 216-100(%rsi),%rcx + jmp .Loop_absorb + +.align 32 +.Ldone_absorb: + movq %rdx,%rax + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + + leaq 280(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq -48(%r11),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbp + movq -8(%r11),%rbx + leaq (%r11),%rsp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_keccak_asm_absorb,.-crypton_keccak_asm_absorb +.globl crypton_keccak_asm_squeeze +.type crypton_keccak_asm_squeeze,@function +.align 32 +crypton_keccak_asm_squeeze: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-16 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-24 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-32 + subq $32,%rsp +.cfi_adjust_cfa_offset 32 + + + shrq $3,%rcx + movq %rdi,%r8 + movq %rsi,%r12 + movq %rdx,%r13 + movq %rcx,%r14 + jmp .Loop_squeeze + +.align 32 +.Loop_squeeze: + cmpq $8,%r13 + jb .Ltail_squeeze + + movq (%r8),%rax + leaq 8(%r8),%r8 + movq %rax,(%r12) + leaq 8(%r12),%r12 + subq $8,%r13 + jz .Ldone_squeeze + + subq $1,%rcx + jnz .Loop_squeeze + + movq %rdi,%rcx + call crypton_keccak_asm_f1600 + movq %rdi,%r8 + movq %r14,%rcx + jmp .Loop_squeeze + +.Ltail_squeeze: + movq %r8,%rsi + movq %r12,%rdi + movq %r13,%rcx +.byte 0xf3,0xa4 + +.Ldone_squeeze: + movq 32(%rsp),%r14 + movq 40(%rsp),%r13 + movq 48(%rsp),%r12 + addq $56,%rsp +.cfi_adjust_cfa_offset -56 +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_keccak_asm_squeeze,.-crypton_keccak_asm_squeeze +.align 256 +.quad 0,0,0,0,0,0,0,0 +.type iotas,@object +iotas: +.quad 0x0000000000000001 +.quad 0x0000000000008082 +.quad 0x800000000000808a +.quad 0x8000000080008000 +.quad 0x000000000000808b +.quad 0x0000000080000001 +.quad 0x8000000080008081 +.quad 0x8000000000008009 +.quad 0x000000000000008a +.quad 0x0000000000000088 +.quad 0x0000000080008009 +.quad 0x000000008000000a +.quad 0x000000008000808b +.quad 0x800000000000008b +.quad 0x8000000000008089 +.quad 0x8000000000008003 +.quad 0x8000000000008002 +.quad 0x8000000000000080 +.quad 0x000000000000800a +.quad 0x800000008000000a +.quad 0x8000000080008081 +.quad 0x8000000000008080 +.quad 0x0000000080000001 +.quad 0x8000000080008008 +.size iotas,.-iotas +.byte 75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 + +.section .note.gnu.property,"a",@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align 8 +2: + +.section .note.GNU-stack,"",@progbits diff --git a/cbits/asm/keccak1600-x86_64-macosx.S b/cbits/asm/keccak1600-x86_64-macosx.S new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-x86_64-macosx.S @@ -0,0 +1,529 @@ +.text + + +.p2align 5 +__crypton_keccak_asm_f1600: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + movq 60(%rdi),%rax + movq 68(%rdi),%rbx + movq 76(%rdi),%rcx + movq 84(%rdi),%rdx + movq 92(%rdi),%rbp + jmp L$oop + +.p2align 5 +L$oop: + movq -100(%rdi),%r8 + movq -52(%rdi),%r9 + movq -4(%rdi),%r10 + movq 44(%rdi),%r11 + + xorq -84(%rdi),%rcx + xorq -76(%rdi),%rdx + xorq %r8,%rax + xorq -92(%rdi),%rbx + xorq -44(%rdi),%rcx + xorq -60(%rdi),%rax + movq %rbp,%r12 + xorq -68(%rdi),%rbp + + xorq %r10,%rcx + xorq -20(%rdi),%rax + xorq -36(%rdi),%rdx + xorq %r9,%rbx + xorq -28(%rdi),%rbp + + xorq 36(%rdi),%rcx + xorq 20(%rdi),%rax + xorq 4(%rdi),%rdx + xorq -12(%rdi),%rbx + xorq 12(%rdi),%rbp + + movq %rcx,%r13 + rolq $1,%rcx + xorq %rax,%rcx + xorq %r11,%rdx + + rolq $1,%rax + xorq %rdx,%rax + xorq 28(%rdi),%rbx + + rolq $1,%rdx + xorq %rbx,%rdx + xorq 52(%rdi),%rbp + + rolq $1,%rbx + xorq %rbp,%rbx + + rolq $1,%rbp + xorq %r13,%rbp + xorq %rcx,%r9 + xorq %rdx,%r10 + rolq $44,%r9 + xorq %rbp,%r11 + xorq %rax,%r12 + rolq $43,%r10 + xorq %rbx,%r8 + movq %r9,%r13 + rolq $21,%r11 + orq %r10,%r9 + xorq %r8,%r9 + rolq $14,%r12 + + xorq (%r15),%r9 + leaq 8(%r15),%r15 + + movq %r12,%r14 + andq %r11,%r12 + movq %r9,-100(%rsi) + xorq %r10,%r12 + notq %r10 + movq %r12,-84(%rsi) + + orq %r11,%r10 + movq 76(%rdi),%r12 + xorq %r13,%r10 + movq %r10,-92(%rsi) + + andq %r8,%r13 + movq -28(%rdi),%r9 + xorq %r14,%r13 + movq -20(%rdi),%r10 + movq %r13,-68(%rsi) + + orq %r8,%r14 + movq -76(%rdi),%r8 + xorq %r11,%r14 + movq 28(%rdi),%r11 + movq %r14,-76(%rsi) + + + xorq %rbp,%r8 + xorq %rdx,%r12 + rolq $28,%r8 + xorq %rcx,%r11 + xorq %rax,%r9 + rolq $61,%r12 + rolq $45,%r11 + xorq %rbx,%r10 + rolq $20,%r9 + movq %r8,%r13 + orq %r12,%r8 + rolq $3,%r10 + + xorq %r11,%r8 + movq %r8,-36(%rsi) + + movq %r9,%r14 + andq %r13,%r9 + movq -92(%rdi),%r8 + xorq %r12,%r9 + notq %r12 + movq %r9,-28(%rsi) + + orq %r11,%r12 + movq -44(%rdi),%r9 + xorq %r10,%r12 + movq %r12,-44(%rsi) + + andq %r10,%r11 + movq 60(%rdi),%r12 + xorq %r14,%r11 + movq %r11,-52(%rsi) + + orq %r10,%r14 + movq 4(%rdi),%r10 + xorq %r13,%r14 + movq 52(%rdi),%r11 + movq %r14,-60(%rsi) + + + xorq %rbp,%r10 + xorq %rax,%r11 + rolq $25,%r10 + xorq %rdx,%r9 + rolq $8,%r11 + xorq %rbx,%r12 + rolq $6,%r9 + xorq %rcx,%r8 + rolq $18,%r12 + movq %r10,%r13 + andq %r11,%r10 + rolq $1,%r8 + + notq %r11 + xorq %r9,%r10 + movq %r10,-12(%rsi) + + movq %r12,%r14 + andq %r11,%r12 + movq -12(%rdi),%r10 + xorq %r13,%r12 + movq %r12,-4(%rsi) + + orq %r9,%r13 + movq 84(%rdi),%r12 + xorq %r8,%r13 + movq %r13,-20(%rsi) + + andq %r8,%r9 + xorq %r14,%r9 + movq %r9,12(%rsi) + + orq %r8,%r14 + movq -60(%rdi),%r9 + xorq %r11,%r14 + movq 36(%rdi),%r11 + movq %r14,4(%rsi) + + + movq -68(%rdi),%r8 + + xorq %rcx,%r10 + xorq %rdx,%r11 + rolq $10,%r10 + xorq %rbx,%r9 + rolq $15,%r11 + xorq %rbp,%r12 + rolq $36,%r9 + xorq %rax,%r8 + rolq $56,%r12 + movq %r10,%r13 + orq %r11,%r10 + rolq $27,%r8 + + notq %r11 + xorq %r9,%r10 + movq %r10,28(%rsi) + + movq %r12,%r14 + orq %r11,%r12 + xorq %r13,%r12 + movq %r12,36(%rsi) + + andq %r9,%r13 + xorq %r8,%r13 + movq %r13,20(%rsi) + + orq %r8,%r9 + xorq %r14,%r9 + movq %r9,52(%rsi) + + andq %r14,%r8 + xorq %r11,%r8 + movq %r8,44(%rsi) + + + xorq -84(%rdi),%rdx + xorq -36(%rdi),%rbp + rolq $62,%rdx + xorq 68(%rdi),%rcx + rolq $55,%rbp + xorq 12(%rdi),%rax + rolq $2,%rcx + xorq 20(%rdi),%rbx + xchgq %rsi,%rdi + rolq $39,%rax + rolq $41,%rbx + movq %rdx,%r13 + andq %rbp,%rdx + notq %rbp + xorq %rcx,%rdx + movq %rdx,92(%rdi) + + movq %rax,%r14 + andq %rbp,%rax + xorq %r13,%rax + movq %rax,60(%rdi) + + orq %rcx,%r13 + xorq %rbx,%r13 + movq %r13,84(%rdi) + + andq %rbx,%rcx + xorq %r14,%rcx + movq %rcx,76(%rdi) + + orq %r14,%rbx + xorq %rbp,%rbx + movq %rbx,68(%rdi) + + movq %rdx,%rbp + movq %r13,%rdx + + testq $255,%r15 + jnz L$oop + + leaq -192(%r15),%r15 + .byte 0xf3,0xc3 +.cfi_endproc + + +.globl _crypton_keccak_asm_f1600 + +.p2align 5 +_crypton_keccak_asm_f1600: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + + leaq 100(%rdi),%rdi + subq $200,%rsp +.cfi_adjust_cfa_offset 200 + + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + + leaq iotas(%rip),%r15 + leaq 100(%rsp),%rsi + + call __crypton_keccak_asm_f1600 + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + leaq -100(%rdi),%rdi + + leaq 248(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq -48(%r11),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbp + movq -8(%r11),%rbx + leaq (%r11),%rsp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + +.globl _crypton_keccak_asm_absorb + +.p2align 5 +_crypton_keccak_asm_absorb: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + + leaq 100(%rdi),%rdi + subq $232,%rsp +.cfi_adjust_cfa_offset 232 + + + movq %rsi,%r9 + leaq 100(%rsp),%rsi + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + leaq iotas(%rip),%r15 + + movq %rcx,216-100(%rsi) + +L$oop_absorb: + cmpq %rcx,%rdx + jc L$done_absorb + + shrq $3,%rcx + leaq -100(%rdi),%r8 + +L$block_absorb: + movq (%r9),%rax + leaq 8(%r9),%r9 + xorq (%r8),%rax + leaq 8(%r8),%r8 + subq $8,%rdx + movq %rax,-8(%r8) + subq $1,%rcx + jnz L$block_absorb + + movq %r9,200-100(%rsi) + movq %rdx,208-100(%rsi) + call __crypton_keccak_asm_f1600 + movq 200-100(%rsi),%r9 + movq 208-100(%rsi),%rdx + movq 216-100(%rsi),%rcx + jmp L$oop_absorb + +.p2align 5 +L$done_absorb: + movq %rdx,%rax + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + + leaq 280(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq -48(%r11),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbp + movq -8(%r11),%rbx + leaq (%r11),%rsp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + +.globl _crypton_keccak_asm_squeeze + +.p2align 5 +_crypton_keccak_asm_squeeze: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-16 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-24 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-32 + subq $32,%rsp +.cfi_adjust_cfa_offset 32 + + + shrq $3,%rcx + movq %rdi,%r8 + movq %rsi,%r12 + movq %rdx,%r13 + movq %rcx,%r14 + jmp L$oop_squeeze + +.p2align 5 +L$oop_squeeze: + cmpq $8,%r13 + jb L$tail_squeeze + + movq (%r8),%rax + leaq 8(%r8),%r8 + movq %rax,(%r12) + leaq 8(%r12),%r12 + subq $8,%r13 + jz L$done_squeeze + + subq $1,%rcx + jnz L$oop_squeeze + + movq %rdi,%rcx + call _crypton_keccak_asm_f1600 + movq %rdi,%r8 + movq %r14,%rcx + jmp L$oop_squeeze + +L$tail_squeeze: + movq %r8,%rsi + movq %r12,%rdi + movq %r13,%rcx +.byte 0xf3,0xa4 + +L$done_squeeze: + movq 32(%rsp),%r14 + movq 40(%rsp),%r13 + movq 48(%rsp),%r12 + addq $56,%rsp +.cfi_adjust_cfa_offset -56 +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 + .byte 0xf3,0xc3 +.cfi_endproc + +.p2align 8 +.quad 0,0,0,0,0,0,0,0 + +iotas: +.quad 0x0000000000000001 +.quad 0x0000000000008082 +.quad 0x800000000000808a +.quad 0x8000000080008000 +.quad 0x000000000000808b +.quad 0x0000000080000001 +.quad 0x8000000080008081 +.quad 0x8000000000008009 +.quad 0x000000000000008a +.quad 0x0000000000000088 +.quad 0x0000000080008009 +.quad 0x000000008000000a +.quad 0x000000008000808b +.quad 0x800000000000008b +.quad 0x8000000000008089 +.quad 0x8000000000008003 +.quad 0x8000000000008002 +.quad 0x8000000000000080 +.quad 0x000000000000800a +.quad 0x800000008000000a +.quad 0x8000000080008081 +.quad 0x8000000000008080 +.quad 0x0000000080000001 +.quad 0x8000000080008008 + +.byte 75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 diff --git a/cbits/asm/keccak1600-x86_64-mingw64.S b/cbits/asm/keccak1600-x86_64-mingw64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-x86_64-mingw64.S @@ -0,0 +1,648 @@ +.text + +.def __crypton_keccak_asm_f1600; .scl 3; .type 32; .endef +.p2align 5 +__crypton_keccak_asm_f1600: + .byte 0xf3,0x0f,0x1e,0xfa + + movq 60(%rdi),%rax + movq 68(%rdi),%rbx + movq 76(%rdi),%rcx + movq 84(%rdi),%rdx + movq 92(%rdi),%rbp + jmp .Loop + +.p2align 5 +.Loop: + movq -100(%rdi),%r8 + movq -52(%rdi),%r9 + movq -4(%rdi),%r10 + movq 44(%rdi),%r11 + + xorq -84(%rdi),%rcx + xorq -76(%rdi),%rdx + xorq %r8,%rax + xorq -92(%rdi),%rbx + xorq -44(%rdi),%rcx + xorq -60(%rdi),%rax + movq %rbp,%r12 + xorq -68(%rdi),%rbp + + xorq %r10,%rcx + xorq -20(%rdi),%rax + xorq -36(%rdi),%rdx + xorq %r9,%rbx + xorq -28(%rdi),%rbp + + xorq 36(%rdi),%rcx + xorq 20(%rdi),%rax + xorq 4(%rdi),%rdx + xorq -12(%rdi),%rbx + xorq 12(%rdi),%rbp + + movq %rcx,%r13 + rolq $1,%rcx + xorq %rax,%rcx + xorq %r11,%rdx + + rolq $1,%rax + xorq %rdx,%rax + xorq 28(%rdi),%rbx + + rolq $1,%rdx + xorq %rbx,%rdx + xorq 52(%rdi),%rbp + + rolq $1,%rbx + xorq %rbp,%rbx + + rolq $1,%rbp + xorq %r13,%rbp + xorq %rcx,%r9 + xorq %rdx,%r10 + rolq $44,%r9 + xorq %rbp,%r11 + xorq %rax,%r12 + rolq $43,%r10 + xorq %rbx,%r8 + movq %r9,%r13 + rolq $21,%r11 + orq %r10,%r9 + xorq %r8,%r9 + rolq $14,%r12 + + xorq (%r15),%r9 + leaq 8(%r15),%r15 + + movq %r12,%r14 + andq %r11,%r12 + movq %r9,-100(%rsi) + xorq %r10,%r12 + notq %r10 + movq %r12,-84(%rsi) + + orq %r11,%r10 + movq 76(%rdi),%r12 + xorq %r13,%r10 + movq %r10,-92(%rsi) + + andq %r8,%r13 + movq -28(%rdi),%r9 + xorq %r14,%r13 + movq -20(%rdi),%r10 + movq %r13,-68(%rsi) + + orq %r8,%r14 + movq -76(%rdi),%r8 + xorq %r11,%r14 + movq 28(%rdi),%r11 + movq %r14,-76(%rsi) + + + xorq %rbp,%r8 + xorq %rdx,%r12 + rolq $28,%r8 + xorq %rcx,%r11 + xorq %rax,%r9 + rolq $61,%r12 + rolq $45,%r11 + xorq %rbx,%r10 + rolq $20,%r9 + movq %r8,%r13 + orq %r12,%r8 + rolq $3,%r10 + + xorq %r11,%r8 + movq %r8,-36(%rsi) + + movq %r9,%r14 + andq %r13,%r9 + movq -92(%rdi),%r8 + xorq %r12,%r9 + notq %r12 + movq %r9,-28(%rsi) + + orq %r11,%r12 + movq -44(%rdi),%r9 + xorq %r10,%r12 + movq %r12,-44(%rsi) + + andq %r10,%r11 + movq 60(%rdi),%r12 + xorq %r14,%r11 + movq %r11,-52(%rsi) + + orq %r10,%r14 + movq 4(%rdi),%r10 + xorq %r13,%r14 + movq 52(%rdi),%r11 + movq %r14,-60(%rsi) + + + xorq %rbp,%r10 + xorq %rax,%r11 + rolq $25,%r10 + xorq %rdx,%r9 + rolq $8,%r11 + xorq %rbx,%r12 + rolq $6,%r9 + xorq %rcx,%r8 + rolq $18,%r12 + movq %r10,%r13 + andq %r11,%r10 + rolq $1,%r8 + + notq %r11 + xorq %r9,%r10 + movq %r10,-12(%rsi) + + movq %r12,%r14 + andq %r11,%r12 + movq -12(%rdi),%r10 + xorq %r13,%r12 + movq %r12,-4(%rsi) + + orq %r9,%r13 + movq 84(%rdi),%r12 + xorq %r8,%r13 + movq %r13,-20(%rsi) + + andq %r8,%r9 + xorq %r14,%r9 + movq %r9,12(%rsi) + + orq %r8,%r14 + movq -60(%rdi),%r9 + xorq %r11,%r14 + movq 36(%rdi),%r11 + movq %r14,4(%rsi) + + + movq -68(%rdi),%r8 + + xorq %rcx,%r10 + xorq %rdx,%r11 + rolq $10,%r10 + xorq %rbx,%r9 + rolq $15,%r11 + xorq %rbp,%r12 + rolq $36,%r9 + xorq %rax,%r8 + rolq $56,%r12 + movq %r10,%r13 + orq %r11,%r10 + rolq $27,%r8 + + notq %r11 + xorq %r9,%r10 + movq %r10,28(%rsi) + + movq %r12,%r14 + orq %r11,%r12 + xorq %r13,%r12 + movq %r12,36(%rsi) + + andq %r9,%r13 + xorq %r8,%r13 + movq %r13,20(%rsi) + + orq %r8,%r9 + xorq %r14,%r9 + movq %r9,52(%rsi) + + andq %r14,%r8 + xorq %r11,%r8 + movq %r8,44(%rsi) + + + xorq -84(%rdi),%rdx + xorq -36(%rdi),%rbp + rolq $62,%rdx + xorq 68(%rdi),%rcx + rolq $55,%rbp + xorq 12(%rdi),%rax + rolq $2,%rcx + xorq 20(%rdi),%rbx + xchgq %rsi,%rdi + rolq $39,%rax + rolq $41,%rbx + movq %rdx,%r13 + andq %rbp,%rdx + notq %rbp + xorq %rcx,%rdx + movq %rdx,92(%rdi) + + movq %rax,%r14 + andq %rbp,%rax + xorq %r13,%rax + movq %rax,60(%rdi) + + orq %rcx,%r13 + xorq %rbx,%r13 + movq %r13,84(%rdi) + + andq %rbx,%rcx + xorq %r14,%rcx + movq %rcx,76(%rdi) + + orq %r14,%rbx + xorq %rbp,%rbx + movq %rbx,68(%rdi) + + movq %rdx,%rbp + movq %r13,%rdx + + testq $255,%r15 + jnz .Loop + + leaq -192(%r15),%r15 + .byte 0xf3,0xc3 + + +.globl crypton_keccak_asm_f1600 +.def crypton_keccak_asm_f1600; .scl 2; .type 32; .endef +.p2align 5 +crypton_keccak_asm_f1600: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_keccak_asm_f1600: + + + movq %rcx,%rdi + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + + leaq 100(%rdi),%rdi + subq $200,%rsp + +.LSEH_body_crypton_keccak_asm_f1600: + + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + + leaq iotas(%rip),%r15 + leaq 100(%rsp),%rsi + + call __crypton_keccak_asm_f1600 + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + leaq -100(%rdi),%rdi + + leaq 248(%rsp),%r11 + + movq -48(%r11),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbp + movq -8(%r11),%rbx + leaq (%r11),%rsp +.LSEH_epilogue_crypton_keccak_asm_f1600: + mov 8(%r11),%rdi + mov 16(%r11),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_keccak_asm_f1600: +.globl crypton_keccak_asm_absorb +.def crypton_keccak_asm_absorb; .scl 2; .type 32; .endef +.p2align 5 +crypton_keccak_asm_absorb: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_keccak_asm_absorb: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + + leaq 100(%rdi),%rdi + subq $232,%rsp + +.LSEH_body_crypton_keccak_asm_absorb: + + + movq %rsi,%r9 + leaq 100(%rsp),%rsi + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + leaq iotas(%rip),%r15 + + movq %rcx,216-100(%rsi) + +.Loop_absorb: + cmpq %rcx,%rdx + jc .Ldone_absorb + + shrq $3,%rcx + leaq -100(%rdi),%r8 + +.Lblock_absorb: + movq (%r9),%rax + leaq 8(%r9),%r9 + xorq (%r8),%rax + leaq 8(%r8),%r8 + subq $8,%rdx + movq %rax,-8(%r8) + subq $1,%rcx + jnz .Lblock_absorb + + movq %r9,200-100(%rsi) + movq %rdx,208-100(%rsi) + call __crypton_keccak_asm_f1600 + movq 200-100(%rsi),%r9 + movq 208-100(%rsi),%rdx + movq 216-100(%rsi),%rcx + jmp .Loop_absorb + +.p2align 5 +.Ldone_absorb: + movq %rdx,%rax + + notq -92(%rdi) + notq -84(%rdi) + notq -36(%rdi) + notq -4(%rdi) + notq 36(%rdi) + notq 60(%rdi) + + leaq 280(%rsp),%r11 + + movq -48(%r11),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbp + movq -8(%r11),%rbx + leaq (%r11),%rsp +.LSEH_epilogue_crypton_keccak_asm_absorb: + mov 8(%r11),%rdi + mov 16(%r11),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_keccak_asm_absorb: +.globl crypton_keccak_asm_squeeze +.def crypton_keccak_asm_squeeze; .scl 2; .type 32; .endef +.p2align 5 +crypton_keccak_asm_squeeze: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_keccak_asm_squeeze: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + pushq %r12 + + pushq %r13 + + pushq %r14 + + subq $32,%rsp + +.LSEH_body_crypton_keccak_asm_squeeze: + + + shrq $3,%rcx + movq %rdi,%r8 + movq %rsi,%r12 + movq %rdx,%r13 + movq %rcx,%r14 + jmp .Loop_squeeze + +.p2align 5 +.Loop_squeeze: + cmpq $8,%r13 + jb .Ltail_squeeze + + movq (%r8),%rax + leaq 8(%r8),%r8 + movq %rax,(%r12) + leaq 8(%r12),%r12 + subq $8,%r13 + jz .Ldone_squeeze + + subq $1,%rcx + jnz .Loop_squeeze + + movq %rdi,%rcx + call crypton_keccak_asm_f1600 + movq %rdi,%r8 + movq %r14,%rcx + jmp .Loop_squeeze + +.Ltail_squeeze: + movq %r8,%rsi + movq %r12,%rdi + movq %r13,%rcx +.byte 0xf3,0xa4 + +.Ldone_squeeze: + movq 32(%rsp),%r14 + movq 40(%rsp),%r13 + movq 48(%rsp),%r12 + addq $56,%rsp + +.LSEH_epilogue_crypton_keccak_asm_squeeze: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_keccak_asm_squeeze: +.p2align 8 +.quad 0,0,0,0,0,0,0,0 + +iotas: +.quad 0x0000000000000001 +.quad 0x0000000000008082 +.quad 0x800000000000808a +.quad 0x8000000080008000 +.quad 0x000000000000808b +.quad 0x0000000080000001 +.quad 0x8000000080008081 +.quad 0x8000000000008009 +.quad 0x000000000000008a +.quad 0x0000000000000088 +.quad 0x0000000080008009 +.quad 0x000000008000000a +.quad 0x000000008000808b +.quad 0x800000000000008b +.quad 0x8000000000008089 +.quad 0x8000000000008003 +.quad 0x8000000000008002 +.quad 0x8000000000000080 +.quad 0x000000000000800a +.quad 0x800000008000000a +.quad 0x8000000080008081 +.quad 0x8000000000008080 +.quad 0x0000000080000001 +.quad 0x8000000080008008 + +.byte 75,101,99,99,97,107,45,49,54,48,48,32,97,98,115,111,114,98,32,97,110,100,32,115,113,117,101,101,122,101,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,60,97,112,112,114,111,64,111,112,101,110,115,115,108,46,111,114,103,62,0 +.section .pdata +.p2align 2 +.rva .LSEH_begin_crypton_keccak_asm_f1600 +.rva .LSEH_body_crypton_keccak_asm_f1600 +.rva .LSEH_info_crypton_keccak_asm_f1600_prologue + +.rva .LSEH_body_crypton_keccak_asm_f1600 +.rva .LSEH_epilogue_crypton_keccak_asm_f1600 +.rva .LSEH_info_crypton_keccak_asm_f1600_body + +.rva .LSEH_epilogue_crypton_keccak_asm_f1600 +.rva .LSEH_end_crypton_keccak_asm_f1600 +.rva .LSEH_info_crypton_keccak_asm_f1600_epilogue + +.rva .LSEH_begin_crypton_keccak_asm_absorb +.rva .LSEH_body_crypton_keccak_asm_absorb +.rva .LSEH_info_crypton_keccak_asm_absorb_prologue + +.rva .LSEH_body_crypton_keccak_asm_absorb +.rva .LSEH_epilogue_crypton_keccak_asm_absorb +.rva .LSEH_info_crypton_keccak_asm_absorb_body + +.rva .LSEH_epilogue_crypton_keccak_asm_absorb +.rva .LSEH_end_crypton_keccak_asm_absorb +.rva .LSEH_info_crypton_keccak_asm_absorb_epilogue + +.rva .LSEH_begin_crypton_keccak_asm_squeeze +.rva .LSEH_body_crypton_keccak_asm_squeeze +.rva .LSEH_info_crypton_keccak_asm_squeeze_prologue + +.rva .LSEH_body_crypton_keccak_asm_squeeze +.rva .LSEH_epilogue_crypton_keccak_asm_squeeze +.rva .LSEH_info_crypton_keccak_asm_squeeze_body + +.rva .LSEH_epilogue_crypton_keccak_asm_squeeze +.rva .LSEH_end_crypton_keccak_asm_squeeze +.rva .LSEH_info_crypton_keccak_asm_squeeze_epilogue + +.section .xdata +.p2align 3 +.LSEH_info_crypton_keccak_asm_f1600_prologue: +.byte 1,0,5,0x0b +.byte 0,0x74,1,0 +.byte 0,0x64,2,0 +.byte 0,0xb3 +.byte 0,0 +.long 0,0 +.LSEH_info_crypton_keccak_asm_f1600_body: +.byte 1,0,18,0 +.byte 0x00,0xf4,0x19,0x00 +.byte 0x00,0xe4,0x1a,0x00 +.byte 0x00,0xd4,0x1b,0x00 +.byte 0x00,0xc4,0x1c,0x00 +.byte 0x00,0x54,0x1d,0x00 +.byte 0x00,0x34,0x1e,0x00 +.byte 0x00,0x74,0x20,0x00 +.byte 0x00,0x64,0x21,0x00 +.byte 0x00,0x01,0x1f,0x00 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_keccak_asm_f1600_epilogue: +.byte 1,0,5,11 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0xb3 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_keccak_asm_absorb_prologue: +.byte 1,0,5,0x0b +.byte 0,0x74,1,0 +.byte 0,0x64,2,0 +.byte 0,0xb3 +.byte 0,0 +.long 0,0 +.LSEH_info_crypton_keccak_asm_absorb_body: +.byte 1,0,18,0 +.byte 0x00,0xf4,0x1d,0x00 +.byte 0x00,0xe4,0x1e,0x00 +.byte 0x00,0xd4,0x1f,0x00 +.byte 0x00,0xc4,0x20,0x00 +.byte 0x00,0x54,0x21,0x00 +.byte 0x00,0x34,0x22,0x00 +.byte 0x00,0x74,0x24,0x00 +.byte 0x00,0x64,0x25,0x00 +.byte 0x00,0x01,0x23,0x00 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_keccak_asm_absorb_epilogue: +.byte 1,0,5,11 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0xb3 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_keccak_asm_squeeze_prologue: +.byte 1,0,5,0x0b +.byte 0,0x74,1,0 +.byte 0,0x64,2,0 +.byte 0,0xb3 +.byte 0,0 +.long 0,0 +.LSEH_info_crypton_keccak_asm_squeeze_body: +.byte 1,0,11,0 +.byte 0x00,0xe4,0x04,0x00 +.byte 0x00,0xd4,0x05,0x00 +.byte 0x00,0xc4,0x06,0x00 +.byte 0x00,0x74,0x08,0x00 +.byte 0x00,0x64,0x09,0x00 +.byte 0x00,0x62 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.LSEH_info_crypton_keccak_asm_squeeze_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + diff --git a/cbits/asm/keccak1600-x86_64.pl b/cbits/asm/keccak1600-x86_64.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/keccak1600-x86_64.pl @@ -0,0 +1,601 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov for the OpenSSL +# project. The module is, however, dual licensed under OpenSSL and +# CRYPTOGAMS licenses depending on where you obtain it. For further +# details see http://www.openssl.org/~appro/cryptogams/. +# ==================================================================== +# +# Keccak-1600 for x86_64. +# +# June 2017. +# +# Below code is [lane complementing] KECCAK_2X implementation (see +# sha/keccak1600.c) with C[5] and D[5] held in register bank. Though +# instead of actually unrolling the loop pair-wise I simply flip +# pointers to T[][] and A[][] at the end of round. Since number of +# rounds is even, last round writes to A[][] and everything works out. +# How does it compare to x86_64 assembly module in Keccak Code Package? +# Depending on processor it's either as fast or faster by up to 15%... +# +######################################################################## +# Numbers are cycles per processed byte out of large message. +# +# r=1088(*) +# +# P4 25.8 +# Core 2 12.9 +# Westmere 13.7 +# Sandy Bridge 12.9(**) +# Haswell 9.6 +# Skylake 9.4 +# Ice Lake 8.6 +# Silvermont 22.8 +# Goldmont 15.8 +# VIA Nano 17.3 +# Sledgehammer 13.3 +# Bulldozer 16.5 +# Ryzen 8.8 +# Zen 4 7.6 +# +# (*) Corresponds to SHA3-256. Improvement over compiler-generate +# varies a lot, most commont coefficient is 15% in comparison to +# gcc-5.x, 50% for gcc-4.x, 90% for gcc-3.x. +# (**) Sandy Bridge has broken rotate instruction. Performance can be +# improved by 14% by replacing rotates with double-precision +# shift with same register as source and destination. + +$flavour = shift; +$output = shift; +if ($flavour =~ /\./) { $output = $flavour; undef $flavour; } + +$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/); + +$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; +( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or +( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or +die "can't locate x86_64-xlate.pl"; + +open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\""; +*STDOUT=*OUT; + +my @A = map([ 8*$_-100, 8*($_+1)-100, 8*($_+2)-100, + 8*($_+3)-100, 8*($_+4)-100 ], (0,5,10,15,20)); + +my @C = ("%rax","%rbx","%rcx","%rdx","%rbp"); +my @D = map("%r$_",(8..12)); +my @T = map("%r$_",(13..14)); +my $iotas = "%r15"; + +my @rhotates = ([ 0, 1, 62, 28, 27 ], + [ 36, 44, 6, 55, 20 ], + [ 3, 10, 43, 25, 39 ], + [ 41, 45, 15, 21, 8 ], + [ 18, 2, 61, 56, 14 ]); + +$code.=<<___; +.text + +.type __KeccakF1600,\@abi-omnipotent +.align 32 +__KeccakF1600: + mov $A[4][0](%rdi),@C[0] + mov $A[4][1](%rdi),@C[1] + mov $A[4][2](%rdi),@C[2] + mov $A[4][3](%rdi),@C[3] + mov $A[4][4](%rdi),@C[4] + jmp .Loop + +.align 32 +.Loop: + mov $A[0][0](%rdi),@D[0] + mov $A[1][1](%rdi),@D[1] + mov $A[2][2](%rdi),@D[2] + mov $A[3][3](%rdi),@D[3] + + xor $A[0][2](%rdi),@C[2] + xor $A[0][3](%rdi),@C[3] + xor @D[0], @C[0] + xor $A[0][1](%rdi),@C[1] + xor $A[1][2](%rdi),@C[2] + xor $A[1][0](%rdi),@C[0] + mov @C[4],@D[4] + xor $A[0][4](%rdi),@C[4] + + xor @D[2], @C[2] + xor $A[2][0](%rdi),@C[0] + xor $A[1][3](%rdi),@C[3] + xor @D[1], @C[1] + xor $A[1][4](%rdi),@C[4] + + xor $A[3][2](%rdi),@C[2] + xor $A[3][0](%rdi),@C[0] + xor $A[2][3](%rdi),@C[3] + xor $A[2][1](%rdi),@C[1] + xor $A[2][4](%rdi),@C[4] + + mov @C[2],@T[0] + rol \$1,@C[2] + xor @C[0],@C[2] # D[1] = ROL64(C[2], 1) ^ C[0] + xor @D[3], @C[3] + + rol \$1,@C[0] + xor @C[3],@C[0] # D[4] = ROL64(C[0], 1) ^ C[3] + xor $A[3][1](%rdi),@C[1] + + rol \$1,@C[3] + xor @C[1],@C[3] # D[2] = ROL64(C[3], 1) ^ C[1] + xor $A[3][4](%rdi),@C[4] + + rol \$1,@C[1] + xor @C[4],@C[1] # D[0] = ROL64(C[1], 1) ^ C[4] + + rol \$1,@C[4] + xor @T[0],@C[4] # D[3] = ROL64(C[4], 1) ^ C[2] +___ + (@D[0..4], @C) = (@C[1..4,0], @D); +$code.=<<___; + xor @D[1],@C[1] + xor @D[2],@C[2] + rol \$$rhotates[1][1],@C[1] + xor @D[3],@C[3] + xor @D[4],@C[4] + rol \$$rhotates[2][2],@C[2] + xor @D[0],@C[0] + mov @C[1],@T[0] + rol \$$rhotates[3][3],@C[3] + or @C[2],@C[1] + xor @C[0],@C[1] # C[0] ^ ( C[1] | C[2]) + rol \$$rhotates[4][4],@C[4] + + xor ($iotas),@C[1] + lea 8($iotas),$iotas + + mov @C[4],@T[1] + and @C[3],@C[4] + mov @C[1],$A[0][0](%rsi) # R[0][0] = C[0] ^ ( C[1] | C[2]) ^ iotas[i] + xor @C[2],@C[4] # C[2] ^ ( C[4] & C[3]) + not @C[2] + mov @C[4],$A[0][2](%rsi) # R[0][2] = C[2] ^ ( C[4] & C[3]) + + or @C[3],@C[2] + mov $A[4][2](%rdi),@C[4] + xor @T[0],@C[2] # C[1] ^ (~C[2] | C[3]) + mov @C[2],$A[0][1](%rsi) # R[0][1] = C[1] ^ (~C[2] | C[3]) + + and @C[0],@T[0] + mov $A[1][4](%rdi),@C[1] + xor @T[1],@T[0] # C[4] ^ ( C[1] & C[0]) + mov $A[2][0](%rdi),@C[2] + mov @T[0],$A[0][4](%rsi) # R[0][4] = C[4] ^ ( C[1] & C[0]) + + or @C[0],@T[1] + mov $A[0][3](%rdi),@C[0] + xor @C[3],@T[1] # C[3] ^ ( C[4] | C[0]) + mov $A[3][1](%rdi),@C[3] + mov @T[1],$A[0][3](%rsi) # R[0][3] = C[3] ^ ( C[4] | C[0]) + + + xor @D[3],@C[0] + xor @D[2],@C[4] + rol \$$rhotates[0][3],@C[0] + xor @D[1],@C[3] + xor @D[4],@C[1] + rol \$$rhotates[4][2],@C[4] + rol \$$rhotates[3][1],@C[3] + xor @D[0],@C[2] + rol \$$rhotates[1][4],@C[1] + mov @C[0],@T[0] + or @C[4],@C[0] + rol \$$rhotates[2][0],@C[2] + + xor @C[3],@C[0] # C[3] ^ (C[0] | C[4]) + mov @C[0],$A[1][3](%rsi) # R[1][3] = C[3] ^ (C[0] | C[4]) + + mov @C[1],@T[1] + and @T[0],@C[1] + mov $A[0][1](%rdi),@C[0] + xor @C[4],@C[1] # C[4] ^ (C[1] & C[0]) + not @C[4] + mov @C[1],$A[1][4](%rsi) # R[1][4] = C[4] ^ (C[1] & C[0]) + + or @C[3],@C[4] + mov $A[1][2](%rdi),@C[1] + xor @C[2],@C[4] # C[2] ^ (~C[4] | C[3]) + mov @C[4],$A[1][2](%rsi) # R[1][2] = C[2] ^ (~C[4] | C[3]) + + and @C[2],@C[3] + mov $A[4][0](%rdi),@C[4] + xor @T[1],@C[3] # C[1] ^ (C[3] & C[2]) + mov @C[3],$A[1][1](%rsi) # R[1][1] = C[1] ^ (C[3] & C[2]) + + or @C[2],@T[1] + mov $A[2][3](%rdi),@C[2] + xor @T[0],@T[1] # C[0] ^ (C[1] | C[2]) + mov $A[3][4](%rdi),@C[3] + mov @T[1],$A[1][0](%rsi) # R[1][0] = C[0] ^ (C[1] | C[2]) + + + xor @D[3],@C[2] + xor @D[4],@C[3] + rol \$$rhotates[2][3],@C[2] + xor @D[2],@C[1] + rol \$$rhotates[3][4],@C[3] + xor @D[0],@C[4] + rol \$$rhotates[1][2],@C[1] + xor @D[1],@C[0] + rol \$$rhotates[4][0],@C[4] + mov @C[2],@T[0] + and @C[3],@C[2] + rol \$$rhotates[0][1],@C[0] + + not @C[3] + xor @C[1],@C[2] # C[1] ^ ( C[2] & C[3]) + mov @C[2],$A[2][1](%rsi) # R[2][1] = C[1] ^ ( C[2] & C[3]) + + mov @C[4],@T[1] + and @C[3],@C[4] + mov $A[2][1](%rdi),@C[2] + xor @T[0],@C[4] # C[2] ^ ( C[4] & ~C[3]) + mov @C[4],$A[2][2](%rsi) # R[2][2] = C[2] ^ ( C[4] & ~C[3]) + + or @C[1],@T[0] + mov $A[4][3](%rdi),@C[4] + xor @C[0],@T[0] # C[0] ^ ( C[2] | C[1]) + mov @T[0],$A[2][0](%rsi) # R[2][0] = C[0] ^ ( C[2] | C[1]) + + and @C[0],@C[1] + xor @T[1],@C[1] # C[4] ^ ( C[1] & C[0]) + mov @C[1],$A[2][4](%rsi) # R[2][4] = C[4] ^ ( C[1] & C[0]) + + or @C[0],@T[1] + mov $A[1][0](%rdi),@C[1] + xor @C[3],@T[1] # ~C[3] ^ ( C[0] | C[4]) + mov $A[3][2](%rdi),@C[3] + mov @T[1],$A[2][3](%rsi) # R[2][3] = ~C[3] ^ ( C[0] | C[4]) + + + mov $A[0][4](%rdi),@C[0] + + xor @D[1],@C[2] + xor @D[2],@C[3] + rol \$$rhotates[2][1],@C[2] + xor @D[0],@C[1] + rol \$$rhotates[3][2],@C[3] + xor @D[3],@C[4] + rol \$$rhotates[1][0],@C[1] + xor @D[4],@C[0] + rol \$$rhotates[4][3],@C[4] + mov @C[2],@T[0] + or @C[3],@C[2] + rol \$$rhotates[0][4],@C[0] + + not @C[3] + xor @C[1],@C[2] # C[1] ^ ( C[2] | C[3]) + mov @C[2],$A[3][1](%rsi) # R[3][1] = C[1] ^ ( C[2] | C[3]) + + mov @C[4],@T[1] + or @C[3],@C[4] + xor @T[0],@C[4] # C[2] ^ ( C[4] | ~C[3]) + mov @C[4],$A[3][2](%rsi) # R[3][2] = C[2] ^ ( C[4] | ~C[3]) + + and @C[1],@T[0] + xor @C[0],@T[0] # C[0] ^ ( C[2] & C[1]) + mov @T[0],$A[3][0](%rsi) # R[3][0] = C[0] ^ ( C[2] & C[1]) + + or @C[0],@C[1] + xor @T[1],@C[1] # C[4] ^ ( C[1] | C[0]) + mov @C[1],$A[3][4](%rsi) # R[3][4] = C[4] ^ ( C[1] | C[0]) + + and @T[1],@C[0] + xor @C[3],@C[0] # ~C[3] ^ ( C[0] & C[4]) + mov @C[0],$A[3][3](%rsi) # R[3][3] = ~C[3] ^ ( C[0] & C[4]) + + + xor $A[0][2](%rdi),@D[2] + xor $A[1][3](%rdi),@D[3] + rol \$$rhotates[0][2],@D[2] + xor $A[4][1](%rdi),@D[1] + rol \$$rhotates[1][3],@D[3] + xor $A[2][4](%rdi),@D[4] + rol \$$rhotates[4][1],@D[1] + xor $A[3][0](%rdi),@D[0] + xchg %rsi,%rdi + rol \$$rhotates[2][4],@D[4] + rol \$$rhotates[3][0],@D[0] +___ + @C = @D[2..4,0,1]; +$code.=<<___; + mov @C[0],@T[0] + and @C[1],@C[0] + not @C[1] + xor @C[4],@C[0] # C[4] ^ ( C[0] & C[1]) + mov @C[0],$A[4][4](%rdi) # R[4][4] = C[4] ^ ( C[0] & C[1]) + + mov @C[2],@T[1] + and @C[1],@C[2] + xor @T[0],@C[2] # C[0] ^ ( C[2] & ~C[1]) + mov @C[2],$A[4][0](%rdi) # R[4][0] = C[0] ^ ( C[2] & ~C[1]) + + or @C[4],@T[0] + xor @C[3],@T[0] # C[3] ^ ( C[0] | C[4]) + mov @T[0],$A[4][3](%rdi) # R[4][3] = C[3] ^ ( C[0] | C[4]) + + and @C[3],@C[4] + xor @T[1],@C[4] # C[2] ^ ( C[4] & C[3]) + mov @C[4],$A[4][2](%rdi) # R[4][2] = C[2] ^ ( C[4] & C[3]) + + or @T[1],@C[3] + xor @C[1],@C[3] # ~C[1] ^ ( C[2] | C[3]) + mov @C[3],$A[4][1](%rdi) # R[4][1] = ~C[1] ^ ( C[2] | C[3]) + + mov @C[0],@C[1] # harmonize with the loop top + mov @T[0],@C[0] + + test \$255,$iotas + jnz .Loop + + lea -192($iotas),$iotas # rewind iotas + ret +.size __KeccakF1600,.-__KeccakF1600 + +.globl KeccakF1600 +.type KeccakF1600,\@function,1,"unwind" +.align 32 +KeccakF1600: +.cfi_startproc + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + + lea 100(%rdi),%rdi # size optimization + sub \$200,%rsp +.cfi_alloca 200 +.cfi_end_prologue + + notq $A[0][1](%rdi) + notq $A[0][2](%rdi) + notq $A[1][3](%rdi) + notq $A[2][2](%rdi) + notq $A[3][2](%rdi) + notq $A[4][0](%rdi) + + lea iotas(%rip),$iotas + lea 100(%rsp),%rsi # size optimization + + call __KeccakF1600 + + notq $A[0][1](%rdi) + notq $A[0][2](%rdi) + notq $A[1][3](%rdi) + notq $A[2][2](%rdi) + notq $A[3][2](%rdi) + notq $A[4][0](%rdi) + lea -100(%rdi),%rdi # preserve A[][] + + lea 248(%rsp),%r11 +.cfi_def_cfa %r11,8 + mov -48(%r11),%r15 + mov -40(%r11),%r14 + mov -32(%r11),%r13 + mov -24(%r11),%r12 + mov -16(%r11),%rbp + mov -8(%r11),%rbx + lea (%r11),%rsp +.cfi_epilogue + ret +.cfi_endproc +.size KeccakF1600,.-KeccakF1600 +___ + +{ my ($A_flat,$inp,$len,$bsz) = ("%rdi","%rsi","%rdx","%rcx"); + ($A_flat,$inp) = ("%r8","%r9"); +$code.=<<___; +.globl SHA3_absorb +.type SHA3_absorb,\@function,4,"unwind" +.align 32 +SHA3_absorb: +.cfi_startproc + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + + lea 100(%rdi),%rdi # size optimization + sub \$232,%rsp +.cfi_alloca 232 +.cfi_end_prologue + + mov %rsi,$inp + lea 100(%rsp),%rsi # size optimization + + notq $A[0][1](%rdi) + notq $A[0][2](%rdi) + notq $A[1][3](%rdi) + notq $A[2][2](%rdi) + notq $A[3][2](%rdi) + notq $A[4][0](%rdi) + lea iotas(%rip),$iotas + + mov $bsz,216-100(%rsi) # save bsz + +.Loop_absorb: + cmp $bsz,$len + jc .Ldone_absorb + + shr \$3,$bsz + lea -100(%rdi),$A_flat + +.Lblock_absorb: + mov ($inp),%rax + lea 8($inp),$inp + xor ($A_flat),%rax + lea 8($A_flat),$A_flat + sub \$8,$len + mov %rax,-8($A_flat) + sub \$1,$bsz + jnz .Lblock_absorb + + mov $inp,200-100(%rsi) # save inp + mov $len,208-100(%rsi) # save len + call __KeccakF1600 + mov 200-100(%rsi),$inp # pull inp + mov 208-100(%rsi),$len # pull len + mov 216-100(%rsi),$bsz # pull bsz + jmp .Loop_absorb + +.align 32 +.Ldone_absorb: + mov $len,%rax # return value + + notq $A[0][1](%rdi) + notq $A[0][2](%rdi) + notq $A[1][3](%rdi) + notq $A[2][2](%rdi) + notq $A[3][2](%rdi) + notq $A[4][0](%rdi) + + lea 280(%rsp),%r11 +.cfi_def_cfa %r11,8 + mov -48(%r11),%r15 + mov -40(%r11),%r14 + mov -32(%r11),%r13 + mov -24(%r11),%r12 + mov -16(%r11),%rbp + mov -8(%r11),%rbx + lea (%r11),%rsp +.cfi_epilogue + ret +.cfi_endproc +.size SHA3_absorb,.-SHA3_absorb +___ +} +{ my ($A_flat,$out,$len,$bsz) = ("%rdi","%rsi","%rdx","%rcx"); + ($out,$len,$bsz) = ("%r12","%r13","%r14"); + +$code.=<<___; +.globl SHA3_squeeze +.type SHA3_squeeze,\@function,4,"unwind" +.align 32 +SHA3_squeeze: +.cfi_startproc + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + sub \$32,%rsp # Windows thing +.cfi_alloca 32 +.cfi_end_prologue + + shr \$3,%rcx + mov $A_flat,%r8 + mov %rsi,$out + mov %rdx,$len + mov %rcx,$bsz + jmp .Loop_squeeze + +.align 32 +.Loop_squeeze: + cmp \$8,$len + jb .Ltail_squeeze + + mov (%r8),%rax + lea 8(%r8),%r8 + mov %rax,($out) + lea 8($out),$out + sub \$8,$len # len -= 8 + jz .Ldone_squeeze + + sub \$1,%rcx # bsz-- + jnz .Loop_squeeze + + mov %rdi,%rcx # Windows thing + call KeccakF1600 + mov $A_flat,%r8 + mov $bsz,%rcx + jmp .Loop_squeeze + +.Ltail_squeeze: + mov %r8, %rsi + mov $out,%rdi + mov $len,%rcx + .byte 0xf3,0xa4 # rep movsb + +.Ldone_squeeze: + mov 32(%rsp),%r14 + mov 40(%rsp),%r13 + mov 48(%rsp),%r12 + add \$56,%rsp +.cfi_alloca -56 +.cfi_epilogue + ret +.cfi_endproc +.size SHA3_squeeze,.-SHA3_squeeze +___ +} +$code.=<<___; +.align 256 + .quad 0,0,0,0,0,0,0,0 +.type iotas,\@object +iotas: + .quad 0x0000000000000001 + .quad 0x0000000000008082 + .quad 0x800000000000808a + .quad 0x8000000080008000 + .quad 0x000000000000808b + .quad 0x0000000080000001 + .quad 0x8000000080008081 + .quad 0x8000000000008009 + .quad 0x000000000000008a + .quad 0x0000000000000088 + .quad 0x0000000080008009 + .quad 0x000000008000000a + .quad 0x000000008000808b + .quad 0x800000000000008b + .quad 0x8000000000008089 + .quad 0x8000000000008003 + .quad 0x8000000000008002 + .quad 0x8000000000000080 + .quad 0x000000000000800a + .quad 0x800000008000000a + .quad 0x8000000080008081 + .quad 0x8000000000008080 + .quad 0x0000000080000001 + .quad 0x8000000080008008 +.size iotas,.-iotas +.asciz "Keccak-1600 absorb and squeeze for x86_64, CRYPTOGAMS by " +___ + +foreach (split("\n",$code)) { + # Below replacement results in 11.2 on Sandy Bridge, 9.4 on + # Haswell, but it hurts other processors by up to 2-3-4x... + #s/rol\s+(\$[0-9]+),(%[a-z][a-z0-9]+)/shld\t$1,$2,$2/; + + # Below replacement results in 9.3 on Haswell [as well as + # on Ryzen, i.e. it *hurts* Ryzen]... + #s/rol\s+\$([0-9]+),(%[a-z][a-z0-9]+)/rorx\t\$64-$1,$2,$2/; + + print $_, "\n"; +} + +close STDOUT; diff --git a/cbits/asm/poly1305-armv8-ios64.S b/cbits/asm/poly1305-armv8-ios64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-armv8-ios64.S @@ -0,0 +1,844 @@ +#ifndef __KERNEL__ +# include "arm_arch.h" + +#endif + +.text + +// forward "declarations" are required for Apple +.globl _crypton_poly1305_asm_blocks +.globl _crypton_poly1305_asm_emit + +.globl _crypton_poly1305_asm_init + +.align 5 +_crypton_poly1305_asm_init: + cmp x1,xzr + stp xzr,xzr,[x0] // zero hash value + stp xzr,xzr,[x0,#16] // [along with is_base2_26] + + csel x0,xzr,x0,eq + b.eq Lno_key + +#ifndef __KERNEL__ + adrp x17,_crypton_armcap_P@PAGE + ldr w17,[x17,_crypton_armcap_P@PAGEOFF] +#endif + + ldp x7,x8,[x1] // load key + mov x9,#0xfffffffc0fffffff + movk x9,#0x0fff,lsl#48 +#ifdef __AARCH64EB__ + rev x7,x7 // flip bytes + rev x8,x8 +#endif + and x7,x7,x9 // &=0ffffffc0fffffff + and x9,x9,#-4 + and x8,x8,x9 // &=0ffffffc0ffffffc + mov w9,#-1 + stp x7,x8,[x0,#32] // save key value + str w9,[x0,#48] // impossible key power value + +#ifndef __KERNEL__ + tst w17,#ARMV7_NEON + + adr x13,Lcrypton_poly1305_asm_blocks + adr x15,Lcrypton_poly1305_asm_blocks_neon + adr x14,Lcrypton_poly1305_asm_emit + + csel x13,x13,x15,eq +# ifdef __CHERI_PURE_CAPABILITY__ + add x13, x13, #1 + add x14, x14, #1 + seal x13, x13, rb + seal x14, x14, rb +# endif + +# ifdef __ILP32__ + stp w13,w14,[x2] +# else + stp x13,x14,[x2] +# endif + mov x0,#1 +#else + mov x0,#0 +#endif +Lno_key: + ret + + + +.align 5 +_crypton_poly1305_asm_blocks: +Lcrypton_poly1305_asm_blocks: + ands x2,x2,#-16 + b.eq Lno_data + + ldp x4,x5,[x0] // load hash value + ldp x6,x17,[x0,#16] // [along with is_base2_26] + ldp x7,x8,[x0,#32] // load key value + +#ifdef __AARCH64EB__ + lsr x12,x4,#32 + mov w13,w4 + lsr x14,x5,#32 + mov w15,w5 + lsr x16,x6,#32 +#else + mov w12,w4 + lsr x13,x4,#32 + mov w14,w5 + lsr x15,x5,#32 + mov w16,w6 +#endif + + add x12,x12,x13,lsl#26 // base 2^26 -> base 2^64 + lsr x13,x14,#12 + adds x12,x12,x14,lsl#52 + add x13,x13,x15,lsl#14 + adc x13,x13,xzr + lsr x14,x16,#24 + adds x13,x13,x16,lsl#40 + adc x14,x14,xzr + + cmp x17,#0 // is_base2_26? + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) + csel x4,x4,x12,eq // choose between radixes + csel x5,x5,x13,eq + csel x6,x6,x14,eq + +Loop: + ldp x10,x11,[x1],#16 // load input + sub x2,x2,#16 +#ifdef __AARCH64EB__ + rev x10,x10 + rev x11,x11 +#endif + adds x4,x4,x10 // accumulate input + adcs x5,x5,x11 + + mul x12,x4,x7 // h0*r0 + adc x6,x6,x3 + umulh x13,x4,x7 + + mul x10,x5,x9 // h1*5*r1 + umulh x11,x5,x9 + + adds x12,x12,x10 + mul x10,x4,x8 // h0*r1 + adc x13,x13,x11 + umulh x14,x4,x8 + + adds x13,x13,x10 + mul x10,x5,x7 // h1*r0 + adc x14,x14,xzr + umulh x11,x5,x7 + + adds x13,x13,x10 + mul x10,x6,x9 // h2*5*r1 + adc x14,x14,x11 + mul x11,x6,x7 // h2*r0 + + adds x13,x13,x10 + adc x14,x14,x11 + + and x10,x14,#-4 // final reduction + and x6,x14,#3 + add x10,x10,x14,lsr#2 + adds x4,x12,x10 + adcs x5,x13,xzr + adc x6,x6,xzr + + cbnz x2,Loop + + stp x4,x5,[x0] // store hash value + stp x6,xzr,[x0,#16] // [and clear is_base2_26] + +Lno_data: + ret + + + +.align 5 +_crypton_poly1305_asm_emit: +Lcrypton_poly1305_asm_emit: + ldp x4,x5,[x0] // load hash base 2^64 + ldp x6,x7,[x0,#16] // [along with is_base2_26] + ldp x10,x11,[x2] // load nonce + +#ifdef __AARCH64EB__ + lsr x12,x4,#32 + mov w13,w4 + lsr x14,x5,#32 + mov w15,w5 + lsr x16,x6,#32 +#else + mov w12,w4 + lsr x13,x4,#32 + mov w14,w5 + lsr x15,x5,#32 + mov w16,w6 +#endif + + add x12,x12,x13,lsl#26 // base 2^26 -> base 2^64 + lsr x13,x14,#12 + adds x12,x12,x14,lsl#52 + add x13,x13,x15,lsl#14 + adc x13,x13,xzr + lsr x14,x16,#24 + adds x13,x13,x16,lsl#40 + adc x14,x14,xzr + + cmp x7,#0 // is_base2_26? + csel x4,x4,x12,eq // choose between radixes + csel x5,x5,x13,eq + csel x6,x6,x14,eq + + adds x12,x4,#5 // compare to modulus + adcs x13,x5,xzr + adc x14,x6,xzr + + tst x14,#-4 // see if it's carried/borrowed + + csel x4,x4,x12,eq + csel x5,x5,x13,eq + +#ifdef __AARCH64EB__ + ror x10,x10,#32 // flip nonce words + ror x11,x11,#32 +#endif + adds x4,x4,x10 // accumulate nonce + adc x5,x5,x11 +#ifdef __AARCH64EB__ + rev x4,x4 // flip output bytes + rev x5,x5 +#endif + stp x4,x5,[x1] // write result + + ret + + +.align 5 +crypton_poly1305_asm_mult: + mul x12,x4,x7 // h0*r0 + umulh x13,x4,x7 + + mul x10,x5,x9 // h1*5*r1 + umulh x11,x5,x9 + + adds x12,x12,x10 + mul x10,x4,x8 // h0*r1 + adc x13,x13,x11 + umulh x14,x4,x8 + + adds x13,x13,x10 + mul x10,x5,x7 // h1*r0 + adc x14,x14,xzr + umulh x11,x5,x7 + + adds x13,x13,x10 + mul x10,x6,x9 // h2*5*r1 + adc x14,x14,x11 + mul x11,x6,x7 // h2*r0 + + adds x13,x13,x10 + adc x14,x14,x11 + + and x10,x14,#-4 // final reduction + and x6,x14,#3 + add x10,x10,x14,lsr#2 + adds x4,x12,x10 + adcs x5,x13,xzr + adc x6,x6,xzr + + ret + + + +.align 4 +crypton_poly1305_asm_splat: + and x12,x4,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x13,x4,#26,#26 + extr x14,x5,x4,#52 + and x14,x14,#0x03ffffff + ubfx x15,x5,#14,#26 + extr x16,x6,x5,#40 + + str w12,[x0,#16*0] // r0 + add w12,w13,w13,lsl#2 // r1*5 + str w13,[x0,#16*1] // r1 + add w13,w14,w14,lsl#2 // r2*5 + str w12,[x0,#16*2] // s1 + str w14,[x0,#16*3] // r2 + add w14,w15,w15,lsl#2 // r3*5 + str w13,[x0,#16*4] // s2 + str w15,[x0,#16*5] // r3 + add w15,w16,w16,lsl#2 // r4*5 + str w14,[x0,#16*6] // s3 + str w16,[x0,#16*7] // r4 + str w15,[x0,#16*8] // s4 + + ret + + +#ifdef __KERNEL__ +.globl _crypton_poly1305_asm_blocks_neon +#endif + +.align 5 +_crypton_poly1305_asm_blocks_neon: +Lcrypton_poly1305_asm_blocks_neon: + ldr x17,[x0,#24] + cmp x2,#128 + b.lo Lcrypton_poly1305_asm_blocks + +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]! + add x29,sp,#0 + + stp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // meet ABI requirements + stp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + + cbz x17,Lbase2_64_neon + + ldp w10,w11,[x0] // load hash value base 2^26 + ldp w12,w13,[x0,#8] + ldr w14,[x0,#16] + + tst x2,#31 + b.eq Leven_neon + + ldp x7,x8,[x0,#32] // load key value + + add x4,x10,x11,lsl#26 // base 2^26 -> base 2^64 + lsr x5,x12,#12 + adds x4,x4,x12,lsl#52 + add x5,x5,x13,lsl#14 + adc x5,x5,xzr + lsr x6,x14,#24 + adds x5,x5,x14,lsl#40 + adc x14,x6,xzr // can be partially reduced... + + ldp x12,x13,[x1],#16 // load input + sub x2,x2,#16 + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) + +#ifdef __AARCH64EB__ + rev x12,x12 + rev x13,x13 +#endif + adds x4,x4,x12 // accumulate input + adcs x5,x5,x13 + adc x6,x6,x3 + + bl crypton_poly1305_asm_mult + + and x10,x4,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x11,x4,#26,#26 + extr x12,x5,x4,#52 + and x12,x12,#0x03ffffff + ubfx x13,x5,#14,#26 + extr x14,x6,x5,#40 + + b Leven_neon + +.align 4 +Lbase2_64_neon: + ldp x7,x8,[x0,#32] // load key value + + ldp x4,x5,[x0] // load hash value base 2^64 + ldr x6,[x0,#16] + + tst x2,#31 + b.eq Linit_neon + + ldp x12,x13,[x1],#16 // load input + sub x2,x2,#16 + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) +#ifdef __AARCH64EB__ + rev x12,x12 + rev x13,x13 +#endif + adds x4,x4,x12 // accumulate input + adcs x5,x5,x13 + adc x6,x6,x3 + + bl crypton_poly1305_asm_mult + +Linit_neon: + ldr w17,[x0,#48] // first table element + and x10,x4,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x11,x4,#26,#26 + extr x12,x5,x4,#52 + and x12,x12,#0x03ffffff + ubfx x13,x5,#14,#26 + extr x14,x6,x5,#40 + + cmp w17,#-1 // is value impossible? + b.ne Leven_neon + + fmov d24,x10 + fmov d25,x11 + fmov d26,x12 + fmov d27,x13 + fmov d28,x14 + + ////////////////////////////////// initialize r^n table + mov x4,x7 // r^1 + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) + mov x5,x8 + mov x6,xzr + add x0,x0,#48+12 + bl crypton_poly1305_asm_splat + + bl crypton_poly1305_asm_mult // r^2 + sub x0,x0,#4 + bl crypton_poly1305_asm_splat + + bl crypton_poly1305_asm_mult // r^3 + sub x0,x0,#4 + bl crypton_poly1305_asm_splat + + bl crypton_poly1305_asm_mult // r^4 + sub x0,x0,#4 + bl crypton_poly1305_asm_splat + sub x0,x0,#48 + b Ldo_neon + +.align 4 +Leven_neon: + fmov d24,x10 + fmov d25,x11 + fmov d26,x12 + fmov d27,x13 + fmov d28,x14 + +Ldo_neon: + ldp x8,x12,[x1,#32] // inp[2:3] + subs x2,x2,#64 + ldp x9,x13,[x1,#48] + add x16,x1,#96 + adr x17,Lzeros + + lsl x3,x3,#24 + add x15,x0,#48 + +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + and x5,x9,#0x03ffffff + ubfx x6,x8,#26,#26 + ubfx x7,x9,#26,#26 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + extr x8,x12,x8,#52 + extr x9,x13,x9,#52 + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + fmov d14,x4 + and x8,x8,#0x03ffffff + and x9,x9,#0x03ffffff + ubfx x10,x12,#14,#26 + ubfx x11,x13,#14,#26 + add x12,x3,x12,lsr#40 + add x13,x3,x13,lsr#40 + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + fmov d15,x6 + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + fmov d16,x8 + fmov d17,x10 + fmov d18,x12 + + ldp x8,x12,[x1],#16 // inp[0:1] + ldp x9,x13,[x1],#48 + + ld1 {v0.4s,v1.4s,v2.4s,v3.4s},[x15],#64 + ld1 {v4.4s,v5.4s,v6.4s,v7.4s},[x15],#64 + ld1 {v8.4s},[x15] + +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + and x5,x9,#0x03ffffff + ubfx x6,x8,#26,#26 + ubfx x7,x9,#26,#26 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + extr x8,x12,x8,#52 + extr x9,x13,x9,#52 + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + fmov d9,x4 + and x8,x8,#0x03ffffff + and x9,x9,#0x03ffffff + ubfx x10,x12,#14,#26 + ubfx x11,x13,#14,#26 + add x12,x3,x12,lsr#40 + add x13,x3,x13,lsr#40 + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + fmov d10,x6 + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + movi v31.2d,#-1 + fmov d11,x8 + fmov d12,x10 + fmov d13,x12 + ushr v31.2d,v31.2d,#38 + + b.ls Lskip_loop + +.align 4 +Loop_neon: + //////////////////////////////////////////////////////////////// + // ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2 + // ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r + // ___________________/ + // ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2 + // ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r + // ___________________/ ____________________/ + // + // Note that we start with inp[2:3]*r^2. This is because it + // doesn't depend on reduction in previous iteration. + //////////////////////////////////////////////////////////////// + // d4 = h0*r4 + h1*r3 + h2*r2 + h3*r1 + h4*r0 + // d3 = h0*r3 + h1*r2 + h2*r1 + h3*r0 + h4*5*r4 + // d2 = h0*r2 + h1*r1 + h2*r0 + h3*5*r4 + h4*5*r3 + // d1 = h0*r1 + h1*r0 + h2*5*r4 + h3*5*r3 + h4*5*r2 + // d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1 + + subs x2,x2,#64 + umull v23.2d,v14.2s,v7.s[2] + csel x16,x17,x16,lo + umull v22.2d,v14.2s,v5.s[2] + umull v21.2d,v14.2s,v3.s[2] + ldp x8,x12,[x16],#16 // inp[2:3] (or zero) + umull v20.2d,v14.2s,v1.s[2] + ldp x9,x13,[x16],#48 + umull v19.2d,v14.2s,v0.s[2] +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + + umlal v23.2d,v15.2s,v5.s[2] + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + umlal v22.2d,v15.2s,v3.s[2] + and x5,x9,#0x03ffffff + umlal v21.2d,v15.2s,v1.s[2] + ubfx x6,x8,#26,#26 + umlal v20.2d,v15.2s,v0.s[2] + ubfx x7,x9,#26,#26 + umlal v19.2d,v15.2s,v8.s[2] + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + + umlal v23.2d,v16.2s,v3.s[2] + extr x8,x12,x8,#52 + umlal v22.2d,v16.2s,v1.s[2] + extr x9,x13,x9,#52 + umlal v21.2d,v16.2s,v0.s[2] + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + umlal v20.2d,v16.2s,v8.s[2] + fmov d14,x4 + umlal v19.2d,v16.2s,v6.s[2] + and x8,x8,#0x03ffffff + + umlal v23.2d,v17.2s,v1.s[2] + and x9,x9,#0x03ffffff + umlal v22.2d,v17.2s,v0.s[2] + ubfx x10,x12,#14,#26 + umlal v21.2d,v17.2s,v8.s[2] + ubfx x11,x13,#14,#26 + umlal v20.2d,v17.2s,v6.s[2] + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + umlal v19.2d,v17.2s,v4.s[2] + fmov d15,x6 + + add v11.2s,v11.2s,v26.2s + add x12,x3,x12,lsr#40 + umlal v23.2d,v18.2s,v0.s[2] + add x13,x3,x13,lsr#40 + umlal v22.2d,v18.2s,v8.s[2] + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + umlal v21.2d,v18.2s,v6.s[2] + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + umlal v20.2d,v18.2s,v4.s[2] + fmov d16,x8 + umlal v19.2d,v18.2s,v2.s[2] + fmov d17,x10 + + //////////////////////////////////////////////////////////////// + // (hash+inp[0:1])*r^4 and accumulate + + add v9.2s,v9.2s,v24.2s + fmov d18,x12 + umlal v22.2d,v11.2s,v1.s[0] + ldp x8,x12,[x1],#16 // inp[0:1] + umlal v19.2d,v11.2s,v6.s[0] + ldp x9,x13,[x1],#48 + umlal v23.2d,v11.2s,v3.s[0] + umlal v20.2d,v11.2s,v8.s[0] + umlal v21.2d,v11.2s,v0.s[0] +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + + add v10.2s,v10.2s,v25.2s + umlal v22.2d,v9.2s,v5.s[0] + umlal v23.2d,v9.2s,v7.s[0] + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + umlal v21.2d,v9.2s,v3.s[0] + and x5,x9,#0x03ffffff + umlal v19.2d,v9.2s,v0.s[0] + ubfx x6,x8,#26,#26 + umlal v20.2d,v9.2s,v1.s[0] + ubfx x7,x9,#26,#26 + + add v12.2s,v12.2s,v27.2s + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + umlal v22.2d,v10.2s,v3.s[0] + extr x8,x12,x8,#52 + umlal v23.2d,v10.2s,v5.s[0] + extr x9,x13,x9,#52 + umlal v19.2d,v10.2s,v8.s[0] + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + umlal v21.2d,v10.2s,v1.s[0] + fmov d9,x4 + umlal v20.2d,v10.2s,v0.s[0] + and x8,x8,#0x03ffffff + + add v13.2s,v13.2s,v28.2s + and x9,x9,#0x03ffffff + umlal v22.2d,v12.2s,v0.s[0] + ubfx x10,x12,#14,#26 + umlal v19.2d,v12.2s,v4.s[0] + ubfx x11,x13,#14,#26 + umlal v23.2d,v12.2s,v1.s[0] + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + umlal v20.2d,v12.2s,v6.s[0] + fmov d10,x6 + umlal v21.2d,v12.2s,v8.s[0] + add x12,x3,x12,lsr#40 + + umlal v22.2d,v13.2s,v8.s[0] + add x13,x3,x13,lsr#40 + umlal v19.2d,v13.2s,v2.s[0] + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + umlal v23.2d,v13.2s,v0.s[0] + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + umlal v20.2d,v13.2s,v4.s[0] + fmov d11,x8 + umlal v21.2d,v13.2s,v6.s[0] + fmov d12,x10 + fmov d13,x12 + + ///////////////////////////////////////////////////////////////// + // lazy reduction as discussed in "NEON crypto" by D.J. Bernstein + // and P. Schwabe + // + // [see discussion in poly1305-armv4 module] + + ushr v29.2d,v22.2d,#26 + xtn v27.2s,v22.2d + ushr v30.2d,v19.2d,#26 + and v19.16b,v19.16b,v31.16b + add v23.2d,v23.2d,v29.2d // h3 -> h4 + bic v27.2s,#0xfc,lsl#24 // &=0x03ffffff + add v20.2d,v20.2d,v30.2d // h0 -> h1 + + ushr v29.2d,v23.2d,#26 + xtn v28.2s,v23.2d + ushr v30.2d,v20.2d,#26 + xtn v25.2s,v20.2d + bic v28.2s,#0xfc,lsl#24 + add v21.2d,v21.2d,v30.2d // h1 -> h2 + + add v19.2d,v19.2d,v29.2d + shl v29.2d,v29.2d,#2 + shrn v30.2s,v21.2d,#26 + xtn v26.2s,v21.2d + add v19.2d,v19.2d,v29.2d // h4 -> h0 + bic v25.2s,#0xfc,lsl#24 + add v27.2s,v27.2s,v30.2s // h2 -> h3 + bic v26.2s,#0xfc,lsl#24 + + shrn v29.2s,v19.2d,#26 + xtn v24.2s,v19.2d + ushr v30.2s,v27.2s,#26 + bic v27.2s,#0xfc,lsl#24 + bic v24.2s,#0xfc,lsl#24 + add v25.2s,v25.2s,v29.2s // h0 -> h1 + add v28.2s,v28.2s,v30.2s // h3 -> h4 + + b.hi Loop_neon + +Lskip_loop: + dup v16.2d,v16.d[0] + add v11.2s,v11.2s,v26.2s + + //////////////////////////////////////////////////////////////// + // multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1 + + adds x2,x2,#32 + b.ne Long_tail + + dup v16.2d,v11.d[0] + add v14.2s,v9.2s,v24.2s + add v17.2s,v12.2s,v27.2s + add v15.2s,v10.2s,v25.2s + add v18.2s,v13.2s,v28.2s + +Long_tail: + dup v14.2d,v14.d[0] + umull2 v19.2d,v16.4s,v6.4s + umull2 v22.2d,v16.4s,v1.4s + umull2 v23.2d,v16.4s,v3.4s + umull2 v21.2d,v16.4s,v0.4s + umull2 v20.2d,v16.4s,v8.4s + + dup v15.2d,v15.d[0] + umlal2 v19.2d,v14.4s,v0.4s + umlal2 v21.2d,v14.4s,v3.4s + umlal2 v22.2d,v14.4s,v5.4s + umlal2 v23.2d,v14.4s,v7.4s + umlal2 v20.2d,v14.4s,v1.4s + + dup v17.2d,v17.d[0] + umlal2 v19.2d,v15.4s,v8.4s + umlal2 v22.2d,v15.4s,v3.4s + umlal2 v21.2d,v15.4s,v1.4s + umlal2 v23.2d,v15.4s,v5.4s + umlal2 v20.2d,v15.4s,v0.4s + + dup v18.2d,v18.d[0] + umlal2 v22.2d,v17.4s,v0.4s + umlal2 v23.2d,v17.4s,v1.4s + umlal2 v19.2d,v17.4s,v4.4s + umlal2 v20.2d,v17.4s,v6.4s + umlal2 v21.2d,v17.4s,v8.4s + + umlal2 v22.2d,v18.4s,v8.4s + umlal2 v19.2d,v18.4s,v2.4s + umlal2 v23.2d,v18.4s,v0.4s + umlal2 v20.2d,v18.4s,v4.4s + umlal2 v21.2d,v18.4s,v6.4s + + b.eq Lshort_tail + + //////////////////////////////////////////////////////////////// + // (hash+inp[0:1])*r^4:r^3 and accumulate + + add v9.2s,v9.2s,v24.2s + umlal v22.2d,v11.2s,v1.2s + umlal v19.2d,v11.2s,v6.2s + umlal v23.2d,v11.2s,v3.2s + umlal v20.2d,v11.2s,v8.2s + umlal v21.2d,v11.2s,v0.2s + + add v10.2s,v10.2s,v25.2s + umlal v22.2d,v9.2s,v5.2s + umlal v19.2d,v9.2s,v0.2s + umlal v23.2d,v9.2s,v7.2s + umlal v20.2d,v9.2s,v1.2s + umlal v21.2d,v9.2s,v3.2s + + add v12.2s,v12.2s,v27.2s + umlal v22.2d,v10.2s,v3.2s + umlal v19.2d,v10.2s,v8.2s + umlal v23.2d,v10.2s,v5.2s + umlal v20.2d,v10.2s,v0.2s + umlal v21.2d,v10.2s,v1.2s + + add v13.2s,v13.2s,v28.2s + umlal v22.2d,v12.2s,v0.2s + umlal v19.2d,v12.2s,v4.2s + umlal v23.2d,v12.2s,v1.2s + umlal v20.2d,v12.2s,v6.2s + umlal v21.2d,v12.2s,v8.2s + + umlal v22.2d,v13.2s,v8.2s + umlal v19.2d,v13.2s,v2.2s + umlal v23.2d,v13.2s,v0.2s + umlal v20.2d,v13.2s,v4.2s + umlal v21.2d,v13.2s,v6.2s + +Lshort_tail: + //////////////////////////////////////////////////////////////// + // horizontal add + + addp v22.2d,v22.2d,v22.2d + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // meet ABI requirements + addp v19.2d,v19.2d,v19.2d + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + addp v23.2d,v23.2d,v23.2d + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + addp v20.2d,v20.2d,v20.2d + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + addp v21.2d,v21.2d,v21.2d + ldr x30,[sp,#__SIZEOF_POINTER__] + + //////////////////////////////////////////////////////////////// + // lazy reduction, but without narrowing + + ushr v29.2d,v22.2d,#26 + and v22.16b,v22.16b,v31.16b + ushr v30.2d,v19.2d,#26 + and v19.16b,v19.16b,v31.16b + + add v23.2d,v23.2d,v29.2d // h3 -> h4 + add v20.2d,v20.2d,v30.2d // h0 -> h1 + + ushr v29.2d,v23.2d,#26 + and v23.16b,v23.16b,v31.16b + ushr v30.2d,v20.2d,#26 + and v20.16b,v20.16b,v31.16b + add v21.2d,v21.2d,v30.2d // h1 -> h2 + + add v19.2d,v19.2d,v29.2d + shl v29.2d,v29.2d,#2 + ushr v30.2d,v21.2d,#26 + and v21.16b,v21.16b,v31.16b + add v19.2d,v19.2d,v29.2d // h4 -> h0 + add v22.2d,v22.2d,v30.2d // h2 -> h3 + + ushr v29.2d,v19.2d,#26 + and v19.16b,v19.16b,v31.16b + ushr v30.2d,v22.2d,#26 + and v22.16b,v22.16b,v31.16b + add v20.2d,v20.2d,v29.2d // h0 -> h1 + add v23.2d,v23.2d,v30.2d // h3 -> h4 + + //////////////////////////////////////////////////////////////// + // write the result, can be partially reduced + + st4 {v19.s,v20.s,v21.s,v22.s}[0],[x0],#16 + mov x4,#1 + st1 {v23.s}[0],[x0] + str x4,[x0,#8] // set is_base2_26 + + ldr x29,[sp],#2*__SIZEOF_POINTER__+64 +.long 0xd50323bf // autiasp + ret + + +.align 5 +Lzeros: +.long 0,0,0,0,0,0,0,0 +.byte 80,111,108,121,49,51,48,53,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 +.align 2 +#if !defined(__KERNEL__) && !defined(_WIN64) +.comm __crypton_armcap_P,4 +.private_extern _crypton_armcap_P +#endif diff --git a/cbits/asm/poly1305-armv8-linux64.S b/cbits/asm/poly1305-armv8-linux64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-armv8-linux64.S @@ -0,0 +1,846 @@ +#ifndef __KERNEL__ +# include "arm_arch.h" + +#endif + +.text + +// forward "declarations" are required for Apple +.globl crypton_poly1305_asm_blocks +.globl crypton_poly1305_asm_emit + +.globl crypton_poly1305_asm_init +.type crypton_poly1305_asm_init,%function +.align 5 +crypton_poly1305_asm_init: + cmp x1,xzr + stp xzr,xzr,[x0] // zero hash value + stp xzr,xzr,[x0,#16] // [along with is_base2_26] + + csel x0,xzr,x0,eq + b.eq .Lno_key + +#ifndef __KERNEL__ + adrp x17,crypton_armcap_P + ldr w17,[x17,#:lo12:crypton_armcap_P] +#endif + + ldp x7,x8,[x1] // load key + mov x9,#0xfffffffc0fffffff + movk x9,#0x0fff,lsl#48 +#ifdef __AARCH64EB__ + rev x7,x7 // flip bytes + rev x8,x8 +#endif + and x7,x7,x9 // &=0ffffffc0fffffff + and x9,x9,#-4 + and x8,x8,x9 // &=0ffffffc0ffffffc + mov w9,#-1 + stp x7,x8,[x0,#32] // save key value + str w9,[x0,#48] // impossible key power value + +#ifndef __KERNEL__ + tst w17,#ARMV7_NEON + + adr x13,.Lcrypton_poly1305_asm_blocks + adr x15,.Lcrypton_poly1305_asm_blocks_neon + adr x14,.Lcrypton_poly1305_asm_emit + + csel x13,x13,x15,eq +# ifdef __CHERI_PURE_CAPABILITY__ + add x13, x13, #1 + add x14, x14, #1 + seal x13, x13, rb + seal x14, x14, rb +# endif + +# ifdef __ILP32__ + stp w13,w14,[x2] +# else + stp x13,x14,[x2] +# endif + mov x0,#1 +#else + mov x0,#0 +#endif +.Lno_key: + ret +.size crypton_poly1305_asm_init,.-crypton_poly1305_asm_init + +.type crypton_poly1305_asm_blocks,%function +.align 5 +crypton_poly1305_asm_blocks: +.Lcrypton_poly1305_asm_blocks: + ands x2,x2,#-16 + b.eq .Lno_data + + ldp x4,x5,[x0] // load hash value + ldp x6,x17,[x0,#16] // [along with is_base2_26] + ldp x7,x8,[x0,#32] // load key value + +#ifdef __AARCH64EB__ + lsr x12,x4,#32 + mov w13,w4 + lsr x14,x5,#32 + mov w15,w5 + lsr x16,x6,#32 +#else + mov w12,w4 + lsr x13,x4,#32 + mov w14,w5 + lsr x15,x5,#32 + mov w16,w6 +#endif + + add x12,x12,x13,lsl#26 // base 2^26 -> base 2^64 + lsr x13,x14,#12 + adds x12,x12,x14,lsl#52 + add x13,x13,x15,lsl#14 + adc x13,x13,xzr + lsr x14,x16,#24 + adds x13,x13,x16,lsl#40 + adc x14,x14,xzr + + cmp x17,#0 // is_base2_26? + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) + csel x4,x4,x12,eq // choose between radixes + csel x5,x5,x13,eq + csel x6,x6,x14,eq + +.Loop: + ldp x10,x11,[x1],#16 // load input + sub x2,x2,#16 +#ifdef __AARCH64EB__ + rev x10,x10 + rev x11,x11 +#endif + adds x4,x4,x10 // accumulate input + adcs x5,x5,x11 + + mul x12,x4,x7 // h0*r0 + adc x6,x6,x3 + umulh x13,x4,x7 + + mul x10,x5,x9 // h1*5*r1 + umulh x11,x5,x9 + + adds x12,x12,x10 + mul x10,x4,x8 // h0*r1 + adc x13,x13,x11 + umulh x14,x4,x8 + + adds x13,x13,x10 + mul x10,x5,x7 // h1*r0 + adc x14,x14,xzr + umulh x11,x5,x7 + + adds x13,x13,x10 + mul x10,x6,x9 // h2*5*r1 + adc x14,x14,x11 + mul x11,x6,x7 // h2*r0 + + adds x13,x13,x10 + adc x14,x14,x11 + + and x10,x14,#-4 // final reduction + and x6,x14,#3 + add x10,x10,x14,lsr#2 + adds x4,x12,x10 + adcs x5,x13,xzr + adc x6,x6,xzr + + cbnz x2,.Loop + + stp x4,x5,[x0] // store hash value + stp x6,xzr,[x0,#16] // [and clear is_base2_26] + +.Lno_data: + ret +.size crypton_poly1305_asm_blocks,.-crypton_poly1305_asm_blocks + +.type crypton_poly1305_asm_emit,%function +.align 5 +crypton_poly1305_asm_emit: +.Lcrypton_poly1305_asm_emit: + ldp x4,x5,[x0] // load hash base 2^64 + ldp x6,x7,[x0,#16] // [along with is_base2_26] + ldp x10,x11,[x2] // load nonce + +#ifdef __AARCH64EB__ + lsr x12,x4,#32 + mov w13,w4 + lsr x14,x5,#32 + mov w15,w5 + lsr x16,x6,#32 +#else + mov w12,w4 + lsr x13,x4,#32 + mov w14,w5 + lsr x15,x5,#32 + mov w16,w6 +#endif + + add x12,x12,x13,lsl#26 // base 2^26 -> base 2^64 + lsr x13,x14,#12 + adds x12,x12,x14,lsl#52 + add x13,x13,x15,lsl#14 + adc x13,x13,xzr + lsr x14,x16,#24 + adds x13,x13,x16,lsl#40 + adc x14,x14,xzr + + cmp x7,#0 // is_base2_26? + csel x4,x4,x12,eq // choose between radixes + csel x5,x5,x13,eq + csel x6,x6,x14,eq + + adds x12,x4,#5 // compare to modulus + adcs x13,x5,xzr + adc x14,x6,xzr + + tst x14,#-4 // see if it's carried/borrowed + + csel x4,x4,x12,eq + csel x5,x5,x13,eq + +#ifdef __AARCH64EB__ + ror x10,x10,#32 // flip nonce words + ror x11,x11,#32 +#endif + adds x4,x4,x10 // accumulate nonce + adc x5,x5,x11 +#ifdef __AARCH64EB__ + rev x4,x4 // flip output bytes + rev x5,x5 +#endif + stp x4,x5,[x1] // write result + + ret +.size crypton_poly1305_asm_emit,.-crypton_poly1305_asm_emit +.type crypton_poly1305_asm_mult,%function +.align 5 +crypton_poly1305_asm_mult: + mul x12,x4,x7 // h0*r0 + umulh x13,x4,x7 + + mul x10,x5,x9 // h1*5*r1 + umulh x11,x5,x9 + + adds x12,x12,x10 + mul x10,x4,x8 // h0*r1 + adc x13,x13,x11 + umulh x14,x4,x8 + + adds x13,x13,x10 + mul x10,x5,x7 // h1*r0 + adc x14,x14,xzr + umulh x11,x5,x7 + + adds x13,x13,x10 + mul x10,x6,x9 // h2*5*r1 + adc x14,x14,x11 + mul x11,x6,x7 // h2*r0 + + adds x13,x13,x10 + adc x14,x14,x11 + + and x10,x14,#-4 // final reduction + and x6,x14,#3 + add x10,x10,x14,lsr#2 + adds x4,x12,x10 + adcs x5,x13,xzr + adc x6,x6,xzr + + ret +.size crypton_poly1305_asm_mult,.-crypton_poly1305_asm_mult + +.type crypton_poly1305_asm_splat,%function +.align 4 +crypton_poly1305_asm_splat: + and x12,x4,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x13,x4,#26,#26 + extr x14,x5,x4,#52 + and x14,x14,#0x03ffffff + ubfx x15,x5,#14,#26 + extr x16,x6,x5,#40 + + str w12,[x0,#16*0] // r0 + add w12,w13,w13,lsl#2 // r1*5 + str w13,[x0,#16*1] // r1 + add w13,w14,w14,lsl#2 // r2*5 + str w12,[x0,#16*2] // s1 + str w14,[x0,#16*3] // r2 + add w14,w15,w15,lsl#2 // r3*5 + str w13,[x0,#16*4] // s2 + str w15,[x0,#16*5] // r3 + add w15,w16,w16,lsl#2 // r4*5 + str w14,[x0,#16*6] // s3 + str w16,[x0,#16*7] // r4 + str w15,[x0,#16*8] // s4 + + ret +.size crypton_poly1305_asm_splat,.-crypton_poly1305_asm_splat + +#ifdef __KERNEL__ +.globl crypton_poly1305_asm_blocks_neon +#endif +.type crypton_poly1305_asm_blocks_neon,%function +.align 5 +crypton_poly1305_asm_blocks_neon: +.Lcrypton_poly1305_asm_blocks_neon: + ldr x17,[x0,#24] + cmp x2,#128 + b.lo .Lcrypton_poly1305_asm_blocks + +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__-64]! + add x29,sp,#0 + + stp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // meet ABI requirements + stp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + + cbz x17,.Lbase2_64_neon + + ldp w10,w11,[x0] // load hash value base 2^26 + ldp w12,w13,[x0,#8] + ldr w14,[x0,#16] + + tst x2,#31 + b.eq .Leven_neon + + ldp x7,x8,[x0,#32] // load key value + + add x4,x10,x11,lsl#26 // base 2^26 -> base 2^64 + lsr x5,x12,#12 + adds x4,x4,x12,lsl#52 + add x5,x5,x13,lsl#14 + adc x5,x5,xzr + lsr x6,x14,#24 + adds x5,x5,x14,lsl#40 + adc x14,x6,xzr // can be partially reduced... + + ldp x12,x13,[x1],#16 // load input + sub x2,x2,#16 + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) + +#ifdef __AARCH64EB__ + rev x12,x12 + rev x13,x13 +#endif + adds x4,x4,x12 // accumulate input + adcs x5,x5,x13 + adc x6,x6,x3 + + bl crypton_poly1305_asm_mult + + and x10,x4,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x11,x4,#26,#26 + extr x12,x5,x4,#52 + and x12,x12,#0x03ffffff + ubfx x13,x5,#14,#26 + extr x14,x6,x5,#40 + + b .Leven_neon + +.align 4 +.Lbase2_64_neon: + ldp x7,x8,[x0,#32] // load key value + + ldp x4,x5,[x0] // load hash value base 2^64 + ldr x6,[x0,#16] + + tst x2,#31 + b.eq .Linit_neon + + ldp x12,x13,[x1],#16 // load input + sub x2,x2,#16 + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) +#ifdef __AARCH64EB__ + rev x12,x12 + rev x13,x13 +#endif + adds x4,x4,x12 // accumulate input + adcs x5,x5,x13 + adc x6,x6,x3 + + bl crypton_poly1305_asm_mult + +.Linit_neon: + ldr w17,[x0,#48] // first table element + and x10,x4,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x11,x4,#26,#26 + extr x12,x5,x4,#52 + and x12,x12,#0x03ffffff + ubfx x13,x5,#14,#26 + extr x14,x6,x5,#40 + + cmp w17,#-1 // is value impossible? + b.ne .Leven_neon + + fmov d24,x10 + fmov d25,x11 + fmov d26,x12 + fmov d27,x13 + fmov d28,x14 + + ////////////////////////////////// initialize r^n table + mov x4,x7 // r^1 + add x9,x8,x8,lsr#2 // s1 = r1 + (r1 >> 2) + mov x5,x8 + mov x6,xzr + add x0,x0,#48+12 + bl crypton_poly1305_asm_splat + + bl crypton_poly1305_asm_mult // r^2 + sub x0,x0,#4 + bl crypton_poly1305_asm_splat + + bl crypton_poly1305_asm_mult // r^3 + sub x0,x0,#4 + bl crypton_poly1305_asm_splat + + bl crypton_poly1305_asm_mult // r^4 + sub x0,x0,#4 + bl crypton_poly1305_asm_splat + sub x0,x0,#48 + b .Ldo_neon + +.align 4 +.Leven_neon: + fmov d24,x10 + fmov d25,x11 + fmov d26,x12 + fmov d27,x13 + fmov d28,x14 + +.Ldo_neon: + ldp x8,x12,[x1,#32] // inp[2:3] + subs x2,x2,#64 + ldp x9,x13,[x1,#48] + add x16,x1,#96 + adr x17,.Lzeros + + lsl x3,x3,#24 + add x15,x0,#48 + +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + and x5,x9,#0x03ffffff + ubfx x6,x8,#26,#26 + ubfx x7,x9,#26,#26 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + extr x8,x12,x8,#52 + extr x9,x13,x9,#52 + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + fmov d14,x4 + and x8,x8,#0x03ffffff + and x9,x9,#0x03ffffff + ubfx x10,x12,#14,#26 + ubfx x11,x13,#14,#26 + add x12,x3,x12,lsr#40 + add x13,x3,x13,lsr#40 + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + fmov d15,x6 + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + fmov d16,x8 + fmov d17,x10 + fmov d18,x12 + + ldp x8,x12,[x1],#16 // inp[0:1] + ldp x9,x13,[x1],#48 + + ld1 {v0.4s,v1.4s,v2.4s,v3.4s},[x15],#64 + ld1 {v4.4s,v5.4s,v6.4s,v7.4s},[x15],#64 + ld1 {v8.4s},[x15] + +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + and x5,x9,#0x03ffffff + ubfx x6,x8,#26,#26 + ubfx x7,x9,#26,#26 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + extr x8,x12,x8,#52 + extr x9,x13,x9,#52 + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + fmov d9,x4 + and x8,x8,#0x03ffffff + and x9,x9,#0x03ffffff + ubfx x10,x12,#14,#26 + ubfx x11,x13,#14,#26 + add x12,x3,x12,lsr#40 + add x13,x3,x13,lsr#40 + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + fmov d10,x6 + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + movi v31.2d,#-1 + fmov d11,x8 + fmov d12,x10 + fmov d13,x12 + ushr v31.2d,v31.2d,#38 + + b.ls .Lskip_loop + +.align 4 +.Loop_neon: + //////////////////////////////////////////////////////////////// + // ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2 + // ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r + // ___________________/ + // ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2 + // ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r + // ___________________/ ____________________/ + // + // Note that we start with inp[2:3]*r^2. This is because it + // doesn't depend on reduction in previous iteration. + //////////////////////////////////////////////////////////////// + // d4 = h0*r4 + h1*r3 + h2*r2 + h3*r1 + h4*r0 + // d3 = h0*r3 + h1*r2 + h2*r1 + h3*r0 + h4*5*r4 + // d2 = h0*r2 + h1*r1 + h2*r0 + h3*5*r4 + h4*5*r3 + // d1 = h0*r1 + h1*r0 + h2*5*r4 + h3*5*r3 + h4*5*r2 + // d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1 + + subs x2,x2,#64 + umull v23.2d,v14.2s,v7.s[2] + csel x16,x17,x16,lo + umull v22.2d,v14.2s,v5.s[2] + umull v21.2d,v14.2s,v3.s[2] + ldp x8,x12,[x16],#16 // inp[2:3] (or zero) + umull v20.2d,v14.2s,v1.s[2] + ldp x9,x13,[x16],#48 + umull v19.2d,v14.2s,v0.s[2] +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + + umlal v23.2d,v15.2s,v5.s[2] + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + umlal v22.2d,v15.2s,v3.s[2] + and x5,x9,#0x03ffffff + umlal v21.2d,v15.2s,v1.s[2] + ubfx x6,x8,#26,#26 + umlal v20.2d,v15.2s,v0.s[2] + ubfx x7,x9,#26,#26 + umlal v19.2d,v15.2s,v8.s[2] + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + + umlal v23.2d,v16.2s,v3.s[2] + extr x8,x12,x8,#52 + umlal v22.2d,v16.2s,v1.s[2] + extr x9,x13,x9,#52 + umlal v21.2d,v16.2s,v0.s[2] + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + umlal v20.2d,v16.2s,v8.s[2] + fmov d14,x4 + umlal v19.2d,v16.2s,v6.s[2] + and x8,x8,#0x03ffffff + + umlal v23.2d,v17.2s,v1.s[2] + and x9,x9,#0x03ffffff + umlal v22.2d,v17.2s,v0.s[2] + ubfx x10,x12,#14,#26 + umlal v21.2d,v17.2s,v8.s[2] + ubfx x11,x13,#14,#26 + umlal v20.2d,v17.2s,v6.s[2] + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + umlal v19.2d,v17.2s,v4.s[2] + fmov d15,x6 + + add v11.2s,v11.2s,v26.2s + add x12,x3,x12,lsr#40 + umlal v23.2d,v18.2s,v0.s[2] + add x13,x3,x13,lsr#40 + umlal v22.2d,v18.2s,v8.s[2] + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + umlal v21.2d,v18.2s,v6.s[2] + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + umlal v20.2d,v18.2s,v4.s[2] + fmov d16,x8 + umlal v19.2d,v18.2s,v2.s[2] + fmov d17,x10 + + //////////////////////////////////////////////////////////////// + // (hash+inp[0:1])*r^4 and accumulate + + add v9.2s,v9.2s,v24.2s + fmov d18,x12 + umlal v22.2d,v11.2s,v1.s[0] + ldp x8,x12,[x1],#16 // inp[0:1] + umlal v19.2d,v11.2s,v6.s[0] + ldp x9,x13,[x1],#48 + umlal v23.2d,v11.2s,v3.s[0] + umlal v20.2d,v11.2s,v8.s[0] + umlal v21.2d,v11.2s,v0.s[0] +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + + add v10.2s,v10.2s,v25.2s + umlal v22.2d,v9.2s,v5.s[0] + umlal v23.2d,v9.2s,v7.s[0] + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + umlal v21.2d,v9.2s,v3.s[0] + and x5,x9,#0x03ffffff + umlal v19.2d,v9.2s,v0.s[0] + ubfx x6,x8,#26,#26 + umlal v20.2d,v9.2s,v1.s[0] + ubfx x7,x9,#26,#26 + + add v12.2s,v12.2s,v27.2s + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + umlal v22.2d,v10.2s,v3.s[0] + extr x8,x12,x8,#52 + umlal v23.2d,v10.2s,v5.s[0] + extr x9,x13,x9,#52 + umlal v19.2d,v10.2s,v8.s[0] + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + umlal v21.2d,v10.2s,v1.s[0] + fmov d9,x4 + umlal v20.2d,v10.2s,v0.s[0] + and x8,x8,#0x03ffffff + + add v13.2s,v13.2s,v28.2s + and x9,x9,#0x03ffffff + umlal v22.2d,v12.2s,v0.s[0] + ubfx x10,x12,#14,#26 + umlal v19.2d,v12.2s,v4.s[0] + ubfx x11,x13,#14,#26 + umlal v23.2d,v12.2s,v1.s[0] + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + umlal v20.2d,v12.2s,v6.s[0] + fmov d10,x6 + umlal v21.2d,v12.2s,v8.s[0] + add x12,x3,x12,lsr#40 + + umlal v22.2d,v13.2s,v8.s[0] + add x13,x3,x13,lsr#40 + umlal v19.2d,v13.2s,v2.s[0] + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + umlal v23.2d,v13.2s,v0.s[0] + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + umlal v20.2d,v13.2s,v4.s[0] + fmov d11,x8 + umlal v21.2d,v13.2s,v6.s[0] + fmov d12,x10 + fmov d13,x12 + + ///////////////////////////////////////////////////////////////// + // lazy reduction as discussed in "NEON crypto" by D.J. Bernstein + // and P. Schwabe + // + // [see discussion in poly1305-armv4 module] + + ushr v29.2d,v22.2d,#26 + xtn v27.2s,v22.2d + ushr v30.2d,v19.2d,#26 + and v19.16b,v19.16b,v31.16b + add v23.2d,v23.2d,v29.2d // h3 -> h4 + bic v27.2s,#0xfc,lsl#24 // &=0x03ffffff + add v20.2d,v20.2d,v30.2d // h0 -> h1 + + ushr v29.2d,v23.2d,#26 + xtn v28.2s,v23.2d + ushr v30.2d,v20.2d,#26 + xtn v25.2s,v20.2d + bic v28.2s,#0xfc,lsl#24 + add v21.2d,v21.2d,v30.2d // h1 -> h2 + + add v19.2d,v19.2d,v29.2d + shl v29.2d,v29.2d,#2 + shrn v30.2s,v21.2d,#26 + xtn v26.2s,v21.2d + add v19.2d,v19.2d,v29.2d // h4 -> h0 + bic v25.2s,#0xfc,lsl#24 + add v27.2s,v27.2s,v30.2s // h2 -> h3 + bic v26.2s,#0xfc,lsl#24 + + shrn v29.2s,v19.2d,#26 + xtn v24.2s,v19.2d + ushr v30.2s,v27.2s,#26 + bic v27.2s,#0xfc,lsl#24 + bic v24.2s,#0xfc,lsl#24 + add v25.2s,v25.2s,v29.2s // h0 -> h1 + add v28.2s,v28.2s,v30.2s // h3 -> h4 + + b.hi .Loop_neon + +.Lskip_loop: + dup v16.2d,v16.d[0] + add v11.2s,v11.2s,v26.2s + + //////////////////////////////////////////////////////////////// + // multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1 + + adds x2,x2,#32 + b.ne .Long_tail + + dup v16.2d,v11.d[0] + add v14.2s,v9.2s,v24.2s + add v17.2s,v12.2s,v27.2s + add v15.2s,v10.2s,v25.2s + add v18.2s,v13.2s,v28.2s + +.Long_tail: + dup v14.2d,v14.d[0] + umull2 v19.2d,v16.4s,v6.4s + umull2 v22.2d,v16.4s,v1.4s + umull2 v23.2d,v16.4s,v3.4s + umull2 v21.2d,v16.4s,v0.4s + umull2 v20.2d,v16.4s,v8.4s + + dup v15.2d,v15.d[0] + umlal2 v19.2d,v14.4s,v0.4s + umlal2 v21.2d,v14.4s,v3.4s + umlal2 v22.2d,v14.4s,v5.4s + umlal2 v23.2d,v14.4s,v7.4s + umlal2 v20.2d,v14.4s,v1.4s + + dup v17.2d,v17.d[0] + umlal2 v19.2d,v15.4s,v8.4s + umlal2 v22.2d,v15.4s,v3.4s + umlal2 v21.2d,v15.4s,v1.4s + umlal2 v23.2d,v15.4s,v5.4s + umlal2 v20.2d,v15.4s,v0.4s + + dup v18.2d,v18.d[0] + umlal2 v22.2d,v17.4s,v0.4s + umlal2 v23.2d,v17.4s,v1.4s + umlal2 v19.2d,v17.4s,v4.4s + umlal2 v20.2d,v17.4s,v6.4s + umlal2 v21.2d,v17.4s,v8.4s + + umlal2 v22.2d,v18.4s,v8.4s + umlal2 v19.2d,v18.4s,v2.4s + umlal2 v23.2d,v18.4s,v0.4s + umlal2 v20.2d,v18.4s,v4.4s + umlal2 v21.2d,v18.4s,v6.4s + + b.eq .Lshort_tail + + //////////////////////////////////////////////////////////////// + // (hash+inp[0:1])*r^4:r^3 and accumulate + + add v9.2s,v9.2s,v24.2s + umlal v22.2d,v11.2s,v1.2s + umlal v19.2d,v11.2s,v6.2s + umlal v23.2d,v11.2s,v3.2s + umlal v20.2d,v11.2s,v8.2s + umlal v21.2d,v11.2s,v0.2s + + add v10.2s,v10.2s,v25.2s + umlal v22.2d,v9.2s,v5.2s + umlal v19.2d,v9.2s,v0.2s + umlal v23.2d,v9.2s,v7.2s + umlal v20.2d,v9.2s,v1.2s + umlal v21.2d,v9.2s,v3.2s + + add v12.2s,v12.2s,v27.2s + umlal v22.2d,v10.2s,v3.2s + umlal v19.2d,v10.2s,v8.2s + umlal v23.2d,v10.2s,v5.2s + umlal v20.2d,v10.2s,v0.2s + umlal v21.2d,v10.2s,v1.2s + + add v13.2s,v13.2s,v28.2s + umlal v22.2d,v12.2s,v0.2s + umlal v19.2d,v12.2s,v4.2s + umlal v23.2d,v12.2s,v1.2s + umlal v20.2d,v12.2s,v6.2s + umlal v21.2d,v12.2s,v8.2s + + umlal v22.2d,v13.2s,v8.2s + umlal v19.2d,v13.2s,v2.2s + umlal v23.2d,v13.2s,v0.2s + umlal v20.2d,v13.2s,v4.2s + umlal v21.2d,v13.2s,v6.2s + +.Lshort_tail: + //////////////////////////////////////////////////////////////// + // horizontal add + + addp v22.2d,v22.2d,v22.2d + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // meet ABI requirements + addp v19.2d,v19.2d,v19.2d + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + addp v23.2d,v23.2d,v23.2d + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + addp v20.2d,v20.2d,v20.2d + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + addp v21.2d,v21.2d,v21.2d + ldr x30,[sp,#__SIZEOF_POINTER__] + + //////////////////////////////////////////////////////////////// + // lazy reduction, but without narrowing + + ushr v29.2d,v22.2d,#26 + and v22.16b,v22.16b,v31.16b + ushr v30.2d,v19.2d,#26 + and v19.16b,v19.16b,v31.16b + + add v23.2d,v23.2d,v29.2d // h3 -> h4 + add v20.2d,v20.2d,v30.2d // h0 -> h1 + + ushr v29.2d,v23.2d,#26 + and v23.16b,v23.16b,v31.16b + ushr v30.2d,v20.2d,#26 + and v20.16b,v20.16b,v31.16b + add v21.2d,v21.2d,v30.2d // h1 -> h2 + + add v19.2d,v19.2d,v29.2d + shl v29.2d,v29.2d,#2 + ushr v30.2d,v21.2d,#26 + and v21.16b,v21.16b,v31.16b + add v19.2d,v19.2d,v29.2d // h4 -> h0 + add v22.2d,v22.2d,v30.2d // h2 -> h3 + + ushr v29.2d,v19.2d,#26 + and v19.16b,v19.16b,v31.16b + ushr v30.2d,v22.2d,#26 + and v22.16b,v22.16b,v31.16b + add v20.2d,v20.2d,v29.2d // h0 -> h1 + add v23.2d,v23.2d,v30.2d // h3 -> h4 + + //////////////////////////////////////////////////////////////// + // write the result, can be partially reduced + + st4 {v19.s,v20.s,v21.s,v22.s}[0],[x0],#16 + mov x4,#1 + st1 {v23.s}[0],[x0] + str x4,[x0,#8] // set is_base2_26 + + ldr x29,[sp],#2*__SIZEOF_POINTER__+64 +.inst 0xd50323bf // autiasp + ret +.size crypton_poly1305_asm_blocks_neon,.-crypton_poly1305_asm_blocks_neon + +.align 5 +.Lzeros: +.long 0,0,0,0,0,0,0,0 +.byte 80,111,108,121,49,51,48,53,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 +.align 2 +#if !defined(__KERNEL__) && !defined(_WIN64) +.comm crypton_armcap_P,4,4 +.hidden crypton_armcap_P +#endif + +.section .note.GNU-stack,"",%progbits diff --git a/cbits/asm/poly1305-armv8.pl b/cbits/asm/poly1305-armv8.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-armv8.pl @@ -0,0 +1,927 @@ +#!/usr/bin/env perl +# SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# This module implements Poly1305 hash for ARMv8. +# +# June 2015 +# +# Numbers are cycles per processed byte with poly1305_blocks alone. +# +# IALU/gcc-4.9 NEON +# +# Apple A7 1.86/+5% 0.72 +# Apple A10 0.71 +# Apple A14/M1 0.97/+63% 0.48 +# Cortex-A53 2.69/+58% 1.47 +# Cortex-A57 2.70/+7% 1.14 +# Cortex-A76 2.60 1.00 +# Cortex-X2 1.00 0.66 +# Cortex-X925 1.00 0.53 +# Denver 1.64/+50% 1.18(*) +# X-Gene 2.13/+68% 2.27 +# Mongoose 1.77/+75% 1.12 +# Kryo 2.70/+55% 1.13 +# ThunderX2 1.17/+95% 1.36 +# Snapdragon X 0.95 0.48 +# +# (*) estimate based on resources availability is less than 1.0, +# i.e. measured result is worse than expected, presumably binary +# translator is not almighty; + +$flavour=shift; +$output=shift; + +if ($flavour && $flavour ne "void") { + $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; + ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or + ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or + die "can't locate arm-xlate.pl"; + + open STDOUT,"| \"$^X\" $xlate $flavour $output"; +} else { + open STDOUT,">$output"; +} + +my ($ctx,$inp,$len,$padbit) = map("x$_",(0..3)); +my ($mac,$nonce)=($inp,$len); + +my ($h0,$h1,$h2,$r0,$r1,$s1,$t0,$t1,$d0,$d1,$d2) = map("x$_",(4..14)); + +$code.=<<___; +#ifndef __KERNEL__ +# include "arm_arch.h" +.extern OPENSSL_armcap_P +#endif + +.text + +// forward "declarations" are required for Apple +.globl poly1305_blocks +.globl poly1305_emit + +.globl poly1305_init +.type poly1305_init,%function +.align 5 +poly1305_init: + cmp $inp,xzr + stp xzr,xzr,[$ctx] // zero hash value + stp xzr,xzr,[$ctx,#16] // [along with is_base2_26] + + csel c0,czr,c0,eq + b.eq .Lno_key + +#ifndef __KERNEL__ + adrp c17,OPENSSL_armcap_P + ldr w17,[c17,#:lo12:OPENSSL_armcap_P] +#endif + + ldp $r0,$r1,[$inp] // load key + mov $s1,#0xfffffffc0fffffff + movk $s1,#0x0fff,lsl#48 +#ifdef __AARCH64EB__ + rev $r0,$r0 // flip bytes + rev $r1,$r1 +#endif + and $r0,$r0,$s1 // &=0ffffffc0fffffff + and $s1,$s1,#-4 + and $r1,$r1,$s1 // &=0ffffffc0ffffffc + mov w#$s1,#-1 + stp $r0,$r1,[$ctx,#32] // save key value + str w#$s1,[$ctx,#48] // impossible key power value + +#ifndef __KERNEL__ + tst w17,#ARMV7_NEON + + adr c13,.Lpoly1305_blocks + adr c15,.Lpoly1305_blocks_neon + adr c14,.Lpoly1305_emit + + csel c13,c13,c15,eq +# ifdef __CHERI_PURE_CAPABILITY__ + add c13, c13, #1 + add c14, c14, #1 + seal c13, c13, rb + seal c14, c14, rb +# endif + +# ifdef __ILP32__ + stp w13,w14,[$len] +# else + stp c13,c14,[$len] +# endif + mov x0,#1 +#else + mov x0,#0 +#endif +.Lno_key: + ret +.size poly1305_init,.-poly1305_init + +.type poly1305_blocks,%function +.align 5 +poly1305_blocks: +.Lpoly1305_blocks: + ands $len,$len,#-16 + b.eq .Lno_data + + ldp $h0,$h1,[$ctx] // load hash value + ldp $h2,x17,[$ctx,#16] // [along with is_base2_26] + ldp $r0,$r1,[$ctx,#32] // load key value + +#ifdef __AARCH64EB__ + lsr $d0,$h0,#32 + mov w#$d1,w#$h0 + lsr $d2,$h1,#32 + mov w15,w#$h1 + lsr x16,$h2,#32 +#else + mov w#$d0,w#$h0 + lsr $d1,$h0,#32 + mov w#$d2,w#$h1 + lsr x15,$h1,#32 + mov w16,w#$h2 +#endif + + add $d0,$d0,$d1,lsl#26 // base 2^26 -> base 2^64 + lsr $d1,$d2,#12 + adds $d0,$d0,$d2,lsl#52 + add $d1,$d1,x15,lsl#14 + adc $d1,$d1,xzr + lsr $d2,x16,#24 + adds $d1,$d1,x16,lsl#40 + adc $d2,$d2,xzr + + cmp x17,#0 // is_base2_26? + add $s1,$r1,$r1,lsr#2 // s1 = r1 + (r1 >> 2) + csel $h0,$h0,$d0,eq // choose between radixes + csel $h1,$h1,$d1,eq + csel $h2,$h2,$d2,eq + +.Loop: + ldp $t0,$t1,[$inp],#16 // load input + sub $len,$len,#16 +#ifdef __AARCH64EB__ + rev $t0,$t0 + rev $t1,$t1 +#endif + adds $h0,$h0,$t0 // accumulate input + adcs $h1,$h1,$t1 + + mul $d0,$h0,$r0 // h0*r0 + adc $h2,$h2,$padbit + umulh $d1,$h0,$r0 + + mul $t0,$h1,$s1 // h1*5*r1 + umulh $t1,$h1,$s1 + + adds $d0,$d0,$t0 + mul $t0,$h0,$r1 // h0*r1 + adc $d1,$d1,$t1 + umulh $d2,$h0,$r1 + + adds $d1,$d1,$t0 + mul $t0,$h1,$r0 // h1*r0 + adc $d2,$d2,xzr + umulh $t1,$h1,$r0 + + adds $d1,$d1,$t0 + mul $t0,$h2,$s1 // h2*5*r1 + adc $d2,$d2,$t1 + mul $t1,$h2,$r0 // h2*r0 + + adds $d1,$d1,$t0 + adc $d2,$d2,$t1 + + and $t0,$d2,#-4 // final reduction + and $h2,$d2,#3 + add $t0,$t0,$d2,lsr#2 + adds $h0,$d0,$t0 + adcs $h1,$d1,xzr + adc $h2,$h2,xzr + + cbnz $len,.Loop + + stp $h0,$h1,[$ctx] // store hash value + stp $h2,xzr,[$ctx,#16] // [and clear is_base2_26] + +.Lno_data: + ret +.size poly1305_blocks,.-poly1305_blocks + +.type poly1305_emit,%function +.align 5 +poly1305_emit: +.Lpoly1305_emit: + ldp $h0,$h1,[$ctx] // load hash base 2^64 + ldp $h2,$r0,[$ctx,#16] // [along with is_base2_26] + ldp $t0,$t1,[$nonce] // load nonce + +#ifdef __AARCH64EB__ + lsr $d0,$h0,#32 + mov w#$d1,w#$h0 + lsr $d2,$h1,#32 + mov w15,w#$h1 + lsr x16,$h2,#32 +#else + mov w#$d0,w#$h0 + lsr $d1,$h0,#32 + mov w#$d2,w#$h1 + lsr x15,$h1,#32 + mov w16,w#$h2 +#endif + + add $d0,$d0,$d1,lsl#26 // base 2^26 -> base 2^64 + lsr $d1,$d2,#12 + adds $d0,$d0,$d2,lsl#52 + add $d1,$d1,x15,lsl#14 + adc $d1,$d1,xzr + lsr $d2,x16,#24 + adds $d1,$d1,x16,lsl#40 + adc $d2,$d2,xzr + + cmp $r0,#0 // is_base2_26? + csel $h0,$h0,$d0,eq // choose between radixes + csel $h1,$h1,$d1,eq + csel $h2,$h2,$d2,eq + + adds $d0,$h0,#5 // compare to modulus + adcs $d1,$h1,xzr + adc $d2,$h2,xzr + + tst $d2,#-4 // see if it's carried/borrowed + + csel $h0,$h0,$d0,eq + csel $h1,$h1,$d1,eq + +#ifdef __AARCH64EB__ + ror $t0,$t0,#32 // flip nonce words + ror $t1,$t1,#32 +#endif + adds $h0,$h0,$t0 // accumulate nonce + adc $h1,$h1,$t1 +#ifdef __AARCH64EB__ + rev $h0,$h0 // flip output bytes + rev $h1,$h1 +#endif + stp $h0,$h1,[$mac] // write result + + ret +.size poly1305_emit,.-poly1305_emit +___ +my ($R0,$R1,$S1,$R2,$S2,$R3,$S3,$R4,$S4) = map("v$_.4s",(0..8)); +my ($IN01_0,$IN01_1,$IN01_2,$IN01_3,$IN01_4) = map("v$_.2s",(9..13)); +my ($IN23_0,$IN23_1,$IN23_2,$IN23_3,$IN23_4) = map("v$_.2s",(14..18)); +my ($ACC0,$ACC1,$ACC2,$ACC3,$ACC4) = map("v$_.2d",(19..23)); +my ($H0,$H1,$H2,$H3,$H4) = map("v$_.2s",(24..28)); +my ($T0,$T1,$MASK) = map("v$_",(29..31)); + +my ($in2,$zeros)=("x16","x17"); +my $is_base2_26 = $zeros; # borrow + +$code.=<<___; +.type poly1305_mult,%function +.align 5 +poly1305_mult: + mul $d0,$h0,$r0 // h0*r0 + umulh $d1,$h0,$r0 + + mul $t0,$h1,$s1 // h1*5*r1 + umulh $t1,$h1,$s1 + + adds $d0,$d0,$t0 + mul $t0,$h0,$r1 // h0*r1 + adc $d1,$d1,$t1 + umulh $d2,$h0,$r1 + + adds $d1,$d1,$t0 + mul $t0,$h1,$r0 // h1*r0 + adc $d2,$d2,xzr + umulh $t1,$h1,$r0 + + adds $d1,$d1,$t0 + mul $t0,$h2,$s1 // h2*5*r1 + adc $d2,$d2,$t1 + mul $t1,$h2,$r0 // h2*r0 + + adds $d1,$d1,$t0 + adc $d2,$d2,$t1 + + and $t0,$d2,#-4 // final reduction + and $h2,$d2,#3 + add $t0,$t0,$d2,lsr#2 + adds $h0,$d0,$t0 + adcs $h1,$d1,xzr + adc $h2,$h2,xzr + + ret +.size poly1305_mult,.-poly1305_mult + +.type poly1305_splat,%function +.align 4 +poly1305_splat: + and x12,$h0,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x13,$h0,#26,#26 + extr x14,$h1,$h0,#52 + and x14,x14,#0x03ffffff + ubfx x15,$h1,#14,#26 + extr x16,$h2,$h1,#40 + + str w12,[$ctx,#16*0] // r0 + add w12,w13,w13,lsl#2 // r1*5 + str w13,[$ctx,#16*1] // r1 + add w13,w14,w14,lsl#2 // r2*5 + str w12,[$ctx,#16*2] // s1 + str w14,[$ctx,#16*3] // r2 + add w14,w15,w15,lsl#2 // r3*5 + str w13,[$ctx,#16*4] // s2 + str w15,[$ctx,#16*5] // r3 + add w15,w16,w16,lsl#2 // r4*5 + str w14,[$ctx,#16*6] // s3 + str w16,[$ctx,#16*7] // r4 + str w15,[$ctx,#16*8] // s4 + + ret +.size poly1305_splat,.-poly1305_splat + +#ifdef __KERNEL__ +.globl poly1305_blocks_neon +#endif +.type poly1305_blocks_neon,%function +.align 5 +poly1305_blocks_neon: +.Lpoly1305_blocks_neon: + ldr $is_base2_26,[$ctx,#24] + cmp $len,#128 + b.lo .Lpoly1305_blocks + + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-2*__SIZEOF_POINTER__-64]! + add c29,csp,#0 + + stp d8,d9,[csp,#2*__SIZEOF_POINTER__+0] // meet ABI requirements + stp d10,d11,[csp,#2*__SIZEOF_POINTER__+16] + stp d12,d13,[csp,#2*__SIZEOF_POINTER__+32] + stp d14,d15,[csp,#2*__SIZEOF_POINTER__+48] + + cbz $is_base2_26,.Lbase2_64_neon + + ldp w10,w11,[$ctx] // load hash value base 2^26 + ldp w12,w13,[$ctx,#8] + ldr w14,[$ctx,#16] + + tst $len,#31 + b.eq .Leven_neon + + ldp $r0,$r1,[$ctx,#32] // load key value + + add $h0,x10,x11,lsl#26 // base 2^26 -> base 2^64 + lsr $h1,x12,#12 + adds $h0,$h0,x12,lsl#52 + add $h1,$h1,x13,lsl#14 + adc $h1,$h1,xzr + lsr $h2,x14,#24 + adds $h1,$h1,x14,lsl#40 + adc $d2,$h2,xzr // can be partially reduced... + + ldp $d0,$d1,[$inp],#16 // load input + sub $len,$len,#16 + add $s1,$r1,$r1,lsr#2 // s1 = r1 + (r1 >> 2) + +#ifdef __AARCH64EB__ + rev $d0,$d0 + rev $d1,$d1 +#endif + adds $h0,$h0,$d0 // accumulate input + adcs $h1,$h1,$d1 + adc $h2,$h2,$padbit + + bl poly1305_mult + + and x10,$h0,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x11,$h0,#26,#26 + extr x12,$h1,$h0,#52 + and x12,x12,#0x03ffffff + ubfx x13,$h1,#14,#26 + extr x14,$h2,$h1,#40 + + b .Leven_neon + +.align 4 +.Lbase2_64_neon: + ldp $r0,$r1,[$ctx,#32] // load key value + + ldp $h0,$h1,[$ctx] // load hash value base 2^64 + ldr $h2,[$ctx,#16] + + tst $len,#31 + b.eq .Linit_neon + + ldp $d0,$d1,[$inp],#16 // load input + sub $len,$len,#16 + add $s1,$r1,$r1,lsr#2 // s1 = r1 + (r1 >> 2) +#ifdef __AARCH64EB__ + rev $d0,$d0 + rev $d1,$d1 +#endif + adds $h0,$h0,$d0 // accumulate input + adcs $h1,$h1,$d1 + adc $h2,$h2,$padbit + + bl poly1305_mult + +.Linit_neon: + ldr w17,[$ctx,#48] // first table element + and x10,$h0,#0x03ffffff // base 2^64 -> base 2^26 + ubfx x11,$h0,#26,#26 + extr x12,$h1,$h0,#52 + and x12,x12,#0x03ffffff + ubfx x13,$h1,#14,#26 + extr x14,$h2,$h1,#40 + + cmp w17,#-1 // is value impossible? + b.ne .Leven_neon + + fmov ${H0},x10 + fmov ${H1},x11 + fmov ${H2},x12 + fmov ${H3},x13 + fmov ${H4},x14 + + ////////////////////////////////// initialize r^n table + mov $h0,$r0 // r^1 + add $s1,$r1,$r1,lsr#2 // s1 = r1 + (r1 >> 2) + mov $h1,$r1 + mov $h2,xzr + cadd $ctx,$ctx,#48+12 + bl poly1305_splat + + bl poly1305_mult // r^2 + csub $ctx,$ctx,#4 + bl poly1305_splat + + bl poly1305_mult // r^3 + csub $ctx,$ctx,#4 + bl poly1305_splat + + bl poly1305_mult // r^4 + csub $ctx,$ctx,#4 + bl poly1305_splat + csub $ctx,$ctx,#48 // restore original $ctx + b .Ldo_neon + +.align 4 +.Leven_neon: + fmov ${H0},x10 + fmov ${H1},x11 + fmov ${H2},x12 + fmov ${H3},x13 + fmov ${H4},x14 + +.Ldo_neon: + ldp x8,x12,[$inp,#32] // inp[2:3] + subs $len,$len,#64 + ldp x9,x13,[$inp,#48] + cadd $in2,$inp,#96 + adr $zeros,.Lzeros + + lsl $padbit,$padbit,#24 + cadd x15,$ctx,#48 + +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + and x5,x9,#0x03ffffff + ubfx x6,x8,#26,#26 + ubfx x7,x9,#26,#26 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + extr x8,x12,x8,#52 + extr x9,x13,x9,#52 + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + fmov $IN23_0,x4 + and x8,x8,#0x03ffffff + and x9,x9,#0x03ffffff + ubfx x10,x12,#14,#26 + ubfx x11,x13,#14,#26 + add x12,$padbit,x12,lsr#40 + add x13,$padbit,x13,lsr#40 + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + fmov $IN23_1,x6 + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + fmov $IN23_2,x8 + fmov $IN23_3,x10 + fmov $IN23_4,x12 + + ldp x8,x12,[$inp],#16 // inp[0:1] + ldp x9,x13,[$inp],#48 + + ld1 {$R0,$R1,$S1,$R2},[x15],#64 + ld1 {$S2,$R3,$S3,$R4},[x15],#64 + ld1 {$S4},[x15] + +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + and x5,x9,#0x03ffffff + ubfx x6,x8,#26,#26 + ubfx x7,x9,#26,#26 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + extr x8,x12,x8,#52 + extr x9,x13,x9,#52 + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + fmov $IN01_0,x4 + and x8,x8,#0x03ffffff + and x9,x9,#0x03ffffff + ubfx x10,x12,#14,#26 + ubfx x11,x13,#14,#26 + add x12,$padbit,x12,lsr#40 + add x13,$padbit,x13,lsr#40 + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + fmov $IN01_1,x6 + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + movi $MASK.2d,#-1 + fmov $IN01_2,x8 + fmov $IN01_3,x10 + fmov $IN01_4,x12 + ushr $MASK.2d,$MASK.2d,#38 + + b.ls .Lskip_loop + +.align 4 +.Loop_neon: + //////////////////////////////////////////////////////////////// + // ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2 + // ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r + // \___________________/ + // ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2 + // ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r + // \___________________/ \____________________/ + // + // Note that we start with inp[2:3]*r^2. This is because it + // doesn't depend on reduction in previous iteration. + //////////////////////////////////////////////////////////////// + // d4 = h0*r4 + h1*r3 + h2*r2 + h3*r1 + h4*r0 + // d3 = h0*r3 + h1*r2 + h2*r1 + h3*r0 + h4*5*r4 + // d2 = h0*r2 + h1*r1 + h2*r0 + h3*5*r4 + h4*5*r3 + // d1 = h0*r1 + h1*r0 + h2*5*r4 + h3*5*r3 + h4*5*r2 + // d0 = h0*r0 + h1*5*r4 + h2*5*r3 + h3*5*r2 + h4*5*r1 + + subs $len,$len,#64 + umull $ACC4,$IN23_0,${R4}[2] + csel c#$in2,c#$zeros,c#$in2,lo + umull $ACC3,$IN23_0,${R3}[2] + umull $ACC2,$IN23_0,${R2}[2] + ldp x8,x12,[$in2],#16 // inp[2:3] (or zero) + umull $ACC1,$IN23_0,${R1}[2] + ldp x9,x13,[$in2],#48 + umull $ACC0,$IN23_0,${R0}[2] +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + + umlal $ACC4,$IN23_1,${R3}[2] + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + umlal $ACC3,$IN23_1,${R2}[2] + and x5,x9,#0x03ffffff + umlal $ACC2,$IN23_1,${R1}[2] + ubfx x6,x8,#26,#26 + umlal $ACC1,$IN23_1,${R0}[2] + ubfx x7,x9,#26,#26 + umlal $ACC0,$IN23_1,${S4}[2] + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + + umlal $ACC4,$IN23_2,${R2}[2] + extr x8,x12,x8,#52 + umlal $ACC3,$IN23_2,${R1}[2] + extr x9,x13,x9,#52 + umlal $ACC2,$IN23_2,${R0}[2] + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + umlal $ACC1,$IN23_2,${S4}[2] + fmov $IN23_0,x4 + umlal $ACC0,$IN23_2,${S3}[2] + and x8,x8,#0x03ffffff + + umlal $ACC4,$IN23_3,${R1}[2] + and x9,x9,#0x03ffffff + umlal $ACC3,$IN23_3,${R0}[2] + ubfx x10,x12,#14,#26 + umlal $ACC2,$IN23_3,${S4}[2] + ubfx x11,x13,#14,#26 + umlal $ACC1,$IN23_3,${S3}[2] + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + umlal $ACC0,$IN23_3,${S2}[2] + fmov $IN23_1,x6 + + add $IN01_2,$IN01_2,$H2 + add x12,$padbit,x12,lsr#40 + umlal $ACC4,$IN23_4,${R0}[2] + add x13,$padbit,x13,lsr#40 + umlal $ACC3,$IN23_4,${S4}[2] + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + umlal $ACC2,$IN23_4,${S3}[2] + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + umlal $ACC1,$IN23_4,${S2}[2] + fmov $IN23_2,x8 + umlal $ACC0,$IN23_4,${S1}[2] + fmov $IN23_3,x10 + + //////////////////////////////////////////////////////////////// + // (hash+inp[0:1])*r^4 and accumulate + + add $IN01_0,$IN01_0,$H0 + fmov $IN23_4,x12 + umlal $ACC3,$IN01_2,${R1}[0] + ldp x8,x12,[$inp],#16 // inp[0:1] + umlal $ACC0,$IN01_2,${S3}[0] + ldp x9,x13,[$inp],#48 + umlal $ACC4,$IN01_2,${R2}[0] + umlal $ACC1,$IN01_2,${S4}[0] + umlal $ACC2,$IN01_2,${R0}[0] +#ifdef __AARCH64EB__ + rev x8,x8 + rev x12,x12 + rev x9,x9 + rev x13,x13 +#endif + + add $IN01_1,$IN01_1,$H1 + umlal $ACC3,$IN01_0,${R3}[0] + umlal $ACC4,$IN01_0,${R4}[0] + and x4,x8,#0x03ffffff // base 2^64 -> base 2^26 + umlal $ACC2,$IN01_0,${R2}[0] + and x5,x9,#0x03ffffff + umlal $ACC0,$IN01_0,${R0}[0] + ubfx x6,x8,#26,#26 + umlal $ACC1,$IN01_0,${R1}[0] + ubfx x7,x9,#26,#26 + + add $IN01_3,$IN01_3,$H3 + add x4,x4,x5,lsl#32 // bfi x4,x5,#32,#32 + umlal $ACC3,$IN01_1,${R2}[0] + extr x8,x12,x8,#52 + umlal $ACC4,$IN01_1,${R3}[0] + extr x9,x13,x9,#52 + umlal $ACC0,$IN01_1,${S4}[0] + add x6,x6,x7,lsl#32 // bfi x6,x7,#32,#32 + umlal $ACC2,$IN01_1,${R1}[0] + fmov $IN01_0,x4 + umlal $ACC1,$IN01_1,${R0}[0] + and x8,x8,#0x03ffffff + + add $IN01_4,$IN01_4,$H4 + and x9,x9,#0x03ffffff + umlal $ACC3,$IN01_3,${R0}[0] + ubfx x10,x12,#14,#26 + umlal $ACC0,$IN01_3,${S2}[0] + ubfx x11,x13,#14,#26 + umlal $ACC4,$IN01_3,${R1}[0] + add x8,x8,x9,lsl#32 // bfi x8,x9,#32,#32 + umlal $ACC1,$IN01_3,${S3}[0] + fmov $IN01_1,x6 + umlal $ACC2,$IN01_3,${S4}[0] + add x12,$padbit,x12,lsr#40 + + umlal $ACC3,$IN01_4,${S4}[0] + add x13,$padbit,x13,lsr#40 + umlal $ACC0,$IN01_4,${S1}[0] + add x10,x10,x11,lsl#32 // bfi x10,x11,#32,#32 + umlal $ACC4,$IN01_4,${R0}[0] + add x12,x12,x13,lsl#32 // bfi x12,x13,#32,#32 + umlal $ACC1,$IN01_4,${S2}[0] + fmov $IN01_2,x8 + umlal $ACC2,$IN01_4,${S3}[0] + fmov $IN01_3,x10 + fmov $IN01_4,x12 + + ///////////////////////////////////////////////////////////////// + // lazy reduction as discussed in "NEON crypto" by D.J. Bernstein + // and P. Schwabe + // + // [see discussion in poly1305-armv4 module] + + ushr $T0.2d,$ACC3,#26 + xtn $H3,$ACC3 + ushr $T1.2d,$ACC0,#26 + and $ACC0,$ACC0,$MASK.2d + add $ACC4,$ACC4,$T0.2d // h3 -> h4 + bic $H3,#0xfc,lsl#24 // &=0x03ffffff + add $ACC1,$ACC1,$T1.2d // h0 -> h1 + + ushr $T0.2d,$ACC4,#26 + xtn $H4,$ACC4 + ushr $T1.2d,$ACC1,#26 + xtn $H1,$ACC1 + bic $H4,#0xfc,lsl#24 + add $ACC2,$ACC2,$T1.2d // h1 -> h2 + + add $ACC0,$ACC0,$T0.2d + shl $T0.2d,$T0.2d,#2 + shrn $T1.2s,$ACC2,#26 + xtn $H2,$ACC2 + add $ACC0,$ACC0,$T0.2d // h4 -> h0 + bic $H1,#0xfc,lsl#24 + add $H3,$H3,$T1.2s // h2 -> h3 + bic $H2,#0xfc,lsl#24 + + shrn $T0.2s,$ACC0,#26 + xtn $H0,$ACC0 + ushr $T1.2s,$H3,#26 + bic $H3,#0xfc,lsl#24 + bic $H0,#0xfc,lsl#24 + add $H1,$H1,$T0.2s // h0 -> h1 + add $H4,$H4,$T1.2s // h3 -> h4 + + b.hi .Loop_neon + +.Lskip_loop: + dup $IN23_2,${IN23_2}[0] + add $IN01_2,$IN01_2,$H2 + + //////////////////////////////////////////////////////////////// + // multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1 + + adds $len,$len,#32 + b.ne .Long_tail + + dup $IN23_2,${IN01_2}[0] + add $IN23_0,$IN01_0,$H0 + add $IN23_3,$IN01_3,$H3 + add $IN23_1,$IN01_1,$H1 + add $IN23_4,$IN01_4,$H4 + +.Long_tail: + dup $IN23_0,${IN23_0}[0] + umull2 $ACC0,$IN23_2,${S3} + umull2 $ACC3,$IN23_2,${R1} + umull2 $ACC4,$IN23_2,${R2} + umull2 $ACC2,$IN23_2,${R0} + umull2 $ACC1,$IN23_2,${S4} + + dup $IN23_1,${IN23_1}[0] + umlal2 $ACC0,$IN23_0,${R0} + umlal2 $ACC2,$IN23_0,${R2} + umlal2 $ACC3,$IN23_0,${R3} + umlal2 $ACC4,$IN23_0,${R4} + umlal2 $ACC1,$IN23_0,${R1} + + dup $IN23_3,${IN23_3}[0] + umlal2 $ACC0,$IN23_1,${S4} + umlal2 $ACC3,$IN23_1,${R2} + umlal2 $ACC2,$IN23_1,${R1} + umlal2 $ACC4,$IN23_1,${R3} + umlal2 $ACC1,$IN23_1,${R0} + + dup $IN23_4,${IN23_4}[0] + umlal2 $ACC3,$IN23_3,${R0} + umlal2 $ACC4,$IN23_3,${R1} + umlal2 $ACC0,$IN23_3,${S2} + umlal2 $ACC1,$IN23_3,${S3} + umlal2 $ACC2,$IN23_3,${S4} + + umlal2 $ACC3,$IN23_4,${S4} + umlal2 $ACC0,$IN23_4,${S1} + umlal2 $ACC4,$IN23_4,${R0} + umlal2 $ACC1,$IN23_4,${S2} + umlal2 $ACC2,$IN23_4,${S3} + + b.eq .Lshort_tail + + //////////////////////////////////////////////////////////////// + // (hash+inp[0:1])*r^4:r^3 and accumulate + + add $IN01_0,$IN01_0,$H0 + umlal $ACC3,$IN01_2,${R1} + umlal $ACC0,$IN01_2,${S3} + umlal $ACC4,$IN01_2,${R2} + umlal $ACC1,$IN01_2,${S4} + umlal $ACC2,$IN01_2,${R0} + + add $IN01_1,$IN01_1,$H1 + umlal $ACC3,$IN01_0,${R3} + umlal $ACC0,$IN01_0,${R0} + umlal $ACC4,$IN01_0,${R4} + umlal $ACC1,$IN01_0,${R1} + umlal $ACC2,$IN01_0,${R2} + + add $IN01_3,$IN01_3,$H3 + umlal $ACC3,$IN01_1,${R2} + umlal $ACC0,$IN01_1,${S4} + umlal $ACC4,$IN01_1,${R3} + umlal $ACC1,$IN01_1,${R0} + umlal $ACC2,$IN01_1,${R1} + + add $IN01_4,$IN01_4,$H4 + umlal $ACC3,$IN01_3,${R0} + umlal $ACC0,$IN01_3,${S2} + umlal $ACC4,$IN01_3,${R1} + umlal $ACC1,$IN01_3,${S3} + umlal $ACC2,$IN01_3,${S4} + + umlal $ACC3,$IN01_4,${S4} + umlal $ACC0,$IN01_4,${S1} + umlal $ACC4,$IN01_4,${R0} + umlal $ACC1,$IN01_4,${S2} + umlal $ACC2,$IN01_4,${S3} + +.Lshort_tail: + //////////////////////////////////////////////////////////////// + // horizontal add + + addp $ACC3,$ACC3,$ACC3 + ldp d8,d9,[sp,#2*__SIZEOF_POINTER__+0] // meet ABI requirements + addp $ACC0,$ACC0,$ACC0 + ldp d10,d11,[sp,#2*__SIZEOF_POINTER__+16] + addp $ACC4,$ACC4,$ACC4 + ldp d12,d13,[sp,#2*__SIZEOF_POINTER__+32] + addp $ACC1,$ACC1,$ACC1 + ldp d14,d15,[sp,#2*__SIZEOF_POINTER__+48] + addp $ACC2,$ACC2,$ACC2 + ldr c30,[csp,#__SIZEOF_POINTER__] + + //////////////////////////////////////////////////////////////// + // lazy reduction, but without narrowing + + ushr $T0.2d,$ACC3,#26 + and $ACC3,$ACC3,$MASK.2d + ushr $T1.2d,$ACC0,#26 + and $ACC0,$ACC0,$MASK.2d + + add $ACC4,$ACC4,$T0.2d // h3 -> h4 + add $ACC1,$ACC1,$T1.2d // h0 -> h1 + + ushr $T0.2d,$ACC4,#26 + and $ACC4,$ACC4,$MASK.2d + ushr $T1.2d,$ACC1,#26 + and $ACC1,$ACC1,$MASK.2d + add $ACC2,$ACC2,$T1.2d // h1 -> h2 + + add $ACC0,$ACC0,$T0.2d + shl $T0.2d,$T0.2d,#2 + ushr $T1.2d,$ACC2,#26 + and $ACC2,$ACC2,$MASK.2d + add $ACC0,$ACC0,$T0.2d // h4 -> h0 + add $ACC3,$ACC3,$T1.2d // h2 -> h3 + + ushr $T0.2d,$ACC0,#26 + and $ACC0,$ACC0,$MASK.2d + ushr $T1.2d,$ACC3,#26 + and $ACC3,$ACC3,$MASK.2d + add $ACC1,$ACC1,$T0.2d // h0 -> h1 + add $ACC4,$ACC4,$T1.2d // h3 -> h4 + + //////////////////////////////////////////////////////////////// + // write the result, can be partially reduced + + st4 {$ACC0,$ACC1,$ACC2,$ACC3}[0],[$ctx],#16 + mov x4,#1 + st1 {$ACC4}[0],[$ctx] + str x4,[$ctx,#8] // set is_base2_26 + + ldr c29,[csp],#2*__SIZEOF_POINTER__+64 + .inst 0xd50323bf // autiasp + ret +.size poly1305_blocks_neon,.-poly1305_blocks_neon + +.align 5 +.Lzeros: +.long 0,0,0,0,0,0,0,0 +.asciz "Poly1305 for ARMv8, CRYPTOGAMS by \@dot-asm" +.align 2 +#if !defined(__KERNEL__) && !defined(_WIN64) +.comm OPENSSL_armcap_P,4,4 +.hidden OPENSSL_armcap_P +#endif +___ + +foreach (split("\n",$code)) { + s/\b(shrn\s+v[0-9]+)\.[24]d/$1.2s/ or + s/\b(fmov\s+)v([0-9]+)[^,]*,\s*x([0-9]+)/$1d$2,x$3/ or + (m/\bdup\b/ and (s/\.[24]s/.2d/g or 1)) or + (m/\b(eor|and)/ and (s/\.[248][sdh]/.16b/g or 1)) or + (m/\bum(ul|la)l\b/ and (s/\.4s/.2s/g or 1)) or + (m/\bum(ul|la)l2\b/ and (s/\.2s/.4s/g or 1)) or + (m/\bst[1-4]\s+{[^}]+}\[/ and (s/\.[24]d/.s/g or 1)); + + s/\.[124]([sd])\[/.$1\[/; + s/([cw])#x([0-9]+)/$1$2/g; + + print $_,"\n"; +} +close STDOUT; diff --git a/cbits/asm/poly1305-x86_64-elf.S b/cbits/asm/poly1305-x86_64-elf.S new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-x86_64-elf.S @@ -0,0 +1,2033 @@ +.text + + + +.globl crypton_poly1305_asm_init +.hidden crypton_poly1305_asm_init +.globl crypton_poly1305_asm_blocks +.hidden crypton_poly1305_asm_blocks +.globl crypton_poly1305_asm_emit +.hidden crypton_poly1305_asm_emit + +.type crypton_poly1305_asm_init,@function +.align 32 +crypton_poly1305_asm_init: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + xorq %rax,%rax + movq %rax,0(%rdi) + movq %rax,8(%rdi) + movq %rax,16(%rdi) + + cmpq $0,%rsi + je .Lno_key + + movq $0x0ffffffc0fffffff,%rax + leaq -3(%rax),%rcx + andq 0(%rsi),%rax + andq 8(%rsi),%rcx + movq %rax,24(%rdi) + movq %rcx,32(%rdi) + movl $-1,48(%rdi) + leaq crypton_poly1305_asm_blocks(%rip),%r10 + leaq crypton_poly1305_asm_emit(%rip),%r11 + movq crypton_ia32cap_P+4(%rip),%r9 + leaq crypton_poly1305_asm_blocks_avx(%rip),%rax + btq $28,%r9 + cmovcq %rax,%r10 + leaq crypton_poly1305_asm_blocks_avx2(%rip),%rax + btq $37,%r9 + cmovcq %rax,%r10 + movq %r10,0(%rdx) + movq %r11,8(%rdx) + movl $1,%eax +.Lno_key: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_poly1305_asm_init,.-crypton_poly1305_asm_init + +.type crypton_poly1305_asm_blocks,@function +.align 32 +crypton_poly1305_asm_blocks: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +.Lblocks: + shrq $4,%rdx + jz .Lno_data + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lblocks_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movq 16(%rdi),%rbp + + movl %r14d,%eax + movl 4(%rdi),%edx + movl %ebx,%r8d + movl 12(%rdi),%r10d + movl %ebp,%r12d + + shlq $26,%rdx + movq %r8,%r9 + shlq $52,%r8 + addq %rdx,%rax + shrq $12,%r9 + addq %rax,%r8 + adcq $0,%r9 + + shlq $14,%r10 + movq %r12,%rax + shrq $24,%r12 + addq %r10,%r9 + shlq $40,%rax + addq %rax,%r9 + adcq $0,%r12 + + cmpq $4,%rbp + + cmovaq %r8,%r14 + cmovaq %r9,%rbx + cmovaq %r12,%rbp + + movq %r13,%r12 + shrq $2,%r13 + movq %r12,%rax + addq %r12,%r13 + jmp .Loop + +.align 32 +.Loop: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + mulq %r14 + movq %rax,%r9 + movq %r11,%rax + movq %rdx,%r10 + + mulq %r14 + movq %rax,%r14 + movq %r11,%rax + movq %rdx,%r8 + + mulq %rbx + addq %rax,%r9 + movq %r13,%rax + adcq %rdx,%r10 + + mulq %rbx + movq %rbp,%rbx + addq %rax,%r14 + adcq %rdx,%r8 + + imulq %r13,%rbx + addq %rbx,%r9 + movq %r8,%rbx + adcq $0,%r10 + + imulq %r11,%rbp + addq %r9,%rbx + movq $-4,%rax + adcq %rbp,%r10 + + andq %r10,%rax + movq %r10,%rbp + shrq $2,%r10 + andq $3,%rbp + addq %r10,%rax + addq %rax,%r14 + adcq $0,%rbx + adcq $0,%rbp + movq %r12,%rax + decq %r15 + jnz .Loop + + movq %r14,0(%rdi) + movq %rbx,8(%rdi) + movq %rbp,16(%rdi) + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lno_data: +.Lblocks_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_poly1305_asm_blocks,.-crypton_poly1305_asm_blocks + +.type crypton_poly1305_asm_emit,@function +.align 32 +crypton_poly1305_asm_emit: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + movl 0(%rdi),%eax + movl 4(%rdi),%ecx + movl 8(%rdi),%r8d + movl 12(%rdi),%r11d + movl 16(%rdi),%r10d + + shlq $26,%rcx + movq %r8,%r9 + shlq $52,%r8 + addq %rcx,%rax + shrq $12,%r9 + addq %rax,%r8 + adcq $0,%r9 + + shlq $14,%r11 + movq %r10,%rax + shrq $24,%r10 + addq %r11,%r9 + movq 0(%rdi),%rcx + shlq $40,%rax + movq 8(%rdi),%r11 + addq %rax,%r9 + movq 16(%rdi),%rax + adcq $0,%r10 + + cmpq $4,%rax + + cmovbeq %rcx,%r8 + cmovbeq %r11,%r9 + cmovbeq %rax,%r10 + + movq %r8,%rax + addq $5,%r8 + movq %r9,%rcx + adcq $0,%r9 + adcq $0,%r10 + shrq $2,%r10 + cmovnzq %r8,%rax + cmovnzq %r9,%rcx + + addq 0(%rdx),%rax + adcq 8(%rdx),%rcx + movq %rax,0(%rsi) + movq %rcx,8(%rsi) + + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_poly1305_asm_emit,.-crypton_poly1305_asm_emit +.type __crypton_poly1305_asm_block,@function +.align 32 +__crypton_poly1305_asm_block: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + mulq %r14 + movq %rax,%r9 + movq %r11,%rax + movq %rdx,%r10 + + mulq %r14 + movq %rax,%r14 + movq %r11,%rax + movq %rdx,%r8 + + mulq %rbx + addq %rax,%r9 + movq %r13,%rax + adcq %rdx,%r10 + + mulq %rbx + movq %rbp,%rbx + addq %rax,%r14 + adcq %rdx,%r8 + + imulq %r13,%rbx + addq %rbx,%r9 + movq %r8,%rbx + adcq $0,%r10 + + imulq %r11,%rbp + addq %r9,%rbx + movq $-4,%rax + adcq %rbp,%r10 + + andq %r10,%rax + movq %r10,%rbp + shrq $2,%r10 + andq $3,%rbp + addq %r10,%rax + addq %rax,%r14 + adcq $0,%rbx + adcq $0,%rbp + .byte 0xf3,0xc3 +.cfi_endproc +.size __crypton_poly1305_asm_block,.-__crypton_poly1305_asm_block + +.type __crypton_poly1305_asm_init_avx,@function +.align 32 +__crypton_poly1305_asm_init_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + cmpl $-1,48(%rdi) + jne .Ldone_init_avx + + movq %r11,%r14 + movq %r12,%rbx + xorq %rbp,%rbp + + leaq 48+64(%rdi),%rdi + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + movq %r14,%r8 + andl %r14d,%eax + movq %r11,%r9 + andl %r11d,%edx + movl %eax,-64(%rdi) + shrq $26,%r8 + movl %edx,-60(%rdi) + shrq $26,%r9 + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + andl %r8d,%eax + andl %r9d,%edx + movl %eax,-48(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,-44(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,-32(%rdi) + shrq $26,%r8 + movl %edx,-28(%rdi) + shrq $26,%r9 + + movq %rbx,%rax + movq %r12,%rdx + shlq $12,%rax + shlq $12,%rdx + orq %r8,%rax + orq %r9,%rdx + andl $0x3ffffff,%eax + andl $0x3ffffff,%edx + movl %eax,-16(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,-12(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,0(%rdi) + movq %rbx,%r8 + movl %edx,4(%rdi) + movq %r12,%r9 + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + shrq $14,%r8 + shrq $14,%r9 + andl %r8d,%eax + andl %r9d,%edx + movl %eax,16(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,20(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,32(%rdi) + shrq $26,%r8 + movl %edx,36(%rdi) + shrq $26,%r9 + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,48(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r9d,52(%rdi) + leaq (%r9,%r9,4),%r9 + movl %r8d,64(%rdi) + movl %r9d,68(%rdi) + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movq %r14,%r8 + andl %r14d,%eax + shrq $26,%r8 + movl %eax,-52(%rdi) + + movl $0x3ffffff,%edx + andl %r8d,%edx + movl %edx,-36(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,-20(%rdi) + + movq %rbx,%rax + shlq $12,%rax + orq %r8,%rax + andl $0x3ffffff,%eax + movl %eax,-4(%rdi) + leal (%rax,%rax,4),%eax + movq %rbx,%r8 + movl %eax,12(%rdi) + + movl $0x3ffffff,%edx + shrq $14,%r8 + andl %r8d,%edx + movl %edx,28(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,44(%rdi) + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,60(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r8d,76(%rdi) + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movq %r14,%r8 + andl %r14d,%eax + shrq $26,%r8 + movl %eax,-56(%rdi) + + movl $0x3ffffff,%edx + andl %r8d,%edx + movl %edx,-40(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,-24(%rdi) + + movq %rbx,%rax + shlq $12,%rax + orq %r8,%rax + andl $0x3ffffff,%eax + movl %eax,-8(%rdi) + leal (%rax,%rax,4),%eax + movq %rbx,%r8 + movl %eax,8(%rdi) + + movl $0x3ffffff,%edx + shrq $14,%r8 + andl %r8d,%edx + movl %edx,24(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,40(%rdi) + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,56(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r8d,72(%rdi) + + leaq -48-64(%rdi),%rdi +.Ldone_init_avx: + .byte 0xf3,0xc3 +.cfi_endproc +.size __crypton_poly1305_asm_init_avx,.-__crypton_poly1305_asm_init_avx + +.type crypton_poly1305_asm_blocks_avx,@function +.align 32 +crypton_poly1305_asm_blocks_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + movl 20(%rdi),%r8d + cmpq $128,%rdx + jb .Lblocks + + andq $-16,%rdx + + vzeroupper + + testl %r8d,%r8d + jz .Lbase2_64_avx + + testq $31,%rdx + jz .Leven_avx + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lblocks_avx_body: + + movq %rdx,%r15 + + movq 0(%rdi),%r8 + movq 8(%rdi),%r9 + movl 16(%rdi),%ebp + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + + movl %r8d,%r14d + andq $-2147483648,%r8 + movq %r9,%r12 + movl %r9d,%ebx + andq $-2147483648,%r9 + + shrq $6,%r8 + shlq $52,%r12 + addq %r8,%r14 + shrq $12,%rbx + shrq $18,%r9 + addq %r12,%r14 + adcq %r9,%rbx + + movq %rbp,%r8 + shlq $40,%r8 + shrq $24,%rbp + addq %r8,%rbx + adcq $0,%rbp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + + call __crypton_poly1305_asm_block + + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r11 + movq %rbx,%r12 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r11 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r11,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r12 + andq $0x3ffffff,%rbx + orq %r12,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + + leaq -16(%r15),%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lblocks_avx_epilogue: + jmp .Ldo_avx +.cfi_endproc + +.align 32 +.Lbase2_64_avx: +.cfi_startproc + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lbase2_64_avx_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movl 16(%rdi),%ebp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + testq $31,%rdx + jz .Linit_avx + + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + +.Linit_avx: + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r8 + movq %rbx,%r9 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r8 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r8,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r9 + andq $0x3ffffff,%rbx + orq %r9,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + movl $1,20(%rdi) + + call __crypton_poly1305_asm_init_avx + + movq %r15,%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lbase2_64_avx_epilogue: + jmp .Ldo_avx +.cfi_endproc + +.align 32 +.Leven_avx: +.cfi_startproc + vmovd 0(%rdi),%xmm0 + vmovd 4(%rdi),%xmm1 + vmovd 8(%rdi),%xmm2 + vmovd 12(%rdi),%xmm3 + vmovd 16(%rdi),%xmm4 + +.Ldo_avx: + leaq -88(%rsp),%r11 +.cfi_def_cfa %r11,0x60 + subq $0x178,%rsp + subq $64,%rdx + leaq -32(%rsi),%rax + cmovcq %rax,%rsi + + vmovdqu 48(%rdi),%xmm14 + leaq 112(%rdi),%rdi + leaq .Lconst(%rip),%rcx + + + + vmovdqu 32(%rsi),%xmm5 + vmovdqu 48(%rsi),%xmm6 + vmovdqa 64(%rcx),%xmm15 + + vpsrldq $6,%xmm5,%xmm7 + vpsrldq $6,%xmm6,%xmm8 + vpunpckhqdq %xmm6,%xmm5,%xmm9 + vpunpcklqdq %xmm6,%xmm5,%xmm5 + vpunpcklqdq %xmm8,%xmm7,%xmm8 + + vpsrlq $40,%xmm9,%xmm9 + vpsrlq $26,%xmm5,%xmm6 + vpand %xmm15,%xmm5,%xmm5 + vpsrlq $4,%xmm8,%xmm7 + vpand %xmm15,%xmm6,%xmm6 + vpsrlq $30,%xmm8,%xmm8 + vpand %xmm15,%xmm7,%xmm7 + vpand %xmm15,%xmm8,%xmm8 + vpor 32(%rcx),%xmm9,%xmm9 + + jbe .Lskip_loop_avx + + + vmovdqu -48(%rdi),%xmm11 + vmovdqu -32(%rdi),%xmm12 + vpshufd $0xEE,%xmm14,%xmm13 + vpshufd $0x44,%xmm14,%xmm10 + vmovdqa %xmm13,-144(%r11) + vmovdqa %xmm10,0(%rsp) + vpshufd $0xEE,%xmm11,%xmm14 + vmovdqu -16(%rdi),%xmm10 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm14,-128(%r11) + vmovdqa %xmm11,16(%rsp) + vpshufd $0xEE,%xmm12,%xmm13 + vmovdqu 0(%rdi),%xmm11 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm13,-112(%r11) + vmovdqa %xmm12,32(%rsp) + vpshufd $0xEE,%xmm10,%xmm14 + vmovdqu 16(%rdi),%xmm12 + vpshufd $0x44,%xmm10,%xmm10 + vmovdqa %xmm14,-96(%r11) + vmovdqa %xmm10,48(%rsp) + vpshufd $0xEE,%xmm11,%xmm13 + vmovdqu 32(%rdi),%xmm10 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm13,-80(%r11) + vmovdqa %xmm11,64(%rsp) + vpshufd $0xEE,%xmm12,%xmm14 + vmovdqu 48(%rdi),%xmm11 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm14,-64(%r11) + vmovdqa %xmm12,80(%rsp) + vpshufd $0xEE,%xmm10,%xmm13 + vmovdqu 64(%rdi),%xmm12 + vpshufd $0x44,%xmm10,%xmm10 + vmovdqa %xmm13,-48(%r11) + vmovdqa %xmm10,96(%rsp) + vpshufd $0xEE,%xmm11,%xmm14 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm14,-32(%r11) + vmovdqa %xmm11,112(%rsp) + vpshufd $0xEE,%xmm12,%xmm13 + vmovdqa 0(%rsp),%xmm14 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm13,-16(%r11) + vmovdqa %xmm12,128(%rsp) + + jmp .Loop_avx + +.align 32 +.Loop_avx: + + + + + + + + + + + + + + + + + + + + + vpmuludq %xmm5,%xmm14,%xmm10 + vpmuludq %xmm6,%xmm14,%xmm11 + vmovdqa %xmm2,32(%r11) + vpmuludq %xmm7,%xmm14,%xmm12 + vmovdqa 16(%rsp),%xmm2 + vpmuludq %xmm8,%xmm14,%xmm13 + vpmuludq %xmm9,%xmm14,%xmm14 + + vmovdqa %xmm0,0(%r11) + vpmuludq 32(%rsp),%xmm9,%xmm0 + vmovdqa %xmm1,16(%r11) + vpmuludq %xmm8,%xmm2,%xmm1 + vpaddq %xmm0,%xmm10,%xmm10 + vpaddq %xmm1,%xmm14,%xmm14 + vmovdqa %xmm3,48(%r11) + vpmuludq %xmm7,%xmm2,%xmm0 + vpmuludq %xmm6,%xmm2,%xmm1 + vpaddq %xmm0,%xmm13,%xmm13 + vmovdqa 48(%rsp),%xmm3 + vpaddq %xmm1,%xmm12,%xmm12 + vmovdqa %xmm4,64(%r11) + vpmuludq %xmm5,%xmm2,%xmm2 + vpmuludq %xmm7,%xmm3,%xmm0 + vpaddq %xmm2,%xmm11,%xmm11 + + vmovdqa 64(%rsp),%xmm4 + vpaddq %xmm0,%xmm14,%xmm14 + vpmuludq %xmm6,%xmm3,%xmm1 + vpmuludq %xmm5,%xmm3,%xmm3 + vpaddq %xmm1,%xmm13,%xmm13 + vmovdqa 80(%rsp),%xmm2 + vpaddq %xmm3,%xmm12,%xmm12 + vpmuludq %xmm9,%xmm4,%xmm0 + vpmuludq %xmm8,%xmm4,%xmm4 + vpaddq %xmm0,%xmm11,%xmm11 + vmovdqa 96(%rsp),%xmm3 + vpaddq %xmm4,%xmm10,%xmm10 + + vmovdqa 128(%rsp),%xmm4 + vpmuludq %xmm6,%xmm2,%xmm1 + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm1,%xmm14,%xmm14 + vpaddq %xmm2,%xmm13,%xmm13 + vpmuludq %xmm9,%xmm3,%xmm0 + vpmuludq %xmm8,%xmm3,%xmm1 + vpaddq %xmm0,%xmm12,%xmm12 + vmovdqu 0(%rsi),%xmm0 + vpaddq %xmm1,%xmm11,%xmm11 + vpmuludq %xmm7,%xmm3,%xmm3 + vpmuludq %xmm7,%xmm4,%xmm7 + vpaddq %xmm3,%xmm10,%xmm10 + + vmovdqu 16(%rsi),%xmm1 + vpaddq %xmm7,%xmm11,%xmm11 + vpmuludq %xmm8,%xmm4,%xmm8 + vpmuludq %xmm9,%xmm4,%xmm9 + vpsrldq $6,%xmm0,%xmm2 + vpaddq %xmm8,%xmm12,%xmm12 + vpaddq %xmm9,%xmm13,%xmm13 + vpsrldq $6,%xmm1,%xmm3 + vpmuludq 112(%rsp),%xmm5,%xmm9 + vpmuludq %xmm6,%xmm4,%xmm5 + vpunpckhqdq %xmm1,%xmm0,%xmm4 + vpaddq %xmm9,%xmm14,%xmm14 + vmovdqa -144(%r11),%xmm9 + vpaddq %xmm5,%xmm10,%xmm10 + + vpunpcklqdq %xmm1,%xmm0,%xmm0 + vpunpcklqdq %xmm3,%xmm2,%xmm3 + + + vpsrldq $5,%xmm4,%xmm4 + vpsrlq $26,%xmm0,%xmm1 + vpand %xmm15,%xmm0,%xmm0 + vpsrlq $4,%xmm3,%xmm2 + vpand %xmm15,%xmm1,%xmm1 + vpand 0(%rcx),%xmm4,%xmm4 + vpsrlq $30,%xmm3,%xmm3 + vpand %xmm15,%xmm2,%xmm2 + vpand %xmm15,%xmm3,%xmm3 + vpor 32(%rcx),%xmm4,%xmm4 + + vpaddq 0(%r11),%xmm0,%xmm0 + vpaddq 16(%r11),%xmm1,%xmm1 + vpaddq 32(%r11),%xmm2,%xmm2 + vpaddq 48(%r11),%xmm3,%xmm3 + vpaddq 64(%r11),%xmm4,%xmm4 + + leaq 32(%rsi),%rax + leaq 64(%rsi),%rsi + subq $64,%rdx + cmovcq %rax,%rsi + + + + + + + + + + + vpmuludq %xmm0,%xmm9,%xmm5 + vpmuludq %xmm1,%xmm9,%xmm6 + vpaddq %xmm5,%xmm10,%xmm10 + vpaddq %xmm6,%xmm11,%xmm11 + vmovdqa -128(%r11),%xmm7 + vpmuludq %xmm2,%xmm9,%xmm5 + vpmuludq %xmm3,%xmm9,%xmm6 + vpaddq %xmm5,%xmm12,%xmm12 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm4,%xmm9,%xmm9 + vpmuludq -112(%r11),%xmm4,%xmm5 + vpaddq %xmm9,%xmm14,%xmm14 + + vpaddq %xmm5,%xmm10,%xmm10 + vpmuludq %xmm2,%xmm7,%xmm6 + vpmuludq %xmm3,%xmm7,%xmm5 + vpaddq %xmm6,%xmm13,%xmm13 + vmovdqa -96(%r11),%xmm8 + vpaddq %xmm5,%xmm14,%xmm14 + vpmuludq %xmm1,%xmm7,%xmm6 + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm6,%xmm12,%xmm12 + vpaddq %xmm7,%xmm11,%xmm11 + + vmovdqa -80(%r11),%xmm9 + vpmuludq %xmm2,%xmm8,%xmm5 + vpmuludq %xmm1,%xmm8,%xmm6 + vpaddq %xmm5,%xmm14,%xmm14 + vpaddq %xmm6,%xmm13,%xmm13 + vmovdqa -64(%r11),%xmm7 + vpmuludq %xmm0,%xmm8,%xmm8 + vpmuludq %xmm4,%xmm9,%xmm5 + vpaddq %xmm8,%xmm12,%xmm12 + vpaddq %xmm5,%xmm11,%xmm11 + vmovdqa -48(%r11),%xmm8 + vpmuludq %xmm3,%xmm9,%xmm9 + vpmuludq %xmm1,%xmm7,%xmm6 + vpaddq %xmm9,%xmm10,%xmm10 + + vmovdqa -16(%r11),%xmm9 + vpaddq %xmm6,%xmm14,%xmm14 + vpmuludq %xmm0,%xmm7,%xmm7 + vpmuludq %xmm4,%xmm8,%xmm5 + vpaddq %xmm7,%xmm13,%xmm13 + vpaddq %xmm5,%xmm12,%xmm12 + vmovdqu 32(%rsi),%xmm5 + vpmuludq %xmm3,%xmm8,%xmm7 + vpmuludq %xmm2,%xmm8,%xmm8 + vpaddq %xmm7,%xmm11,%xmm11 + vmovdqu 48(%rsi),%xmm6 + vpaddq %xmm8,%xmm10,%xmm10 + + vpmuludq %xmm2,%xmm9,%xmm2 + vpmuludq %xmm3,%xmm9,%xmm3 + vpsrldq $6,%xmm5,%xmm7 + vpaddq %xmm2,%xmm11,%xmm11 + vpmuludq %xmm4,%xmm9,%xmm4 + vpsrldq $6,%xmm6,%xmm8 + vpaddq %xmm3,%xmm12,%xmm2 + vpaddq %xmm4,%xmm13,%xmm3 + vpmuludq -32(%r11),%xmm0,%xmm4 + vpmuludq %xmm1,%xmm9,%xmm0 + vpunpckhqdq %xmm6,%xmm5,%xmm9 + vpaddq %xmm4,%xmm14,%xmm4 + vpaddq %xmm0,%xmm10,%xmm0 + + vpunpcklqdq %xmm6,%xmm5,%xmm5 + vpunpcklqdq %xmm8,%xmm7,%xmm8 + + + vpsrldq $5,%xmm9,%xmm9 + vpsrlq $26,%xmm5,%xmm6 + vmovdqa 0(%rsp),%xmm14 + vpand %xmm15,%xmm5,%xmm5 + vpsrlq $4,%xmm8,%xmm7 + vpand %xmm15,%xmm6,%xmm6 + vpand 0(%rcx),%xmm9,%xmm9 + vpsrlq $30,%xmm8,%xmm8 + vpand %xmm15,%xmm7,%xmm7 + vpand %xmm15,%xmm8,%xmm8 + vpor 32(%rcx),%xmm9,%xmm9 + + + + + + vpsrlq $26,%xmm3,%xmm13 + vpand %xmm15,%xmm3,%xmm3 + vpaddq %xmm13,%xmm4,%xmm4 + + vpsrlq $26,%xmm0,%xmm10 + vpand %xmm15,%xmm0,%xmm0 + vpaddq %xmm10,%xmm11,%xmm1 + + vpsrlq $26,%xmm4,%xmm10 + vpand %xmm15,%xmm4,%xmm4 + + vpsrlq $26,%xmm1,%xmm11 + vpand %xmm15,%xmm1,%xmm1 + vpaddq %xmm11,%xmm2,%xmm2 + + vpaddq %xmm10,%xmm0,%xmm0 + vpsllq $2,%xmm10,%xmm10 + vpaddq %xmm10,%xmm0,%xmm0 + + vpsrlq $26,%xmm2,%xmm12 + vpand %xmm15,%xmm2,%xmm2 + vpaddq %xmm12,%xmm3,%xmm3 + + vpsrlq $26,%xmm0,%xmm10 + vpand %xmm15,%xmm0,%xmm0 + vpaddq %xmm10,%xmm1,%xmm1 + + vpsrlq $26,%xmm3,%xmm13 + vpand %xmm15,%xmm3,%xmm3 + vpaddq %xmm13,%xmm4,%xmm4 + + ja .Loop_avx + +.Lskip_loop_avx: + + + + vpshufd $0x10,%xmm14,%xmm14 + addq $32,%rdx + jnz .Long_tail_avx + + vpaddq %xmm2,%xmm7,%xmm7 + vpaddq %xmm0,%xmm5,%xmm5 + vpaddq %xmm1,%xmm6,%xmm6 + vpaddq %xmm3,%xmm8,%xmm8 + vpaddq %xmm4,%xmm9,%xmm9 + +.Long_tail_avx: + vmovdqa %xmm2,32(%r11) + vmovdqa %xmm0,0(%r11) + vmovdqa %xmm1,16(%r11) + vmovdqa %xmm3,48(%r11) + vmovdqa %xmm4,64(%r11) + + + + + + + + vpmuludq %xmm7,%xmm14,%xmm12 + vpmuludq %xmm5,%xmm14,%xmm10 + vpshufd $0x10,-48(%rdi),%xmm2 + vpmuludq %xmm6,%xmm14,%xmm11 + vpmuludq %xmm8,%xmm14,%xmm13 + vpmuludq %xmm9,%xmm14,%xmm14 + + vpmuludq %xmm8,%xmm2,%xmm0 + vpaddq %xmm0,%xmm14,%xmm14 + vpshufd $0x10,-32(%rdi),%xmm3 + vpmuludq %xmm7,%xmm2,%xmm1 + vpaddq %xmm1,%xmm13,%xmm13 + vpshufd $0x10,-16(%rdi),%xmm4 + vpmuludq %xmm6,%xmm2,%xmm0 + vpaddq %xmm0,%xmm12,%xmm12 + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm2,%xmm11,%xmm11 + vpmuludq %xmm9,%xmm3,%xmm3 + vpaddq %xmm3,%xmm10,%xmm10 + + vpshufd $0x10,0(%rdi),%xmm2 + vpmuludq %xmm7,%xmm4,%xmm1 + vpaddq %xmm1,%xmm14,%xmm14 + vpmuludq %xmm6,%xmm4,%xmm0 + vpaddq %xmm0,%xmm13,%xmm13 + vpshufd $0x10,16(%rdi),%xmm3 + vpmuludq %xmm5,%xmm4,%xmm4 + vpaddq %xmm4,%xmm12,%xmm12 + vpmuludq %xmm9,%xmm2,%xmm1 + vpaddq %xmm1,%xmm11,%xmm11 + vpshufd $0x10,32(%rdi),%xmm4 + vpmuludq %xmm8,%xmm2,%xmm2 + vpaddq %xmm2,%xmm10,%xmm10 + + vpmuludq %xmm6,%xmm3,%xmm0 + vpaddq %xmm0,%xmm14,%xmm14 + vpmuludq %xmm5,%xmm3,%xmm3 + vpaddq %xmm3,%xmm13,%xmm13 + vpshufd $0x10,48(%rdi),%xmm2 + vpmuludq %xmm9,%xmm4,%xmm1 + vpaddq %xmm1,%xmm12,%xmm12 + vpshufd $0x10,64(%rdi),%xmm3 + vpmuludq %xmm8,%xmm4,%xmm0 + vpaddq %xmm0,%xmm11,%xmm11 + vpmuludq %xmm7,%xmm4,%xmm4 + vpaddq %xmm4,%xmm10,%xmm10 + + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm2,%xmm14,%xmm14 + vpmuludq %xmm9,%xmm3,%xmm1 + vpaddq %xmm1,%xmm13,%xmm13 + vpmuludq %xmm8,%xmm3,%xmm0 + vpaddq %xmm0,%xmm12,%xmm12 + vpmuludq %xmm7,%xmm3,%xmm1 + vpaddq %xmm1,%xmm11,%xmm11 + vpmuludq %xmm6,%xmm3,%xmm3 + vpaddq %xmm3,%xmm10,%xmm10 + + jz .Lshort_tail_avx + + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + + vpsrldq $6,%xmm0,%xmm2 + vpsrldq $6,%xmm1,%xmm3 + vpunpckhqdq %xmm1,%xmm0,%xmm4 + vpunpcklqdq %xmm1,%xmm0,%xmm0 + vpunpcklqdq %xmm3,%xmm2,%xmm3 + + vpsrlq $40,%xmm4,%xmm4 + vpsrlq $26,%xmm0,%xmm1 + vpand %xmm15,%xmm0,%xmm0 + vpsrlq $4,%xmm3,%xmm2 + vpand %xmm15,%xmm1,%xmm1 + vpsrlq $30,%xmm3,%xmm3 + vpand %xmm15,%xmm2,%xmm2 + vpand %xmm15,%xmm3,%xmm3 + vpor 32(%rcx),%xmm4,%xmm4 + + vpshufd $0x32,-64(%rdi),%xmm9 + vpaddq 0(%r11),%xmm0,%xmm0 + vpaddq 16(%r11),%xmm1,%xmm1 + vpaddq 32(%r11),%xmm2,%xmm2 + vpaddq 48(%r11),%xmm3,%xmm3 + vpaddq 64(%r11),%xmm4,%xmm4 + + + + + vpmuludq %xmm0,%xmm9,%xmm5 + vpaddq %xmm5,%xmm10,%xmm10 + vpmuludq %xmm1,%xmm9,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpmuludq %xmm2,%xmm9,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpshufd $0x32,-48(%rdi),%xmm7 + vpmuludq %xmm3,%xmm9,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm4,%xmm9,%xmm9 + vpaddq %xmm9,%xmm14,%xmm14 + + vpmuludq %xmm3,%xmm7,%xmm5 + vpaddq %xmm5,%xmm14,%xmm14 + vpshufd $0x32,-32(%rdi),%xmm8 + vpmuludq %xmm2,%xmm7,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpshufd $0x32,-16(%rdi),%xmm9 + vpmuludq %xmm1,%xmm7,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm7,%xmm11,%xmm11 + vpmuludq %xmm4,%xmm8,%xmm8 + vpaddq %xmm8,%xmm10,%xmm10 + + vpshufd $0x32,0(%rdi),%xmm7 + vpmuludq %xmm2,%xmm9,%xmm6 + vpaddq %xmm6,%xmm14,%xmm14 + vpmuludq %xmm1,%xmm9,%xmm5 + vpaddq %xmm5,%xmm13,%xmm13 + vpshufd $0x32,16(%rdi),%xmm8 + vpmuludq %xmm0,%xmm9,%xmm9 + vpaddq %xmm9,%xmm12,%xmm12 + vpmuludq %xmm4,%xmm7,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpshufd $0x32,32(%rdi),%xmm9 + vpmuludq %xmm3,%xmm7,%xmm7 + vpaddq %xmm7,%xmm10,%xmm10 + + vpmuludq %xmm1,%xmm8,%xmm5 + vpaddq %xmm5,%xmm14,%xmm14 + vpmuludq %xmm0,%xmm8,%xmm8 + vpaddq %xmm8,%xmm13,%xmm13 + vpshufd $0x32,48(%rdi),%xmm7 + vpmuludq %xmm4,%xmm9,%xmm6 + vpaddq %xmm6,%xmm12,%xmm12 + vpshufd $0x32,64(%rdi),%xmm8 + vpmuludq %xmm3,%xmm9,%xmm5 + vpaddq %xmm5,%xmm11,%xmm11 + vpmuludq %xmm2,%xmm9,%xmm9 + vpaddq %xmm9,%xmm10,%xmm10 + + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm7,%xmm14,%xmm14 + vpmuludq %xmm4,%xmm8,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm3,%xmm8,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpmuludq %xmm2,%xmm8,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpmuludq %xmm1,%xmm8,%xmm8 + vpaddq %xmm8,%xmm10,%xmm10 + +.Lshort_tail_avx: + + + + vpsrldq $8,%xmm14,%xmm9 + vpsrldq $8,%xmm13,%xmm8 + vpsrldq $8,%xmm11,%xmm6 + vpsrldq $8,%xmm10,%xmm5 + vpsrldq $8,%xmm12,%xmm7 + vpaddq %xmm8,%xmm13,%xmm13 + vpaddq %xmm9,%xmm14,%xmm14 + vpaddq %xmm5,%xmm10,%xmm10 + vpaddq %xmm6,%xmm11,%xmm11 + vpaddq %xmm7,%xmm12,%xmm12 + + + + + vpsrlq $26,%xmm13,%xmm3 + vpand %xmm15,%xmm13,%xmm13 + vpaddq %xmm3,%xmm14,%xmm14 + + vpsrlq $26,%xmm10,%xmm0 + vpand %xmm15,%xmm10,%xmm10 + vpaddq %xmm0,%xmm11,%xmm11 + + vpsrlq $26,%xmm14,%xmm4 + vpand %xmm15,%xmm14,%xmm14 + + vpsrlq $26,%xmm11,%xmm1 + vpand %xmm15,%xmm11,%xmm11 + vpaddq %xmm1,%xmm12,%xmm12 + + vpaddq %xmm4,%xmm10,%xmm10 + vpsllq $2,%xmm4,%xmm4 + vpaddq %xmm4,%xmm10,%xmm10 + + vpsrlq $26,%xmm12,%xmm2 + vpand %xmm15,%xmm12,%xmm12 + vpaddq %xmm2,%xmm13,%xmm13 + + vpsrlq $26,%xmm10,%xmm0 + vpand %xmm15,%xmm10,%xmm10 + vpaddq %xmm0,%xmm11,%xmm11 + + vpsrlq $26,%xmm13,%xmm3 + vpand %xmm15,%xmm13,%xmm13 + vpaddq %xmm3,%xmm14,%xmm14 + + vmovd %xmm10,-112(%rdi) + vmovd %xmm11,-108(%rdi) + vmovd %xmm12,-104(%rdi) + vmovd %xmm13,-100(%rdi) + vmovd %xmm14,-96(%rdi) + leaq 88(%r11),%rsp +.cfi_def_cfa %rsp,8 + vzeroupper + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_poly1305_asm_blocks_avx,.-crypton_poly1305_asm_blocks_avx +.type crypton_poly1305_asm_blocks_avx2,@function +.align 32 +crypton_poly1305_asm_blocks_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + movl 20(%rdi),%r8d + cmpq $128,%rdx + jb .Lblocks + + andq $-16,%rdx + + vzeroupper + + testl %r8d,%r8d + jz .Lbase2_64_avx2 + + testq $63,%rdx + jz .Leven_avx2 + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lblocks_avx2_body: + + movq %rdx,%r15 + + movq 0(%rdi),%r8 + movq 8(%rdi),%r9 + movl 16(%rdi),%ebp + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + + movl %r8d,%r14d + andq $-2147483648,%r8 + movq %r9,%r12 + movl %r9d,%ebx + andq $-2147483648,%r9 + + shrq $6,%r8 + shlq $52,%r12 + addq %r8,%r14 + shrq $12,%rbx + shrq $18,%r9 + addq %r12,%r14 + adcq %r9,%rbx + + movq %rbp,%r8 + shlq $40,%r8 + shrq $24,%rbp + addq %r8,%rbx + adcq $0,%rbp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + +.Lbase2_26_pre_avx2: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + movq %r12,%rax + + testq $63,%r15 + jnz .Lbase2_26_pre_avx2 + + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r11 + movq %rbx,%r12 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r11 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r11,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r12 + andq $0x3ffffff,%rbx + orq %r12,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + + movq %r15,%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lblocks_avx2_epilogue: + jmp .Ldo_avx2 +.cfi_endproc + +.align 32 +.Lbase2_64_avx2: +.cfi_startproc + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lbase2_64_avx2_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movl 16(%rdi),%ebp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + testq $63,%rdx + jz .Linit_avx2 + +.Lbase2_64_pre_avx2: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + movq %r12,%rax + + testq $63,%r15 + jnz .Lbase2_64_pre_avx2 + +.Linit_avx2: + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r8 + movq %rbx,%r9 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r8 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r8,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r9 + andq $0x3ffffff,%rbx + orq %r9,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + movl $1,20(%rdi) + + call __crypton_poly1305_asm_init_avx + + movq %r15,%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lbase2_64_avx2_epilogue: + jmp .Ldo_avx2 +.cfi_endproc + +.align 32 +.Leven_avx2: +.cfi_startproc + vmovd 0(%rdi),%xmm0 + vmovd 4(%rdi),%xmm1 + vmovd 8(%rdi),%xmm2 + vmovd 12(%rdi),%xmm3 + vmovd 16(%rdi),%xmm4 + +.Ldo_avx2: + leaq -8(%rsp),%r11 +.cfi_def_cfa %r11,16 + subq $0x128,%rsp + leaq .Lconst(%rip),%rcx + leaq 48+64(%rdi),%rdi + vmovdqa 96(%rcx),%ymm7 + + + vmovdqu -64(%rdi),%xmm9 + andq $-512,%rsp + vmovdqu -48(%rdi),%xmm10 + vmovdqu -32(%rdi),%xmm6 + vmovdqu -16(%rdi),%xmm11 + vmovdqu 0(%rdi),%xmm12 + vmovdqu 16(%rdi),%xmm13 + leaq 144(%rsp),%rax + vmovdqu 32(%rdi),%xmm14 + vpermd %ymm9,%ymm7,%ymm9 + vmovdqu 48(%rdi),%xmm15 + vpermd %ymm10,%ymm7,%ymm10 + vmovdqu 64(%rdi),%xmm5 + vpermd %ymm6,%ymm7,%ymm6 + vmovdqa %ymm9,0(%rsp) + vpermd %ymm11,%ymm7,%ymm11 + vmovdqa %ymm10,32-144(%rax) + vpermd %ymm12,%ymm7,%ymm12 + vmovdqa %ymm6,64-144(%rax) + vpermd %ymm13,%ymm7,%ymm13 + vmovdqa %ymm11,96-144(%rax) + vpermd %ymm14,%ymm7,%ymm14 + vmovdqa %ymm12,128-144(%rax) + vpermd %ymm15,%ymm7,%ymm15 + vmovdqa %ymm13,160-144(%rax) + vpermd %ymm5,%ymm7,%ymm5 + vmovdqa %ymm14,192-144(%rax) + vmovdqa %ymm15,224-144(%rax) + vmovdqa %ymm5,256-144(%rax) + vmovdqa 64(%rcx),%ymm5 + + + + vmovdqu 0(%rsi),%xmm7 + vmovdqu 16(%rsi),%xmm8 + vinserti128 $1,32(%rsi),%ymm7,%ymm7 + vinserti128 $1,48(%rsi),%ymm8,%ymm8 + leaq 64(%rsi),%rsi + + vpsrldq $6,%ymm7,%ymm9 + vpsrldq $6,%ymm8,%ymm10 + vpunpckhqdq %ymm8,%ymm7,%ymm6 + vpunpcklqdq %ymm10,%ymm9,%ymm9 + vpunpcklqdq %ymm8,%ymm7,%ymm7 + + vpsrlq $30,%ymm9,%ymm10 + vpsrlq $4,%ymm9,%ymm9 + vpsrlq $26,%ymm7,%ymm8 + vpsrlq $40,%ymm6,%ymm6 + vpand %ymm5,%ymm9,%ymm9 + vpand %ymm5,%ymm7,%ymm7 + vpand %ymm5,%ymm8,%ymm8 + vpand %ymm5,%ymm10,%ymm10 + vpor 32(%rcx),%ymm6,%ymm6 + + vpaddq %ymm2,%ymm9,%ymm2 + subq $64,%rdx + jz .Ltail_avx2 + jmp .Loop_avx2 + +.align 32 +.Loop_avx2: + + + + + + + + + vpaddq %ymm0,%ymm7,%ymm0 + vmovdqa 0(%rsp),%ymm7 + vpaddq %ymm1,%ymm8,%ymm1 + vmovdqa 32(%rsp),%ymm8 + vpaddq %ymm3,%ymm10,%ymm3 + vmovdqa 96(%rsp),%ymm9 + vpaddq %ymm4,%ymm6,%ymm4 + vmovdqa 48(%rax),%ymm10 + vmovdqa 112(%rax),%ymm5 + + + + + + + + + + + + + + + + + vpmuludq %ymm2,%ymm7,%ymm13 + vpmuludq %ymm2,%ymm8,%ymm14 + vpmuludq %ymm2,%ymm9,%ymm15 + vpmuludq %ymm2,%ymm10,%ymm11 + vpmuludq %ymm2,%ymm5,%ymm12 + + vpmuludq %ymm0,%ymm8,%ymm6 + vpmuludq %ymm1,%ymm8,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq 64(%rsp),%ymm4,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm11,%ymm11 + vmovdqa -16(%rax),%ymm8 + + vpmuludq %ymm0,%ymm7,%ymm6 + vpmuludq %ymm1,%ymm7,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vpmuludq %ymm3,%ymm7,%ymm6 + vpmuludq %ymm4,%ymm7,%ymm2 + vmovdqu 0(%rsi),%xmm7 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm2,%ymm15,%ymm15 + vinserti128 $1,32(%rsi),%ymm7,%ymm7 + + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq %ymm4,%ymm8,%ymm2 + vmovdqu 16(%rsi),%xmm8 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vmovdqa 16(%rax),%ymm2 + vpmuludq %ymm1,%ymm9,%ymm6 + vpmuludq %ymm0,%ymm9,%ymm9 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm9,%ymm13,%ymm13 + vinserti128 $1,48(%rsi),%ymm8,%ymm8 + leaq 64(%rsi),%rsi + + vpmuludq %ymm1,%ymm2,%ymm6 + vpmuludq %ymm0,%ymm2,%ymm2 + vpsrldq $6,%ymm7,%ymm9 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm14,%ymm14 + vpmuludq %ymm3,%ymm10,%ymm6 + vpmuludq %ymm4,%ymm10,%ymm2 + vpsrldq $6,%ymm8,%ymm10 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpunpckhqdq %ymm8,%ymm7,%ymm6 + + vpmuludq %ymm3,%ymm5,%ymm3 + vpmuludq %ymm4,%ymm5,%ymm4 + vpunpcklqdq %ymm8,%ymm7,%ymm7 + vpaddq %ymm3,%ymm13,%ymm2 + vpaddq %ymm4,%ymm14,%ymm3 + vpunpcklqdq %ymm10,%ymm9,%ymm10 + vpmuludq 80(%rax),%ymm0,%ymm4 + vpmuludq %ymm1,%ymm5,%ymm0 + vmovdqa 64(%rcx),%ymm5 + vpaddq %ymm4,%ymm15,%ymm4 + vpaddq %ymm0,%ymm11,%ymm0 + + + + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm12,%ymm1 + + vpsrlq $26,%ymm4,%ymm15 + vpand %ymm5,%ymm4,%ymm4 + + vpsrlq $4,%ymm10,%ymm9 + + vpsrlq $26,%ymm1,%ymm12 + vpand %ymm5,%ymm1,%ymm1 + vpaddq %ymm12,%ymm2,%ymm2 + + vpaddq %ymm15,%ymm0,%ymm0 + vpsllq $2,%ymm15,%ymm15 + vpaddq %ymm15,%ymm0,%ymm0 + + vpand %ymm5,%ymm9,%ymm9 + vpsrlq $26,%ymm7,%ymm8 + + vpsrlq $26,%ymm2,%ymm13 + vpand %ymm5,%ymm2,%ymm2 + vpaddq %ymm13,%ymm3,%ymm3 + + vpaddq %ymm9,%ymm2,%ymm2 + vpsrlq $30,%ymm10,%ymm10 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm1,%ymm1 + + vpsrlq $40,%ymm6,%ymm6 + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpand %ymm5,%ymm7,%ymm7 + vpand %ymm5,%ymm8,%ymm8 + vpand %ymm5,%ymm10,%ymm10 + vpor 32(%rcx),%ymm6,%ymm6 + + subq $64,%rdx + jnz .Loop_avx2 + +.byte 0x66,0x90 +.Ltail_avx2: + + + + + + + + vpaddq %ymm0,%ymm7,%ymm0 + vmovdqu 4(%rsp),%ymm7 + vpaddq %ymm1,%ymm8,%ymm1 + vmovdqu 36(%rsp),%ymm8 + vpaddq %ymm3,%ymm10,%ymm3 + vmovdqu 100(%rsp),%ymm9 + vpaddq %ymm4,%ymm6,%ymm4 + vmovdqu 52(%rax),%ymm10 + vmovdqu 116(%rax),%ymm5 + + vpmuludq %ymm2,%ymm7,%ymm13 + vpmuludq %ymm2,%ymm8,%ymm14 + vpmuludq %ymm2,%ymm9,%ymm15 + vpmuludq %ymm2,%ymm10,%ymm11 + vpmuludq %ymm2,%ymm5,%ymm12 + + vpmuludq %ymm0,%ymm8,%ymm6 + vpmuludq %ymm1,%ymm8,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq 68(%rsp),%ymm4,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm11,%ymm11 + + vpmuludq %ymm0,%ymm7,%ymm6 + vpmuludq %ymm1,%ymm7,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vmovdqu -12(%rax),%ymm8 + vpaddq %ymm2,%ymm12,%ymm12 + vpmuludq %ymm3,%ymm7,%ymm6 + vpmuludq %ymm4,%ymm7,%ymm2 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm2,%ymm15,%ymm15 + + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq %ymm4,%ymm8,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vmovdqu 20(%rax),%ymm2 + vpmuludq %ymm1,%ymm9,%ymm6 + vpmuludq %ymm0,%ymm9,%ymm9 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm9,%ymm13,%ymm13 + + vpmuludq %ymm1,%ymm2,%ymm6 + vpmuludq %ymm0,%ymm2,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm14,%ymm14 + vpmuludq %ymm3,%ymm10,%ymm6 + vpmuludq %ymm4,%ymm10,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + + vpmuludq %ymm3,%ymm5,%ymm3 + vpmuludq %ymm4,%ymm5,%ymm4 + vpaddq %ymm3,%ymm13,%ymm2 + vpaddq %ymm4,%ymm14,%ymm3 + vpmuludq 84(%rax),%ymm0,%ymm4 + vpmuludq %ymm1,%ymm5,%ymm0 + vmovdqa 64(%rcx),%ymm5 + vpaddq %ymm4,%ymm15,%ymm4 + vpaddq %ymm0,%ymm11,%ymm0 + + + + + vpsrldq $8,%ymm12,%ymm8 + vpsrldq $8,%ymm2,%ymm9 + vpsrldq $8,%ymm3,%ymm10 + vpsrldq $8,%ymm4,%ymm6 + vpsrldq $8,%ymm0,%ymm7 + vpaddq %ymm8,%ymm12,%ymm12 + vpaddq %ymm9,%ymm2,%ymm2 + vpaddq %ymm10,%ymm3,%ymm3 + vpaddq %ymm6,%ymm4,%ymm4 + vpaddq %ymm7,%ymm0,%ymm0 + + vpermq $0x2,%ymm3,%ymm10 + vpermq $0x2,%ymm4,%ymm6 + vpermq $0x2,%ymm0,%ymm7 + vpermq $0x2,%ymm12,%ymm8 + vpermq $0x2,%ymm2,%ymm9 + vpaddq %ymm10,%ymm3,%ymm3 + vpaddq %ymm6,%ymm4,%ymm4 + vpaddq %ymm7,%ymm0,%ymm0 + vpaddq %ymm8,%ymm12,%ymm12 + vpaddq %ymm9,%ymm2,%ymm2 + + + + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm12,%ymm1 + + vpsrlq $26,%ymm4,%ymm15 + vpand %ymm5,%ymm4,%ymm4 + + vpsrlq $26,%ymm1,%ymm12 + vpand %ymm5,%ymm1,%ymm1 + vpaddq %ymm12,%ymm2,%ymm2 + + vpaddq %ymm15,%ymm0,%ymm0 + vpsllq $2,%ymm15,%ymm15 + vpaddq %ymm15,%ymm0,%ymm0 + + vpsrlq $26,%ymm2,%ymm13 + vpand %ymm5,%ymm2,%ymm2 + vpaddq %ymm13,%ymm3,%ymm3 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm1,%ymm1 + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vmovd %xmm0,-112(%rdi) + vmovd %xmm1,-108(%rdi) + vmovd %xmm2,-104(%rdi) + vmovd %xmm3,-100(%rdi) + vmovd %xmm4,-96(%rdi) + leaq 8(%r11),%rsp +.cfi_def_cfa %rsp,8 + vzeroupper + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_poly1305_asm_blocks_avx2,.-crypton_poly1305_asm_blocks_avx2 +.align 64 +.Lconst: +.Lmask24: +.long 0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0 +.L129: +.long 16777216,0,16777216,0,16777216,0,16777216,0 +.Lmask26: +.long 0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0 +.Lpermd_avx2: +.long 2,2,2,3,2,0,2,1 +.Lpermd_avx512: +.long 0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7 + +.L2_44_inp_permd: +.long 0,1,1,2,2,3,7,7 +.L2_44_inp_shift: +.quad 0,12,24,64 +.L2_44_mask: +.quad 0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff +.L2_44_shift_rgt: +.quad 44,44,42,64 +.L2_44_shift_lft: +.quad 8,8,10,64 + +.align 64 +.Lx_mask44: +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.Lx_mask42: +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.byte 80,111,108,121,49,51,48,53,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 16 +.globl crypton_xor128_encrypt_n_pad +.type crypton_xor128_encrypt_n_pad,@function +.align 16 +crypton_xor128_encrypt_n_pad: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + subq %rdx,%rsi + subq %rdx,%rdi + movq %rcx,%r10 + shrq $4,%rcx + jz .Ltail_enc + nop +.Loop_enc_xmm: + movdqu (%rsi,%rdx,1),%xmm0 + pxor (%rdx),%xmm0 + movdqu %xmm0,(%rdi,%rdx,1) + movdqa %xmm0,(%rdx) + leaq 16(%rdx),%rdx + decq %rcx + jnz .Loop_enc_xmm + + andq $15,%r10 + jz .Ldone_enc + +.Ltail_enc: + movq $16,%rcx + subq %r10,%rcx + xorl %eax,%eax +.Loop_enc_byte: + movb (%rsi,%rdx,1),%al + xorb (%rdx),%al + movb %al,(%rdi,%rdx,1) + movb %al,(%rdx) + leaq 1(%rdx),%rdx + decq %r10 + jnz .Loop_enc_byte + + xorl %eax,%eax +.Loop_enc_pad: + movb %al,(%rdx) + leaq 1(%rdx),%rdx + decq %rcx + jnz .Loop_enc_pad + +.Ldone_enc: + movq %rdx,%rax + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_xor128_encrypt_n_pad,.-crypton_xor128_encrypt_n_pad + +.globl crypton_xor128_decrypt_n_pad +.type crypton_xor128_decrypt_n_pad,@function +.align 16 +crypton_xor128_decrypt_n_pad: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + subq %rdx,%rsi + subq %rdx,%rdi + movq %rcx,%r10 + shrq $4,%rcx + jz .Ltail_dec + nop +.Loop_dec_xmm: + movdqu (%rsi,%rdx,1),%xmm0 + movdqa (%rdx),%xmm1 + pxor %xmm0,%xmm1 + movdqu %xmm1,(%rdi,%rdx,1) + movdqa %xmm0,(%rdx) + leaq 16(%rdx),%rdx + decq %rcx + jnz .Loop_dec_xmm + + pxor %xmm1,%xmm1 + andq $15,%r10 + jz .Ldone_dec + +.Ltail_dec: + movq $16,%rcx + subq %r10,%rcx + xorl %eax,%eax + xorq %r11,%r11 +.Loop_dec_byte: + movb (%rsi,%rdx,1),%r11b + movb (%rdx),%al + xorb %r11b,%al + movb %al,(%rdi,%rdx,1) + movb %r11b,(%rdx) + leaq 1(%rdx),%rdx + decq %r10 + jnz .Loop_dec_byte + + xorl %eax,%eax +.Loop_dec_pad: + movb %al,(%rdx) + leaq 1(%rdx),%rdx + decq %rcx + jnz .Loop_dec_pad + +.Ldone_dec: + movq %rdx,%rax + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_xor128_decrypt_n_pad,.-crypton_xor128_decrypt_n_pad + +.section .note.gnu.property,"a",@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align 8 +2: + +.section .note.GNU-stack,"",@progbits diff --git a/cbits/asm/poly1305-x86_64-macosx.S b/cbits/asm/poly1305-x86_64-macosx.S new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-x86_64-macosx.S @@ -0,0 +1,2024 @@ +.text + + + +.globl _crypton_poly1305_asm_init +.private_extern _crypton_poly1305_asm_init +.globl _crypton_poly1305_asm_blocks +.private_extern _crypton_poly1305_asm_blocks +.globl _crypton_poly1305_asm_emit +.private_extern _crypton_poly1305_asm_emit + + +.p2align 5 +_crypton_poly1305_asm_init: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + xorq %rax,%rax + movq %rax,0(%rdi) + movq %rax,8(%rdi) + movq %rax,16(%rdi) + + cmpq $0,%rsi + je L$no_key + + movq $0x0ffffffc0fffffff,%rax + leaq -3(%rax),%rcx + andq 0(%rsi),%rax + andq 8(%rsi),%rcx + movq %rax,24(%rdi) + movq %rcx,32(%rdi) + movl $-1,48(%rdi) + leaq _crypton_poly1305_asm_blocks(%rip),%r10 + leaq _crypton_poly1305_asm_emit(%rip),%r11 + movq _crypton_ia32cap_P+4(%rip),%r9 + leaq crypton_poly1305_asm_blocks_avx(%rip),%rax + btq $28,%r9 + cmovcq %rax,%r10 + leaq crypton_poly1305_asm_blocks_avx2(%rip),%rax + btq $37,%r9 + cmovcq %rax,%r10 + movq %r10,0(%rdx) + movq %r11,8(%rdx) + movl $1,%eax +L$no_key: + .byte 0xf3,0xc3 +.cfi_endproc + + + +.p2align 5 +_crypton_poly1305_asm_blocks: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + +L$blocks: + shrq $4,%rdx + jz L$no_data + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +L$blocks_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movq 16(%rdi),%rbp + + movl %r14d,%eax + movl 4(%rdi),%edx + movl %ebx,%r8d + movl 12(%rdi),%r10d + movl %ebp,%r12d + + shlq $26,%rdx + movq %r8,%r9 + shlq $52,%r8 + addq %rdx,%rax + shrq $12,%r9 + addq %rax,%r8 + adcq $0,%r9 + + shlq $14,%r10 + movq %r12,%rax + shrq $24,%r12 + addq %r10,%r9 + shlq $40,%rax + addq %rax,%r9 + adcq $0,%r12 + + cmpq $4,%rbp + + cmovaq %r8,%r14 + cmovaq %r9,%rbx + cmovaq %r12,%rbp + + movq %r13,%r12 + shrq $2,%r13 + movq %r12,%rax + addq %r12,%r13 + jmp L$oop + +.p2align 5 +L$oop: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + mulq %r14 + movq %rax,%r9 + movq %r11,%rax + movq %rdx,%r10 + + mulq %r14 + movq %rax,%r14 + movq %r11,%rax + movq %rdx,%r8 + + mulq %rbx + addq %rax,%r9 + movq %r13,%rax + adcq %rdx,%r10 + + mulq %rbx + movq %rbp,%rbx + addq %rax,%r14 + adcq %rdx,%r8 + + imulq %r13,%rbx + addq %rbx,%r9 + movq %r8,%rbx + adcq $0,%r10 + + imulq %r11,%rbp + addq %r9,%rbx + movq $-4,%rax + adcq %rbp,%r10 + + andq %r10,%rax + movq %r10,%rbp + shrq $2,%r10 + andq $3,%rbp + addq %r10,%rax + addq %rax,%r14 + adcq $0,%rbx + adcq $0,%rbp + movq %r12,%rax + decq %r15 + jnz L$oop + + movq %r14,0(%rdi) + movq %rbx,8(%rdi) + movq %rbp,16(%rdi) + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +L$no_data: +L$blocks_epilogue: + .byte 0xf3,0xc3 +.cfi_endproc + + + +.p2align 5 +_crypton_poly1305_asm_emit: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + movl 0(%rdi),%eax + movl 4(%rdi),%ecx + movl 8(%rdi),%r8d + movl 12(%rdi),%r11d + movl 16(%rdi),%r10d + + shlq $26,%rcx + movq %r8,%r9 + shlq $52,%r8 + addq %rcx,%rax + shrq $12,%r9 + addq %rax,%r8 + adcq $0,%r9 + + shlq $14,%r11 + movq %r10,%rax + shrq $24,%r10 + addq %r11,%r9 + movq 0(%rdi),%rcx + shlq $40,%rax + movq 8(%rdi),%r11 + addq %rax,%r9 + movq 16(%rdi),%rax + adcq $0,%r10 + + cmpq $4,%rax + + cmovbeq %rcx,%r8 + cmovbeq %r11,%r9 + cmovbeq %rax,%r10 + + movq %r8,%rax + addq $5,%r8 + movq %r9,%rcx + adcq $0,%r9 + adcq $0,%r10 + shrq $2,%r10 + cmovnzq %r8,%rax + cmovnzq %r9,%rcx + + addq 0(%rdx),%rax + adcq 8(%rdx),%rcx + movq %rax,0(%rsi) + movq %rcx,8(%rsi) + + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +__crypton_poly1305_asm_block: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + mulq %r14 + movq %rax,%r9 + movq %r11,%rax + movq %rdx,%r10 + + mulq %r14 + movq %rax,%r14 + movq %r11,%rax + movq %rdx,%r8 + + mulq %rbx + addq %rax,%r9 + movq %r13,%rax + adcq %rdx,%r10 + + mulq %rbx + movq %rbp,%rbx + addq %rax,%r14 + adcq %rdx,%r8 + + imulq %r13,%rbx + addq %rbx,%r9 + movq %r8,%rbx + adcq $0,%r10 + + imulq %r11,%rbp + addq %r9,%rbx + movq $-4,%rax + adcq %rbp,%r10 + + andq %r10,%rax + movq %r10,%rbp + shrq $2,%r10 + andq $3,%rbp + addq %r10,%rax + addq %rax,%r14 + adcq $0,%rbx + adcq $0,%rbp + .byte 0xf3,0xc3 +.cfi_endproc + + + +.p2align 5 +__crypton_poly1305_asm_init_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + cmpl $-1,48(%rdi) + jne L$done_init_avx + + movq %r11,%r14 + movq %r12,%rbx + xorq %rbp,%rbp + + leaq 48+64(%rdi),%rdi + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + movq %r14,%r8 + andl %r14d,%eax + movq %r11,%r9 + andl %r11d,%edx + movl %eax,-64(%rdi) + shrq $26,%r8 + movl %edx,-60(%rdi) + shrq $26,%r9 + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + andl %r8d,%eax + andl %r9d,%edx + movl %eax,-48(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,-44(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,-32(%rdi) + shrq $26,%r8 + movl %edx,-28(%rdi) + shrq $26,%r9 + + movq %rbx,%rax + movq %r12,%rdx + shlq $12,%rax + shlq $12,%rdx + orq %r8,%rax + orq %r9,%rdx + andl $0x3ffffff,%eax + andl $0x3ffffff,%edx + movl %eax,-16(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,-12(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,0(%rdi) + movq %rbx,%r8 + movl %edx,4(%rdi) + movq %r12,%r9 + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + shrq $14,%r8 + shrq $14,%r9 + andl %r8d,%eax + andl %r9d,%edx + movl %eax,16(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,20(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,32(%rdi) + shrq $26,%r8 + movl %edx,36(%rdi) + shrq $26,%r9 + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,48(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r9d,52(%rdi) + leaq (%r9,%r9,4),%r9 + movl %r8d,64(%rdi) + movl %r9d,68(%rdi) + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movq %r14,%r8 + andl %r14d,%eax + shrq $26,%r8 + movl %eax,-52(%rdi) + + movl $0x3ffffff,%edx + andl %r8d,%edx + movl %edx,-36(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,-20(%rdi) + + movq %rbx,%rax + shlq $12,%rax + orq %r8,%rax + andl $0x3ffffff,%eax + movl %eax,-4(%rdi) + leal (%rax,%rax,4),%eax + movq %rbx,%r8 + movl %eax,12(%rdi) + + movl $0x3ffffff,%edx + shrq $14,%r8 + andl %r8d,%edx + movl %edx,28(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,44(%rdi) + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,60(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r8d,76(%rdi) + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movq %r14,%r8 + andl %r14d,%eax + shrq $26,%r8 + movl %eax,-56(%rdi) + + movl $0x3ffffff,%edx + andl %r8d,%edx + movl %edx,-40(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,-24(%rdi) + + movq %rbx,%rax + shlq $12,%rax + orq %r8,%rax + andl $0x3ffffff,%eax + movl %eax,-8(%rdi) + leal (%rax,%rax,4),%eax + movq %rbx,%r8 + movl %eax,8(%rdi) + + movl $0x3ffffff,%edx + shrq $14,%r8 + andl %r8d,%edx + movl %edx,24(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,40(%rdi) + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,56(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r8d,72(%rdi) + + leaq -48-64(%rdi),%rdi +L$done_init_avx: + .byte 0xf3,0xc3 +.cfi_endproc + + + +.p2align 5 +crypton_poly1305_asm_blocks_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + movl 20(%rdi),%r8d + cmpq $128,%rdx + jb L$blocks + + andq $-16,%rdx + + vzeroupper + + testl %r8d,%r8d + jz L$base2_64_avx + + testq $31,%rdx + jz L$even_avx + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +L$blocks_avx_body: + + movq %rdx,%r15 + + movq 0(%rdi),%r8 + movq 8(%rdi),%r9 + movl 16(%rdi),%ebp + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + + movl %r8d,%r14d + andq $-2147483648,%r8 + movq %r9,%r12 + movl %r9d,%ebx + andq $-2147483648,%r9 + + shrq $6,%r8 + shlq $52,%r12 + addq %r8,%r14 + shrq $12,%rbx + shrq $18,%r9 + addq %r12,%r14 + adcq %r9,%rbx + + movq %rbp,%r8 + shlq $40,%r8 + shrq $24,%rbp + addq %r8,%rbx + adcq $0,%rbp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + + call __crypton_poly1305_asm_block + + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r11 + movq %rbx,%r12 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r11 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r11,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r12 + andq $0x3ffffff,%rbx + orq %r12,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + + leaq -16(%r15),%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +L$blocks_avx_epilogue: + jmp L$do_avx +.cfi_endproc + +.p2align 5 +L$base2_64_avx: +.cfi_startproc + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +L$base2_64_avx_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movl 16(%rdi),%ebp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + testq $31,%rdx + jz L$init_avx + + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + +L$init_avx: + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r8 + movq %rbx,%r9 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r8 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r8,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r9 + andq $0x3ffffff,%rbx + orq %r9,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + movl $1,20(%rdi) + + call __crypton_poly1305_asm_init_avx + + movq %r15,%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +L$base2_64_avx_epilogue: + jmp L$do_avx +.cfi_endproc + +.p2align 5 +L$even_avx: +.cfi_startproc + vmovd 0(%rdi),%xmm0 + vmovd 4(%rdi),%xmm1 + vmovd 8(%rdi),%xmm2 + vmovd 12(%rdi),%xmm3 + vmovd 16(%rdi),%xmm4 + +L$do_avx: + leaq -88(%rsp),%r11 +.cfi_def_cfa %r11,0x60 + subq $0x178,%rsp + subq $64,%rdx + leaq -32(%rsi),%rax + cmovcq %rax,%rsi + + vmovdqu 48(%rdi),%xmm14 + leaq 112(%rdi),%rdi + leaq L$const(%rip),%rcx + + + + vmovdqu 32(%rsi),%xmm5 + vmovdqu 48(%rsi),%xmm6 + vmovdqa 64(%rcx),%xmm15 + + vpsrldq $6,%xmm5,%xmm7 + vpsrldq $6,%xmm6,%xmm8 + vpunpckhqdq %xmm6,%xmm5,%xmm9 + vpunpcklqdq %xmm6,%xmm5,%xmm5 + vpunpcklqdq %xmm8,%xmm7,%xmm8 + + vpsrlq $40,%xmm9,%xmm9 + vpsrlq $26,%xmm5,%xmm6 + vpand %xmm15,%xmm5,%xmm5 + vpsrlq $4,%xmm8,%xmm7 + vpand %xmm15,%xmm6,%xmm6 + vpsrlq $30,%xmm8,%xmm8 + vpand %xmm15,%xmm7,%xmm7 + vpand %xmm15,%xmm8,%xmm8 + vpor 32(%rcx),%xmm9,%xmm9 + + jbe L$skip_loop_avx + + + vmovdqu -48(%rdi),%xmm11 + vmovdqu -32(%rdi),%xmm12 + vpshufd $0xEE,%xmm14,%xmm13 + vpshufd $0x44,%xmm14,%xmm10 + vmovdqa %xmm13,-144(%r11) + vmovdqa %xmm10,0(%rsp) + vpshufd $0xEE,%xmm11,%xmm14 + vmovdqu -16(%rdi),%xmm10 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm14,-128(%r11) + vmovdqa %xmm11,16(%rsp) + vpshufd $0xEE,%xmm12,%xmm13 + vmovdqu 0(%rdi),%xmm11 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm13,-112(%r11) + vmovdqa %xmm12,32(%rsp) + vpshufd $0xEE,%xmm10,%xmm14 + vmovdqu 16(%rdi),%xmm12 + vpshufd $0x44,%xmm10,%xmm10 + vmovdqa %xmm14,-96(%r11) + vmovdqa %xmm10,48(%rsp) + vpshufd $0xEE,%xmm11,%xmm13 + vmovdqu 32(%rdi),%xmm10 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm13,-80(%r11) + vmovdqa %xmm11,64(%rsp) + vpshufd $0xEE,%xmm12,%xmm14 + vmovdqu 48(%rdi),%xmm11 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm14,-64(%r11) + vmovdqa %xmm12,80(%rsp) + vpshufd $0xEE,%xmm10,%xmm13 + vmovdqu 64(%rdi),%xmm12 + vpshufd $0x44,%xmm10,%xmm10 + vmovdqa %xmm13,-48(%r11) + vmovdqa %xmm10,96(%rsp) + vpshufd $0xEE,%xmm11,%xmm14 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm14,-32(%r11) + vmovdqa %xmm11,112(%rsp) + vpshufd $0xEE,%xmm12,%xmm13 + vmovdqa 0(%rsp),%xmm14 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm13,-16(%r11) + vmovdqa %xmm12,128(%rsp) + + jmp L$oop_avx + +.p2align 5 +L$oop_avx: + + + + + + + + + + + + + + + + + + + + + vpmuludq %xmm5,%xmm14,%xmm10 + vpmuludq %xmm6,%xmm14,%xmm11 + vmovdqa %xmm2,32(%r11) + vpmuludq %xmm7,%xmm14,%xmm12 + vmovdqa 16(%rsp),%xmm2 + vpmuludq %xmm8,%xmm14,%xmm13 + vpmuludq %xmm9,%xmm14,%xmm14 + + vmovdqa %xmm0,0(%r11) + vpmuludq 32(%rsp),%xmm9,%xmm0 + vmovdqa %xmm1,16(%r11) + vpmuludq %xmm8,%xmm2,%xmm1 + vpaddq %xmm0,%xmm10,%xmm10 + vpaddq %xmm1,%xmm14,%xmm14 + vmovdqa %xmm3,48(%r11) + vpmuludq %xmm7,%xmm2,%xmm0 + vpmuludq %xmm6,%xmm2,%xmm1 + vpaddq %xmm0,%xmm13,%xmm13 + vmovdqa 48(%rsp),%xmm3 + vpaddq %xmm1,%xmm12,%xmm12 + vmovdqa %xmm4,64(%r11) + vpmuludq %xmm5,%xmm2,%xmm2 + vpmuludq %xmm7,%xmm3,%xmm0 + vpaddq %xmm2,%xmm11,%xmm11 + + vmovdqa 64(%rsp),%xmm4 + vpaddq %xmm0,%xmm14,%xmm14 + vpmuludq %xmm6,%xmm3,%xmm1 + vpmuludq %xmm5,%xmm3,%xmm3 + vpaddq %xmm1,%xmm13,%xmm13 + vmovdqa 80(%rsp),%xmm2 + vpaddq %xmm3,%xmm12,%xmm12 + vpmuludq %xmm9,%xmm4,%xmm0 + vpmuludq %xmm8,%xmm4,%xmm4 + vpaddq %xmm0,%xmm11,%xmm11 + vmovdqa 96(%rsp),%xmm3 + vpaddq %xmm4,%xmm10,%xmm10 + + vmovdqa 128(%rsp),%xmm4 + vpmuludq %xmm6,%xmm2,%xmm1 + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm1,%xmm14,%xmm14 + vpaddq %xmm2,%xmm13,%xmm13 + vpmuludq %xmm9,%xmm3,%xmm0 + vpmuludq %xmm8,%xmm3,%xmm1 + vpaddq %xmm0,%xmm12,%xmm12 + vmovdqu 0(%rsi),%xmm0 + vpaddq %xmm1,%xmm11,%xmm11 + vpmuludq %xmm7,%xmm3,%xmm3 + vpmuludq %xmm7,%xmm4,%xmm7 + vpaddq %xmm3,%xmm10,%xmm10 + + vmovdqu 16(%rsi),%xmm1 + vpaddq %xmm7,%xmm11,%xmm11 + vpmuludq %xmm8,%xmm4,%xmm8 + vpmuludq %xmm9,%xmm4,%xmm9 + vpsrldq $6,%xmm0,%xmm2 + vpaddq %xmm8,%xmm12,%xmm12 + vpaddq %xmm9,%xmm13,%xmm13 + vpsrldq $6,%xmm1,%xmm3 + vpmuludq 112(%rsp),%xmm5,%xmm9 + vpmuludq %xmm6,%xmm4,%xmm5 + vpunpckhqdq %xmm1,%xmm0,%xmm4 + vpaddq %xmm9,%xmm14,%xmm14 + vmovdqa -144(%r11),%xmm9 + vpaddq %xmm5,%xmm10,%xmm10 + + vpunpcklqdq %xmm1,%xmm0,%xmm0 + vpunpcklqdq %xmm3,%xmm2,%xmm3 + + + vpsrldq $5,%xmm4,%xmm4 + vpsrlq $26,%xmm0,%xmm1 + vpand %xmm15,%xmm0,%xmm0 + vpsrlq $4,%xmm3,%xmm2 + vpand %xmm15,%xmm1,%xmm1 + vpand 0(%rcx),%xmm4,%xmm4 + vpsrlq $30,%xmm3,%xmm3 + vpand %xmm15,%xmm2,%xmm2 + vpand %xmm15,%xmm3,%xmm3 + vpor 32(%rcx),%xmm4,%xmm4 + + vpaddq 0(%r11),%xmm0,%xmm0 + vpaddq 16(%r11),%xmm1,%xmm1 + vpaddq 32(%r11),%xmm2,%xmm2 + vpaddq 48(%r11),%xmm3,%xmm3 + vpaddq 64(%r11),%xmm4,%xmm4 + + leaq 32(%rsi),%rax + leaq 64(%rsi),%rsi + subq $64,%rdx + cmovcq %rax,%rsi + + + + + + + + + + + vpmuludq %xmm0,%xmm9,%xmm5 + vpmuludq %xmm1,%xmm9,%xmm6 + vpaddq %xmm5,%xmm10,%xmm10 + vpaddq %xmm6,%xmm11,%xmm11 + vmovdqa -128(%r11),%xmm7 + vpmuludq %xmm2,%xmm9,%xmm5 + vpmuludq %xmm3,%xmm9,%xmm6 + vpaddq %xmm5,%xmm12,%xmm12 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm4,%xmm9,%xmm9 + vpmuludq -112(%r11),%xmm4,%xmm5 + vpaddq %xmm9,%xmm14,%xmm14 + + vpaddq %xmm5,%xmm10,%xmm10 + vpmuludq %xmm2,%xmm7,%xmm6 + vpmuludq %xmm3,%xmm7,%xmm5 + vpaddq %xmm6,%xmm13,%xmm13 + vmovdqa -96(%r11),%xmm8 + vpaddq %xmm5,%xmm14,%xmm14 + vpmuludq %xmm1,%xmm7,%xmm6 + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm6,%xmm12,%xmm12 + vpaddq %xmm7,%xmm11,%xmm11 + + vmovdqa -80(%r11),%xmm9 + vpmuludq %xmm2,%xmm8,%xmm5 + vpmuludq %xmm1,%xmm8,%xmm6 + vpaddq %xmm5,%xmm14,%xmm14 + vpaddq %xmm6,%xmm13,%xmm13 + vmovdqa -64(%r11),%xmm7 + vpmuludq %xmm0,%xmm8,%xmm8 + vpmuludq %xmm4,%xmm9,%xmm5 + vpaddq %xmm8,%xmm12,%xmm12 + vpaddq %xmm5,%xmm11,%xmm11 + vmovdqa -48(%r11),%xmm8 + vpmuludq %xmm3,%xmm9,%xmm9 + vpmuludq %xmm1,%xmm7,%xmm6 + vpaddq %xmm9,%xmm10,%xmm10 + + vmovdqa -16(%r11),%xmm9 + vpaddq %xmm6,%xmm14,%xmm14 + vpmuludq %xmm0,%xmm7,%xmm7 + vpmuludq %xmm4,%xmm8,%xmm5 + vpaddq %xmm7,%xmm13,%xmm13 + vpaddq %xmm5,%xmm12,%xmm12 + vmovdqu 32(%rsi),%xmm5 + vpmuludq %xmm3,%xmm8,%xmm7 + vpmuludq %xmm2,%xmm8,%xmm8 + vpaddq %xmm7,%xmm11,%xmm11 + vmovdqu 48(%rsi),%xmm6 + vpaddq %xmm8,%xmm10,%xmm10 + + vpmuludq %xmm2,%xmm9,%xmm2 + vpmuludq %xmm3,%xmm9,%xmm3 + vpsrldq $6,%xmm5,%xmm7 + vpaddq %xmm2,%xmm11,%xmm11 + vpmuludq %xmm4,%xmm9,%xmm4 + vpsrldq $6,%xmm6,%xmm8 + vpaddq %xmm3,%xmm12,%xmm2 + vpaddq %xmm4,%xmm13,%xmm3 + vpmuludq -32(%r11),%xmm0,%xmm4 + vpmuludq %xmm1,%xmm9,%xmm0 + vpunpckhqdq %xmm6,%xmm5,%xmm9 + vpaddq %xmm4,%xmm14,%xmm4 + vpaddq %xmm0,%xmm10,%xmm0 + + vpunpcklqdq %xmm6,%xmm5,%xmm5 + vpunpcklqdq %xmm8,%xmm7,%xmm8 + + + vpsrldq $5,%xmm9,%xmm9 + vpsrlq $26,%xmm5,%xmm6 + vmovdqa 0(%rsp),%xmm14 + vpand %xmm15,%xmm5,%xmm5 + vpsrlq $4,%xmm8,%xmm7 + vpand %xmm15,%xmm6,%xmm6 + vpand 0(%rcx),%xmm9,%xmm9 + vpsrlq $30,%xmm8,%xmm8 + vpand %xmm15,%xmm7,%xmm7 + vpand %xmm15,%xmm8,%xmm8 + vpor 32(%rcx),%xmm9,%xmm9 + + + + + + vpsrlq $26,%xmm3,%xmm13 + vpand %xmm15,%xmm3,%xmm3 + vpaddq %xmm13,%xmm4,%xmm4 + + vpsrlq $26,%xmm0,%xmm10 + vpand %xmm15,%xmm0,%xmm0 + vpaddq %xmm10,%xmm11,%xmm1 + + vpsrlq $26,%xmm4,%xmm10 + vpand %xmm15,%xmm4,%xmm4 + + vpsrlq $26,%xmm1,%xmm11 + vpand %xmm15,%xmm1,%xmm1 + vpaddq %xmm11,%xmm2,%xmm2 + + vpaddq %xmm10,%xmm0,%xmm0 + vpsllq $2,%xmm10,%xmm10 + vpaddq %xmm10,%xmm0,%xmm0 + + vpsrlq $26,%xmm2,%xmm12 + vpand %xmm15,%xmm2,%xmm2 + vpaddq %xmm12,%xmm3,%xmm3 + + vpsrlq $26,%xmm0,%xmm10 + vpand %xmm15,%xmm0,%xmm0 + vpaddq %xmm10,%xmm1,%xmm1 + + vpsrlq $26,%xmm3,%xmm13 + vpand %xmm15,%xmm3,%xmm3 + vpaddq %xmm13,%xmm4,%xmm4 + + ja L$oop_avx + +L$skip_loop_avx: + + + + vpshufd $0x10,%xmm14,%xmm14 + addq $32,%rdx + jnz L$ong_tail_avx + + vpaddq %xmm2,%xmm7,%xmm7 + vpaddq %xmm0,%xmm5,%xmm5 + vpaddq %xmm1,%xmm6,%xmm6 + vpaddq %xmm3,%xmm8,%xmm8 + vpaddq %xmm4,%xmm9,%xmm9 + +L$ong_tail_avx: + vmovdqa %xmm2,32(%r11) + vmovdqa %xmm0,0(%r11) + vmovdqa %xmm1,16(%r11) + vmovdqa %xmm3,48(%r11) + vmovdqa %xmm4,64(%r11) + + + + + + + + vpmuludq %xmm7,%xmm14,%xmm12 + vpmuludq %xmm5,%xmm14,%xmm10 + vpshufd $0x10,-48(%rdi),%xmm2 + vpmuludq %xmm6,%xmm14,%xmm11 + vpmuludq %xmm8,%xmm14,%xmm13 + vpmuludq %xmm9,%xmm14,%xmm14 + + vpmuludq %xmm8,%xmm2,%xmm0 + vpaddq %xmm0,%xmm14,%xmm14 + vpshufd $0x10,-32(%rdi),%xmm3 + vpmuludq %xmm7,%xmm2,%xmm1 + vpaddq %xmm1,%xmm13,%xmm13 + vpshufd $0x10,-16(%rdi),%xmm4 + vpmuludq %xmm6,%xmm2,%xmm0 + vpaddq %xmm0,%xmm12,%xmm12 + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm2,%xmm11,%xmm11 + vpmuludq %xmm9,%xmm3,%xmm3 + vpaddq %xmm3,%xmm10,%xmm10 + + vpshufd $0x10,0(%rdi),%xmm2 + vpmuludq %xmm7,%xmm4,%xmm1 + vpaddq %xmm1,%xmm14,%xmm14 + vpmuludq %xmm6,%xmm4,%xmm0 + vpaddq %xmm0,%xmm13,%xmm13 + vpshufd $0x10,16(%rdi),%xmm3 + vpmuludq %xmm5,%xmm4,%xmm4 + vpaddq %xmm4,%xmm12,%xmm12 + vpmuludq %xmm9,%xmm2,%xmm1 + vpaddq %xmm1,%xmm11,%xmm11 + vpshufd $0x10,32(%rdi),%xmm4 + vpmuludq %xmm8,%xmm2,%xmm2 + vpaddq %xmm2,%xmm10,%xmm10 + + vpmuludq %xmm6,%xmm3,%xmm0 + vpaddq %xmm0,%xmm14,%xmm14 + vpmuludq %xmm5,%xmm3,%xmm3 + vpaddq %xmm3,%xmm13,%xmm13 + vpshufd $0x10,48(%rdi),%xmm2 + vpmuludq %xmm9,%xmm4,%xmm1 + vpaddq %xmm1,%xmm12,%xmm12 + vpshufd $0x10,64(%rdi),%xmm3 + vpmuludq %xmm8,%xmm4,%xmm0 + vpaddq %xmm0,%xmm11,%xmm11 + vpmuludq %xmm7,%xmm4,%xmm4 + vpaddq %xmm4,%xmm10,%xmm10 + + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm2,%xmm14,%xmm14 + vpmuludq %xmm9,%xmm3,%xmm1 + vpaddq %xmm1,%xmm13,%xmm13 + vpmuludq %xmm8,%xmm3,%xmm0 + vpaddq %xmm0,%xmm12,%xmm12 + vpmuludq %xmm7,%xmm3,%xmm1 + vpaddq %xmm1,%xmm11,%xmm11 + vpmuludq %xmm6,%xmm3,%xmm3 + vpaddq %xmm3,%xmm10,%xmm10 + + jz L$short_tail_avx + + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + + vpsrldq $6,%xmm0,%xmm2 + vpsrldq $6,%xmm1,%xmm3 + vpunpckhqdq %xmm1,%xmm0,%xmm4 + vpunpcklqdq %xmm1,%xmm0,%xmm0 + vpunpcklqdq %xmm3,%xmm2,%xmm3 + + vpsrlq $40,%xmm4,%xmm4 + vpsrlq $26,%xmm0,%xmm1 + vpand %xmm15,%xmm0,%xmm0 + vpsrlq $4,%xmm3,%xmm2 + vpand %xmm15,%xmm1,%xmm1 + vpsrlq $30,%xmm3,%xmm3 + vpand %xmm15,%xmm2,%xmm2 + vpand %xmm15,%xmm3,%xmm3 + vpor 32(%rcx),%xmm4,%xmm4 + + vpshufd $0x32,-64(%rdi),%xmm9 + vpaddq 0(%r11),%xmm0,%xmm0 + vpaddq 16(%r11),%xmm1,%xmm1 + vpaddq 32(%r11),%xmm2,%xmm2 + vpaddq 48(%r11),%xmm3,%xmm3 + vpaddq 64(%r11),%xmm4,%xmm4 + + + + + vpmuludq %xmm0,%xmm9,%xmm5 + vpaddq %xmm5,%xmm10,%xmm10 + vpmuludq %xmm1,%xmm9,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpmuludq %xmm2,%xmm9,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpshufd $0x32,-48(%rdi),%xmm7 + vpmuludq %xmm3,%xmm9,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm4,%xmm9,%xmm9 + vpaddq %xmm9,%xmm14,%xmm14 + + vpmuludq %xmm3,%xmm7,%xmm5 + vpaddq %xmm5,%xmm14,%xmm14 + vpshufd $0x32,-32(%rdi),%xmm8 + vpmuludq %xmm2,%xmm7,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpshufd $0x32,-16(%rdi),%xmm9 + vpmuludq %xmm1,%xmm7,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm7,%xmm11,%xmm11 + vpmuludq %xmm4,%xmm8,%xmm8 + vpaddq %xmm8,%xmm10,%xmm10 + + vpshufd $0x32,0(%rdi),%xmm7 + vpmuludq %xmm2,%xmm9,%xmm6 + vpaddq %xmm6,%xmm14,%xmm14 + vpmuludq %xmm1,%xmm9,%xmm5 + vpaddq %xmm5,%xmm13,%xmm13 + vpshufd $0x32,16(%rdi),%xmm8 + vpmuludq %xmm0,%xmm9,%xmm9 + vpaddq %xmm9,%xmm12,%xmm12 + vpmuludq %xmm4,%xmm7,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpshufd $0x32,32(%rdi),%xmm9 + vpmuludq %xmm3,%xmm7,%xmm7 + vpaddq %xmm7,%xmm10,%xmm10 + + vpmuludq %xmm1,%xmm8,%xmm5 + vpaddq %xmm5,%xmm14,%xmm14 + vpmuludq %xmm0,%xmm8,%xmm8 + vpaddq %xmm8,%xmm13,%xmm13 + vpshufd $0x32,48(%rdi),%xmm7 + vpmuludq %xmm4,%xmm9,%xmm6 + vpaddq %xmm6,%xmm12,%xmm12 + vpshufd $0x32,64(%rdi),%xmm8 + vpmuludq %xmm3,%xmm9,%xmm5 + vpaddq %xmm5,%xmm11,%xmm11 + vpmuludq %xmm2,%xmm9,%xmm9 + vpaddq %xmm9,%xmm10,%xmm10 + + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm7,%xmm14,%xmm14 + vpmuludq %xmm4,%xmm8,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm3,%xmm8,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpmuludq %xmm2,%xmm8,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpmuludq %xmm1,%xmm8,%xmm8 + vpaddq %xmm8,%xmm10,%xmm10 + +L$short_tail_avx: + + + + vpsrldq $8,%xmm14,%xmm9 + vpsrldq $8,%xmm13,%xmm8 + vpsrldq $8,%xmm11,%xmm6 + vpsrldq $8,%xmm10,%xmm5 + vpsrldq $8,%xmm12,%xmm7 + vpaddq %xmm8,%xmm13,%xmm13 + vpaddq %xmm9,%xmm14,%xmm14 + vpaddq %xmm5,%xmm10,%xmm10 + vpaddq %xmm6,%xmm11,%xmm11 + vpaddq %xmm7,%xmm12,%xmm12 + + + + + vpsrlq $26,%xmm13,%xmm3 + vpand %xmm15,%xmm13,%xmm13 + vpaddq %xmm3,%xmm14,%xmm14 + + vpsrlq $26,%xmm10,%xmm0 + vpand %xmm15,%xmm10,%xmm10 + vpaddq %xmm0,%xmm11,%xmm11 + + vpsrlq $26,%xmm14,%xmm4 + vpand %xmm15,%xmm14,%xmm14 + + vpsrlq $26,%xmm11,%xmm1 + vpand %xmm15,%xmm11,%xmm11 + vpaddq %xmm1,%xmm12,%xmm12 + + vpaddq %xmm4,%xmm10,%xmm10 + vpsllq $2,%xmm4,%xmm4 + vpaddq %xmm4,%xmm10,%xmm10 + + vpsrlq $26,%xmm12,%xmm2 + vpand %xmm15,%xmm12,%xmm12 + vpaddq %xmm2,%xmm13,%xmm13 + + vpsrlq $26,%xmm10,%xmm0 + vpand %xmm15,%xmm10,%xmm10 + vpaddq %xmm0,%xmm11,%xmm11 + + vpsrlq $26,%xmm13,%xmm3 + vpand %xmm15,%xmm13,%xmm13 + vpaddq %xmm3,%xmm14,%xmm14 + + vmovd %xmm10,-112(%rdi) + vmovd %xmm11,-108(%rdi) + vmovd %xmm12,-104(%rdi) + vmovd %xmm13,-100(%rdi) + vmovd %xmm14,-96(%rdi) + leaq 88(%r11),%rsp +.cfi_def_cfa %rsp,8 + vzeroupper + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 5 +crypton_poly1305_asm_blocks_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + movl 20(%rdi),%r8d + cmpq $128,%rdx + jb L$blocks + + andq $-16,%rdx + + vzeroupper + + testl %r8d,%r8d + jz L$base2_64_avx2 + + testq $63,%rdx + jz L$even_avx2 + + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +L$blocks_avx2_body: + + movq %rdx,%r15 + + movq 0(%rdi),%r8 + movq 8(%rdi),%r9 + movl 16(%rdi),%ebp + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + + movl %r8d,%r14d + andq $-2147483648,%r8 + movq %r9,%r12 + movl %r9d,%ebx + andq $-2147483648,%r9 + + shrq $6,%r8 + shlq $52,%r12 + addq %r8,%r14 + shrq $12,%rbx + shrq $18,%r9 + addq %r12,%r14 + adcq %r9,%rbx + + movq %rbp,%r8 + shlq $40,%r8 + shrq $24,%rbp + addq %r8,%rbx + adcq $0,%rbp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + +L$base2_26_pre_avx2: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + movq %r12,%rax + + testq $63,%r15 + jnz L$base2_26_pre_avx2 + + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r11 + movq %rbx,%r12 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r11 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r11,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r12 + andq $0x3ffffff,%rbx + orq %r12,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + + movq %r15,%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +L$blocks_avx2_epilogue: + jmp L$do_avx2 +.cfi_endproc + +.p2align 5 +L$base2_64_avx2: +.cfi_startproc + pushq %rbx +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbx,-16 + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-24 + pushq %r12 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_adjust_cfa_offset 8 +.cfi_offset %r15,-56 + leaq -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +L$base2_64_avx2_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movl 16(%rdi),%ebp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + testq $63,%rdx + jz L$init_avx2 + +L$base2_64_pre_avx2: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + movq %r12,%rax + + testq $63,%r15 + jnz L$base2_64_pre_avx2 + +L$init_avx2: + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r8 + movq %rbx,%r9 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r8 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r8,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r9 + andq $0x3ffffff,%rbx + orq %r9,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + movl $1,20(%rdi) + + call __crypton_poly1305_asm_init_avx + + movq %r15,%rdx + + movq 8(%rsp),%r15 +.cfi_restore %r15 + movq 16(%rsp),%r14 +.cfi_restore %r14 + movq 24(%rsp),%r13 +.cfi_restore %r13 + movq 32(%rsp),%r12 +.cfi_restore %r12 + movq 40(%rsp),%rbp +.cfi_restore %rbp + movq 48(%rsp),%rbx +.cfi_restore %rbx + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +L$base2_64_avx2_epilogue: + jmp L$do_avx2 +.cfi_endproc + +.p2align 5 +L$even_avx2: +.cfi_startproc + vmovd 0(%rdi),%xmm0 + vmovd 4(%rdi),%xmm1 + vmovd 8(%rdi),%xmm2 + vmovd 12(%rdi),%xmm3 + vmovd 16(%rdi),%xmm4 + +L$do_avx2: + leaq -8(%rsp),%r11 +.cfi_def_cfa %r11,16 + subq $0x128,%rsp + leaq L$const(%rip),%rcx + leaq 48+64(%rdi),%rdi + vmovdqa 96(%rcx),%ymm7 + + + vmovdqu -64(%rdi),%xmm9 + andq $-512,%rsp + vmovdqu -48(%rdi),%xmm10 + vmovdqu -32(%rdi),%xmm6 + vmovdqu -16(%rdi),%xmm11 + vmovdqu 0(%rdi),%xmm12 + vmovdqu 16(%rdi),%xmm13 + leaq 144(%rsp),%rax + vmovdqu 32(%rdi),%xmm14 + vpermd %ymm9,%ymm7,%ymm9 + vmovdqu 48(%rdi),%xmm15 + vpermd %ymm10,%ymm7,%ymm10 + vmovdqu 64(%rdi),%xmm5 + vpermd %ymm6,%ymm7,%ymm6 + vmovdqa %ymm9,0(%rsp) + vpermd %ymm11,%ymm7,%ymm11 + vmovdqa %ymm10,32-144(%rax) + vpermd %ymm12,%ymm7,%ymm12 + vmovdqa %ymm6,64-144(%rax) + vpermd %ymm13,%ymm7,%ymm13 + vmovdqa %ymm11,96-144(%rax) + vpermd %ymm14,%ymm7,%ymm14 + vmovdqa %ymm12,128-144(%rax) + vpermd %ymm15,%ymm7,%ymm15 + vmovdqa %ymm13,160-144(%rax) + vpermd %ymm5,%ymm7,%ymm5 + vmovdqa %ymm14,192-144(%rax) + vmovdqa %ymm15,224-144(%rax) + vmovdqa %ymm5,256-144(%rax) + vmovdqa 64(%rcx),%ymm5 + + + + vmovdqu 0(%rsi),%xmm7 + vmovdqu 16(%rsi),%xmm8 + vinserti128 $1,32(%rsi),%ymm7,%ymm7 + vinserti128 $1,48(%rsi),%ymm8,%ymm8 + leaq 64(%rsi),%rsi + + vpsrldq $6,%ymm7,%ymm9 + vpsrldq $6,%ymm8,%ymm10 + vpunpckhqdq %ymm8,%ymm7,%ymm6 + vpunpcklqdq %ymm10,%ymm9,%ymm9 + vpunpcklqdq %ymm8,%ymm7,%ymm7 + + vpsrlq $30,%ymm9,%ymm10 + vpsrlq $4,%ymm9,%ymm9 + vpsrlq $26,%ymm7,%ymm8 + vpsrlq $40,%ymm6,%ymm6 + vpand %ymm5,%ymm9,%ymm9 + vpand %ymm5,%ymm7,%ymm7 + vpand %ymm5,%ymm8,%ymm8 + vpand %ymm5,%ymm10,%ymm10 + vpor 32(%rcx),%ymm6,%ymm6 + + vpaddq %ymm2,%ymm9,%ymm2 + subq $64,%rdx + jz L$tail_avx2 + jmp L$oop_avx2 + +.p2align 5 +L$oop_avx2: + + + + + + + + + vpaddq %ymm0,%ymm7,%ymm0 + vmovdqa 0(%rsp),%ymm7 + vpaddq %ymm1,%ymm8,%ymm1 + vmovdqa 32(%rsp),%ymm8 + vpaddq %ymm3,%ymm10,%ymm3 + vmovdqa 96(%rsp),%ymm9 + vpaddq %ymm4,%ymm6,%ymm4 + vmovdqa 48(%rax),%ymm10 + vmovdqa 112(%rax),%ymm5 + + + + + + + + + + + + + + + + + vpmuludq %ymm2,%ymm7,%ymm13 + vpmuludq %ymm2,%ymm8,%ymm14 + vpmuludq %ymm2,%ymm9,%ymm15 + vpmuludq %ymm2,%ymm10,%ymm11 + vpmuludq %ymm2,%ymm5,%ymm12 + + vpmuludq %ymm0,%ymm8,%ymm6 + vpmuludq %ymm1,%ymm8,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq 64(%rsp),%ymm4,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm11,%ymm11 + vmovdqa -16(%rax),%ymm8 + + vpmuludq %ymm0,%ymm7,%ymm6 + vpmuludq %ymm1,%ymm7,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vpmuludq %ymm3,%ymm7,%ymm6 + vpmuludq %ymm4,%ymm7,%ymm2 + vmovdqu 0(%rsi),%xmm7 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm2,%ymm15,%ymm15 + vinserti128 $1,32(%rsi),%ymm7,%ymm7 + + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq %ymm4,%ymm8,%ymm2 + vmovdqu 16(%rsi),%xmm8 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vmovdqa 16(%rax),%ymm2 + vpmuludq %ymm1,%ymm9,%ymm6 + vpmuludq %ymm0,%ymm9,%ymm9 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm9,%ymm13,%ymm13 + vinserti128 $1,48(%rsi),%ymm8,%ymm8 + leaq 64(%rsi),%rsi + + vpmuludq %ymm1,%ymm2,%ymm6 + vpmuludq %ymm0,%ymm2,%ymm2 + vpsrldq $6,%ymm7,%ymm9 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm14,%ymm14 + vpmuludq %ymm3,%ymm10,%ymm6 + vpmuludq %ymm4,%ymm10,%ymm2 + vpsrldq $6,%ymm8,%ymm10 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpunpckhqdq %ymm8,%ymm7,%ymm6 + + vpmuludq %ymm3,%ymm5,%ymm3 + vpmuludq %ymm4,%ymm5,%ymm4 + vpunpcklqdq %ymm8,%ymm7,%ymm7 + vpaddq %ymm3,%ymm13,%ymm2 + vpaddq %ymm4,%ymm14,%ymm3 + vpunpcklqdq %ymm10,%ymm9,%ymm10 + vpmuludq 80(%rax),%ymm0,%ymm4 + vpmuludq %ymm1,%ymm5,%ymm0 + vmovdqa 64(%rcx),%ymm5 + vpaddq %ymm4,%ymm15,%ymm4 + vpaddq %ymm0,%ymm11,%ymm0 + + + + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm12,%ymm1 + + vpsrlq $26,%ymm4,%ymm15 + vpand %ymm5,%ymm4,%ymm4 + + vpsrlq $4,%ymm10,%ymm9 + + vpsrlq $26,%ymm1,%ymm12 + vpand %ymm5,%ymm1,%ymm1 + vpaddq %ymm12,%ymm2,%ymm2 + + vpaddq %ymm15,%ymm0,%ymm0 + vpsllq $2,%ymm15,%ymm15 + vpaddq %ymm15,%ymm0,%ymm0 + + vpand %ymm5,%ymm9,%ymm9 + vpsrlq $26,%ymm7,%ymm8 + + vpsrlq $26,%ymm2,%ymm13 + vpand %ymm5,%ymm2,%ymm2 + vpaddq %ymm13,%ymm3,%ymm3 + + vpaddq %ymm9,%ymm2,%ymm2 + vpsrlq $30,%ymm10,%ymm10 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm1,%ymm1 + + vpsrlq $40,%ymm6,%ymm6 + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpand %ymm5,%ymm7,%ymm7 + vpand %ymm5,%ymm8,%ymm8 + vpand %ymm5,%ymm10,%ymm10 + vpor 32(%rcx),%ymm6,%ymm6 + + subq $64,%rdx + jnz L$oop_avx2 + +.byte 0x66,0x90 +L$tail_avx2: + + + + + + + + vpaddq %ymm0,%ymm7,%ymm0 + vmovdqu 4(%rsp),%ymm7 + vpaddq %ymm1,%ymm8,%ymm1 + vmovdqu 36(%rsp),%ymm8 + vpaddq %ymm3,%ymm10,%ymm3 + vmovdqu 100(%rsp),%ymm9 + vpaddq %ymm4,%ymm6,%ymm4 + vmovdqu 52(%rax),%ymm10 + vmovdqu 116(%rax),%ymm5 + + vpmuludq %ymm2,%ymm7,%ymm13 + vpmuludq %ymm2,%ymm8,%ymm14 + vpmuludq %ymm2,%ymm9,%ymm15 + vpmuludq %ymm2,%ymm10,%ymm11 + vpmuludq %ymm2,%ymm5,%ymm12 + + vpmuludq %ymm0,%ymm8,%ymm6 + vpmuludq %ymm1,%ymm8,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq 68(%rsp),%ymm4,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm11,%ymm11 + + vpmuludq %ymm0,%ymm7,%ymm6 + vpmuludq %ymm1,%ymm7,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vmovdqu -12(%rax),%ymm8 + vpaddq %ymm2,%ymm12,%ymm12 + vpmuludq %ymm3,%ymm7,%ymm6 + vpmuludq %ymm4,%ymm7,%ymm2 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm2,%ymm15,%ymm15 + + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq %ymm4,%ymm8,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vmovdqu 20(%rax),%ymm2 + vpmuludq %ymm1,%ymm9,%ymm6 + vpmuludq %ymm0,%ymm9,%ymm9 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm9,%ymm13,%ymm13 + + vpmuludq %ymm1,%ymm2,%ymm6 + vpmuludq %ymm0,%ymm2,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm14,%ymm14 + vpmuludq %ymm3,%ymm10,%ymm6 + vpmuludq %ymm4,%ymm10,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + + vpmuludq %ymm3,%ymm5,%ymm3 + vpmuludq %ymm4,%ymm5,%ymm4 + vpaddq %ymm3,%ymm13,%ymm2 + vpaddq %ymm4,%ymm14,%ymm3 + vpmuludq 84(%rax),%ymm0,%ymm4 + vpmuludq %ymm1,%ymm5,%ymm0 + vmovdqa 64(%rcx),%ymm5 + vpaddq %ymm4,%ymm15,%ymm4 + vpaddq %ymm0,%ymm11,%ymm0 + + + + + vpsrldq $8,%ymm12,%ymm8 + vpsrldq $8,%ymm2,%ymm9 + vpsrldq $8,%ymm3,%ymm10 + vpsrldq $8,%ymm4,%ymm6 + vpsrldq $8,%ymm0,%ymm7 + vpaddq %ymm8,%ymm12,%ymm12 + vpaddq %ymm9,%ymm2,%ymm2 + vpaddq %ymm10,%ymm3,%ymm3 + vpaddq %ymm6,%ymm4,%ymm4 + vpaddq %ymm7,%ymm0,%ymm0 + + vpermq $0x2,%ymm3,%ymm10 + vpermq $0x2,%ymm4,%ymm6 + vpermq $0x2,%ymm0,%ymm7 + vpermq $0x2,%ymm12,%ymm8 + vpermq $0x2,%ymm2,%ymm9 + vpaddq %ymm10,%ymm3,%ymm3 + vpaddq %ymm6,%ymm4,%ymm4 + vpaddq %ymm7,%ymm0,%ymm0 + vpaddq %ymm8,%ymm12,%ymm12 + vpaddq %ymm9,%ymm2,%ymm2 + + + + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm12,%ymm1 + + vpsrlq $26,%ymm4,%ymm15 + vpand %ymm5,%ymm4,%ymm4 + + vpsrlq $26,%ymm1,%ymm12 + vpand %ymm5,%ymm1,%ymm1 + vpaddq %ymm12,%ymm2,%ymm2 + + vpaddq %ymm15,%ymm0,%ymm0 + vpsllq $2,%ymm15,%ymm15 + vpaddq %ymm15,%ymm0,%ymm0 + + vpsrlq $26,%ymm2,%ymm13 + vpand %ymm5,%ymm2,%ymm2 + vpaddq %ymm13,%ymm3,%ymm3 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm1,%ymm1 + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vmovd %xmm0,-112(%rdi) + vmovd %xmm1,-108(%rdi) + vmovd %xmm2,-104(%rdi) + vmovd %xmm3,-100(%rdi) + vmovd %xmm4,-96(%rdi) + leaq 8(%r11),%rsp +.cfi_def_cfa %rsp,8 + vzeroupper + .byte 0xf3,0xc3 +.cfi_endproc + +.p2align 6 +L$const: +L$mask24: +.long 0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0 +L$129: +.long 16777216,0,16777216,0,16777216,0,16777216,0 +L$mask26: +.long 0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0 +L$permd_avx2: +.long 2,2,2,3,2,0,2,1 +L$permd_avx512: +.long 0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7 + +L$2_44_inp_permd: +.long 0,1,1,2,2,3,7,7 +L$2_44_inp_shift: +.quad 0,12,24,64 +L$2_44_mask: +.quad 0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff +L$2_44_shift_rgt: +.quad 44,44,42,64 +L$2_44_shift_lft: +.quad 8,8,10,64 + +.p2align 6 +L$x_mask44: +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +L$x_mask42: +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.byte 80,111,108,121,49,51,48,53,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.p2align 4 +.globl _crypton_xor128_encrypt_n_pad + +.p2align 4 +_crypton_xor128_encrypt_n_pad: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + subq %rdx,%rsi + subq %rdx,%rdi + movq %rcx,%r10 + shrq $4,%rcx + jz L$tail_enc + nop +L$oop_enc_xmm: + movdqu (%rsi,%rdx,1),%xmm0 + pxor (%rdx),%xmm0 + movdqu %xmm0,(%rdi,%rdx,1) + movdqa %xmm0,(%rdx) + leaq 16(%rdx),%rdx + decq %rcx + jnz L$oop_enc_xmm + + andq $15,%r10 + jz L$done_enc + +L$tail_enc: + movq $16,%rcx + subq %r10,%rcx + xorl %eax,%eax +L$oop_enc_byte: + movb (%rsi,%rdx,1),%al + xorb (%rdx),%al + movb %al,(%rdi,%rdx,1) + movb %al,(%rdx) + leaq 1(%rdx),%rdx + decq %r10 + jnz L$oop_enc_byte + + xorl %eax,%eax +L$oop_enc_pad: + movb %al,(%rdx) + leaq 1(%rdx),%rdx + decq %rcx + jnz L$oop_enc_pad + +L$done_enc: + movq %rdx,%rax + .byte 0xf3,0xc3 +.cfi_endproc + + +.globl _crypton_xor128_decrypt_n_pad + +.p2align 4 +_crypton_xor128_decrypt_n_pad: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + subq %rdx,%rsi + subq %rdx,%rdi + movq %rcx,%r10 + shrq $4,%rcx + jz L$tail_dec + nop +L$oop_dec_xmm: + movdqu (%rsi,%rdx,1),%xmm0 + movdqa (%rdx),%xmm1 + pxor %xmm0,%xmm1 + movdqu %xmm1,(%rdi,%rdx,1) + movdqa %xmm0,(%rdx) + leaq 16(%rdx),%rdx + decq %rcx + jnz L$oop_dec_xmm + + pxor %xmm1,%xmm1 + andq $15,%r10 + jz L$done_dec + +L$tail_dec: + movq $16,%rcx + subq %r10,%rcx + xorl %eax,%eax + xorq %r11,%r11 +L$oop_dec_byte: + movb (%rsi,%rdx,1),%r11b + movb (%rdx),%al + xorb %r11b,%al + movb %al,(%rdi,%rdx,1) + movb %r11b,(%rdx) + leaq 1(%rdx),%rdx + decq %r10 + jnz L$oop_dec_byte + + xorl %eax,%eax +L$oop_dec_pad: + movb %al,(%rdx) + leaq 1(%rdx),%rdx + decq %rcx + jnz L$oop_dec_pad + +L$done_dec: + movq %rdx,%rax + .byte 0xf3,0xc3 +.cfi_endproc + diff --git a/cbits/asm/poly1305-x86_64-mingw64.S b/cbits/asm/poly1305-x86_64-mingw64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-x86_64-mingw64.S @@ -0,0 +1,2281 @@ +.text + + + +.globl crypton_poly1305_asm_init + +.globl crypton_poly1305_asm_blocks + +.globl crypton_poly1305_asm_emit + + +.def crypton_poly1305_asm_init; .scl 2; .type 32; .endef +.p2align 5 +crypton_poly1305_asm_init: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_poly1305_asm_init: + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + xorq %rax,%rax + movq %rax,0(%rdi) + movq %rax,8(%rdi) + movq %rax,16(%rdi) + + cmpq $0,%rsi + je .Lno_key + + movq $0x0ffffffc0fffffff,%rax + leaq -3(%rax),%rcx + andq 0(%rsi),%rax + andq 8(%rsi),%rcx + movq %rax,24(%rdi) + movq %rcx,32(%rdi) + movl $-1,48(%rdi) + leaq crypton_poly1305_asm_blocks(%rip),%r10 + leaq crypton_poly1305_asm_emit(%rip),%r11 + movq crypton_ia32cap_P+4(%rip),%r9 + leaq crypton_poly1305_asm_blocks_avx(%rip),%rax + btq $28,%r9 + cmovcq %rax,%r10 + leaq crypton_poly1305_asm_blocks_avx2(%rip),%rax + btq $37,%r9 + cmovcq %rax,%r10 + movq %r10,0(%rdx) + movq %r11,8(%rdx) + movl $1,%eax +.Lno_key: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 +.LSEH_end_crypton_poly1305_asm_init: + +.def crypton_poly1305_asm_blocks; .scl 2; .type 32; .endef +.p2align 5 +crypton_poly1305_asm_blocks: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_poly1305_asm_blocks: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx +.Lblocks: + shrq $4,%rdx + jz .Lno_data + + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -8(%rsp),%rsp + +.Lblocks_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movq 16(%rdi),%rbp + + movl %r14d,%eax + movl 4(%rdi),%edx + movl %ebx,%r8d + movl 12(%rdi),%r10d + movl %ebp,%r12d + + shlq $26,%rdx + movq %r8,%r9 + shlq $52,%r8 + addq %rdx,%rax + shrq $12,%r9 + addq %rax,%r8 + adcq $0,%r9 + + shlq $14,%r10 + movq %r12,%rax + shrq $24,%r12 + addq %r10,%r9 + shlq $40,%rax + addq %rax,%r9 + adcq $0,%r12 + + cmpq $4,%rbp + + cmovaq %r8,%r14 + cmovaq %r9,%rbx + cmovaq %r12,%rbp + + movq %r13,%r12 + shrq $2,%r13 + movq %r12,%rax + addq %r12,%r13 + jmp .Loop + +.p2align 5 +.Loop: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + mulq %r14 + movq %rax,%r9 + movq %r11,%rax + movq %rdx,%r10 + + mulq %r14 + movq %rax,%r14 + movq %r11,%rax + movq %rdx,%r8 + + mulq %rbx + addq %rax,%r9 + movq %r13,%rax + adcq %rdx,%r10 + + mulq %rbx + movq %rbp,%rbx + addq %rax,%r14 + adcq %rdx,%r8 + + imulq %r13,%rbx + addq %rbx,%r9 + movq %r8,%rbx + adcq $0,%r10 + + imulq %r11,%rbp + addq %r9,%rbx + movq $-4,%rax + adcq %rbp,%r10 + + andq %r10,%rax + movq %r10,%rbp + shrq $2,%r10 + andq $3,%rbp + addq %r10,%rax + addq %rax,%r14 + adcq $0,%rbx + adcq $0,%rbp + movq %r12,%rax + decq %r15 + jnz .Loop + + movq %r14,0(%rdi) + movq %rbx,8(%rdi) + movq %rbp,16(%rdi) + + movq 8(%rsp),%r15 + + movq 16(%rsp),%r14 + + movq 24(%rsp),%r13 + + movq 32(%rsp),%r12 + + movq 40(%rsp),%rbp + + movq 48(%rsp),%rbx + + leaq 56(%rsp),%rsp + +.Lno_data: +.Lblocks_epilogue: + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_poly1305_asm_blocks: + +.def crypton_poly1305_asm_emit; .scl 2; .type 32; .endef +.p2align 5 +crypton_poly1305_asm_emit: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_poly1305_asm_emit: + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movl 0(%rdi),%eax + movl 4(%rdi),%ecx + movl 8(%rdi),%r8d + movl 12(%rdi),%r11d + movl 16(%rdi),%r10d + + shlq $26,%rcx + movq %r8,%r9 + shlq $52,%r8 + addq %rcx,%rax + shrq $12,%r9 + addq %rax,%r8 + adcq $0,%r9 + + shlq $14,%r11 + movq %r10,%rax + shrq $24,%r10 + addq %r11,%r9 + movq 0(%rdi),%rcx + shlq $40,%rax + movq 8(%rdi),%r11 + addq %rax,%r9 + movq 16(%rdi),%rax + adcq $0,%r10 + + cmpq $4,%rax + + cmovbeq %rcx,%r8 + cmovbeq %r11,%r9 + cmovbeq %rax,%r10 + + movq %r8,%rax + addq $5,%r8 + movq %r9,%rcx + adcq $0,%r9 + adcq $0,%r10 + shrq $2,%r10 + cmovnzq %r8,%rax + cmovnzq %r9,%rcx + + addq 0(%rdx),%rax + adcq 8(%rdx),%rcx + movq %rax,0(%rsi) + movq %rcx,8(%rsi) + + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 +.LSEH_end_crypton_poly1305_asm_emit: +.def __crypton_poly1305_asm_block; .scl 3; .type 32; .endef +.p2align 5 +__crypton_poly1305_asm_block: + .byte 0xf3,0x0f,0x1e,0xfa + + mulq %r14 + movq %rax,%r9 + movq %r11,%rax + movq %rdx,%r10 + + mulq %r14 + movq %rax,%r14 + movq %r11,%rax + movq %rdx,%r8 + + mulq %rbx + addq %rax,%r9 + movq %r13,%rax + adcq %rdx,%r10 + + mulq %rbx + movq %rbp,%rbx + addq %rax,%r14 + adcq %rdx,%r8 + + imulq %r13,%rbx + addq %rbx,%r9 + movq %r8,%rbx + adcq $0,%r10 + + imulq %r11,%rbp + addq %r9,%rbx + movq $-4,%rax + adcq %rbp,%r10 + + andq %r10,%rax + movq %r10,%rbp + shrq $2,%r10 + andq $3,%rbp + addq %r10,%rax + addq %rax,%r14 + adcq $0,%rbx + adcq $0,%rbp + .byte 0xf3,0xc3 + + +.def __crypton_poly1305_asm_init_avx; .scl 3; .type 32; .endef +.p2align 5 +__crypton_poly1305_asm_init_avx: + .byte 0xf3,0x0f,0x1e,0xfa + + cmpl $-1,48(%rdi) + jne .Ldone_init_avx + + movq %r11,%r14 + movq %r12,%rbx + xorq %rbp,%rbp + + leaq 48+64(%rdi),%rdi + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + movq %r14,%r8 + andl %r14d,%eax + movq %r11,%r9 + andl %r11d,%edx + movl %eax,-64(%rdi) + shrq $26,%r8 + movl %edx,-60(%rdi) + shrq $26,%r9 + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + andl %r8d,%eax + andl %r9d,%edx + movl %eax,-48(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,-44(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,-32(%rdi) + shrq $26,%r8 + movl %edx,-28(%rdi) + shrq $26,%r9 + + movq %rbx,%rax + movq %r12,%rdx + shlq $12,%rax + shlq $12,%rdx + orq %r8,%rax + orq %r9,%rdx + andl $0x3ffffff,%eax + andl $0x3ffffff,%edx + movl %eax,-16(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,-12(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,0(%rdi) + movq %rbx,%r8 + movl %edx,4(%rdi) + movq %r12,%r9 + + movl $0x3ffffff,%eax + movl $0x3ffffff,%edx + shrq $14,%r8 + shrq $14,%r9 + andl %r8d,%eax + andl %r9d,%edx + movl %eax,16(%rdi) + leal (%rax,%rax,4),%eax + movl %edx,20(%rdi) + leal (%rdx,%rdx,4),%edx + movl %eax,32(%rdi) + shrq $26,%r8 + movl %edx,36(%rdi) + shrq $26,%r9 + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,48(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r9d,52(%rdi) + leaq (%r9,%r9,4),%r9 + movl %r8d,64(%rdi) + movl %r9d,68(%rdi) + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movq %r14,%r8 + andl %r14d,%eax + shrq $26,%r8 + movl %eax,-52(%rdi) + + movl $0x3ffffff,%edx + andl %r8d,%edx + movl %edx,-36(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,-20(%rdi) + + movq %rbx,%rax + shlq $12,%rax + orq %r8,%rax + andl $0x3ffffff,%eax + movl %eax,-4(%rdi) + leal (%rax,%rax,4),%eax + movq %rbx,%r8 + movl %eax,12(%rdi) + + movl $0x3ffffff,%edx + shrq $14,%r8 + andl %r8d,%edx + movl %edx,28(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,44(%rdi) + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,60(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r8d,76(%rdi) + + movq %r12,%rax + call __crypton_poly1305_asm_block + + movl $0x3ffffff,%eax + movq %r14,%r8 + andl %r14d,%eax + shrq $26,%r8 + movl %eax,-56(%rdi) + + movl $0x3ffffff,%edx + andl %r8d,%edx + movl %edx,-40(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,-24(%rdi) + + movq %rbx,%rax + shlq $12,%rax + orq %r8,%rax + andl $0x3ffffff,%eax + movl %eax,-8(%rdi) + leal (%rax,%rax,4),%eax + movq %rbx,%r8 + movl %eax,8(%rdi) + + movl $0x3ffffff,%edx + shrq $14,%r8 + andl %r8d,%edx + movl %edx,24(%rdi) + leal (%rdx,%rdx,4),%edx + shrq $26,%r8 + movl %edx,40(%rdi) + + movq %rbp,%rax + shlq $24,%rax + orq %rax,%r8 + movl %r8d,56(%rdi) + leaq (%r8,%r8,4),%r8 + movl %r8d,72(%rdi) + + leaq -48-64(%rdi),%rdi +.Ldone_init_avx: + .byte 0xf3,0xc3 + + +.def crypton_poly1305_asm_blocks_avx; .scl 3; .type 32; .endef +.p2align 5 +crypton_poly1305_asm_blocks_avx: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_poly1305_asm_blocks_avx: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movl 20(%rdi),%r8d + cmpq $128,%rdx + jb .Lblocks + + andq $-16,%rdx + + vzeroupper + + testl %r8d,%r8d + jz .Lbase2_64_avx + + testq $31,%rdx + jz .Leven_avx + + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -8(%rsp),%rsp + +.Lblocks_avx_body: + + movq %rdx,%r15 + + movq 0(%rdi),%r8 + movq 8(%rdi),%r9 + movl 16(%rdi),%ebp + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + + movl %r8d,%r14d + andq $-2147483648,%r8 + movq %r9,%r12 + movl %r9d,%ebx + andq $-2147483648,%r9 + + shrq $6,%r8 + shlq $52,%r12 + addq %r8,%r14 + shrq $12,%rbx + shrq $18,%r9 + addq %r12,%r14 + adcq %r9,%rbx + + movq %rbp,%r8 + shlq $40,%r8 + shrq $24,%rbp + addq %r8,%rbx + adcq $0,%rbp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + + call __crypton_poly1305_asm_block + + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r11 + movq %rbx,%r12 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r11 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r11,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r12 + andq $0x3ffffff,%rbx + orq %r12,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + + leaq -16(%r15),%rdx + + movq 8(%rsp),%r15 + + movq 16(%rsp),%r14 + + movq 24(%rsp),%r13 + + movq 32(%rsp),%r12 + + movq 40(%rsp),%rbp + + movq 48(%rsp),%rbx + + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp + +.Lblocks_avx_epilogue: + jmp .Ldo_avx + + +.p2align 5 +.Lbase2_64_avx: + + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -8(%rsp),%rsp + +.Lbase2_64_avx_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movl 16(%rdi),%ebp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + testq $31,%rdx + jz .Linit_avx + + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + +.Linit_avx: + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r8 + movq %rbx,%r9 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r8 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r8,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r9 + andq $0x3ffffff,%rbx + orq %r9,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + movl $1,20(%rdi) + + call __crypton_poly1305_asm_init_avx + + movq %r15,%rdx + + movq 8(%rsp),%r15 + + movq 16(%rsp),%r14 + + movq 24(%rsp),%r13 + + movq 32(%rsp),%r12 + + movq 40(%rsp),%rbp + + movq 48(%rsp),%rbx + + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp + +.Lbase2_64_avx_epilogue: + jmp .Ldo_avx + + +.p2align 5 +.Leven_avx: + + vmovd 0(%rdi),%xmm0 + vmovd 4(%rdi),%xmm1 + vmovd 8(%rdi),%xmm2 + vmovd 12(%rdi),%xmm3 + vmovd 16(%rdi),%xmm4 + +.Ldo_avx: + leaq -248(%rsp),%r11 + subq $0x218,%rsp + vmovdqa %xmm6,80(%r11) + vmovdqa %xmm7,96(%r11) + vmovdqa %xmm8,112(%r11) + vmovdqa %xmm9,128(%r11) + vmovdqa %xmm10,144(%r11) + vmovdqa %xmm11,160(%r11) + vmovdqa %xmm12,176(%r11) + vmovdqa %xmm13,192(%r11) + vmovdqa %xmm14,208(%r11) + vmovdqa %xmm15,224(%r11) +.Ldo_avx_body: + subq $64,%rdx + leaq -32(%rsi),%rax + cmovcq %rax,%rsi + + vmovdqu 48(%rdi),%xmm14 + leaq 112(%rdi),%rdi + leaq .Lconst(%rip),%rcx + + + + vmovdqu 32(%rsi),%xmm5 + vmovdqu 48(%rsi),%xmm6 + vmovdqa 64(%rcx),%xmm15 + + vpsrldq $6,%xmm5,%xmm7 + vpsrldq $6,%xmm6,%xmm8 + vpunpckhqdq %xmm6,%xmm5,%xmm9 + vpunpcklqdq %xmm6,%xmm5,%xmm5 + vpunpcklqdq %xmm8,%xmm7,%xmm8 + + vpsrlq $40,%xmm9,%xmm9 + vpsrlq $26,%xmm5,%xmm6 + vpand %xmm15,%xmm5,%xmm5 + vpsrlq $4,%xmm8,%xmm7 + vpand %xmm15,%xmm6,%xmm6 + vpsrlq $30,%xmm8,%xmm8 + vpand %xmm15,%xmm7,%xmm7 + vpand %xmm15,%xmm8,%xmm8 + vpor 32(%rcx),%xmm9,%xmm9 + + jbe .Lskip_loop_avx + + + vmovdqu -48(%rdi),%xmm11 + vmovdqu -32(%rdi),%xmm12 + vpshufd $0xEE,%xmm14,%xmm13 + vpshufd $0x44,%xmm14,%xmm10 + vmovdqa %xmm13,-144(%r11) + vmovdqa %xmm10,0(%rsp) + vpshufd $0xEE,%xmm11,%xmm14 + vmovdqu -16(%rdi),%xmm10 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm14,-128(%r11) + vmovdqa %xmm11,16(%rsp) + vpshufd $0xEE,%xmm12,%xmm13 + vmovdqu 0(%rdi),%xmm11 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm13,-112(%r11) + vmovdqa %xmm12,32(%rsp) + vpshufd $0xEE,%xmm10,%xmm14 + vmovdqu 16(%rdi),%xmm12 + vpshufd $0x44,%xmm10,%xmm10 + vmovdqa %xmm14,-96(%r11) + vmovdqa %xmm10,48(%rsp) + vpshufd $0xEE,%xmm11,%xmm13 + vmovdqu 32(%rdi),%xmm10 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm13,-80(%r11) + vmovdqa %xmm11,64(%rsp) + vpshufd $0xEE,%xmm12,%xmm14 + vmovdqu 48(%rdi),%xmm11 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm14,-64(%r11) + vmovdqa %xmm12,80(%rsp) + vpshufd $0xEE,%xmm10,%xmm13 + vmovdqu 64(%rdi),%xmm12 + vpshufd $0x44,%xmm10,%xmm10 + vmovdqa %xmm13,-48(%r11) + vmovdqa %xmm10,96(%rsp) + vpshufd $0xEE,%xmm11,%xmm14 + vpshufd $0x44,%xmm11,%xmm11 + vmovdqa %xmm14,-32(%r11) + vmovdqa %xmm11,112(%rsp) + vpshufd $0xEE,%xmm12,%xmm13 + vmovdqa 0(%rsp),%xmm14 + vpshufd $0x44,%xmm12,%xmm12 + vmovdqa %xmm13,-16(%r11) + vmovdqa %xmm12,128(%rsp) + + jmp .Loop_avx + +.p2align 5 +.Loop_avx: + + + + + + + + + + + + + + + + + + + + + vpmuludq %xmm5,%xmm14,%xmm10 + vpmuludq %xmm6,%xmm14,%xmm11 + vmovdqa %xmm2,32(%r11) + vpmuludq %xmm7,%xmm14,%xmm12 + vmovdqa 16(%rsp),%xmm2 + vpmuludq %xmm8,%xmm14,%xmm13 + vpmuludq %xmm9,%xmm14,%xmm14 + + vmovdqa %xmm0,0(%r11) + vpmuludq 32(%rsp),%xmm9,%xmm0 + vmovdqa %xmm1,16(%r11) + vpmuludq %xmm8,%xmm2,%xmm1 + vpaddq %xmm0,%xmm10,%xmm10 + vpaddq %xmm1,%xmm14,%xmm14 + vmovdqa %xmm3,48(%r11) + vpmuludq %xmm7,%xmm2,%xmm0 + vpmuludq %xmm6,%xmm2,%xmm1 + vpaddq %xmm0,%xmm13,%xmm13 + vmovdqa 48(%rsp),%xmm3 + vpaddq %xmm1,%xmm12,%xmm12 + vmovdqa %xmm4,64(%r11) + vpmuludq %xmm5,%xmm2,%xmm2 + vpmuludq %xmm7,%xmm3,%xmm0 + vpaddq %xmm2,%xmm11,%xmm11 + + vmovdqa 64(%rsp),%xmm4 + vpaddq %xmm0,%xmm14,%xmm14 + vpmuludq %xmm6,%xmm3,%xmm1 + vpmuludq %xmm5,%xmm3,%xmm3 + vpaddq %xmm1,%xmm13,%xmm13 + vmovdqa 80(%rsp),%xmm2 + vpaddq %xmm3,%xmm12,%xmm12 + vpmuludq %xmm9,%xmm4,%xmm0 + vpmuludq %xmm8,%xmm4,%xmm4 + vpaddq %xmm0,%xmm11,%xmm11 + vmovdqa 96(%rsp),%xmm3 + vpaddq %xmm4,%xmm10,%xmm10 + + vmovdqa 128(%rsp),%xmm4 + vpmuludq %xmm6,%xmm2,%xmm1 + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm1,%xmm14,%xmm14 + vpaddq %xmm2,%xmm13,%xmm13 + vpmuludq %xmm9,%xmm3,%xmm0 + vpmuludq %xmm8,%xmm3,%xmm1 + vpaddq %xmm0,%xmm12,%xmm12 + vmovdqu 0(%rsi),%xmm0 + vpaddq %xmm1,%xmm11,%xmm11 + vpmuludq %xmm7,%xmm3,%xmm3 + vpmuludq %xmm7,%xmm4,%xmm7 + vpaddq %xmm3,%xmm10,%xmm10 + + vmovdqu 16(%rsi),%xmm1 + vpaddq %xmm7,%xmm11,%xmm11 + vpmuludq %xmm8,%xmm4,%xmm8 + vpmuludq %xmm9,%xmm4,%xmm9 + vpsrldq $6,%xmm0,%xmm2 + vpaddq %xmm8,%xmm12,%xmm12 + vpaddq %xmm9,%xmm13,%xmm13 + vpsrldq $6,%xmm1,%xmm3 + vpmuludq 112(%rsp),%xmm5,%xmm9 + vpmuludq %xmm6,%xmm4,%xmm5 + vpunpckhqdq %xmm1,%xmm0,%xmm4 + vpaddq %xmm9,%xmm14,%xmm14 + vmovdqa -144(%r11),%xmm9 + vpaddq %xmm5,%xmm10,%xmm10 + + vpunpcklqdq %xmm1,%xmm0,%xmm0 + vpunpcklqdq %xmm3,%xmm2,%xmm3 + + + vpsrldq $5,%xmm4,%xmm4 + vpsrlq $26,%xmm0,%xmm1 + vpand %xmm15,%xmm0,%xmm0 + vpsrlq $4,%xmm3,%xmm2 + vpand %xmm15,%xmm1,%xmm1 + vpand 0(%rcx),%xmm4,%xmm4 + vpsrlq $30,%xmm3,%xmm3 + vpand %xmm15,%xmm2,%xmm2 + vpand %xmm15,%xmm3,%xmm3 + vpor 32(%rcx),%xmm4,%xmm4 + + vpaddq 0(%r11),%xmm0,%xmm0 + vpaddq 16(%r11),%xmm1,%xmm1 + vpaddq 32(%r11),%xmm2,%xmm2 + vpaddq 48(%r11),%xmm3,%xmm3 + vpaddq 64(%r11),%xmm4,%xmm4 + + leaq 32(%rsi),%rax + leaq 64(%rsi),%rsi + subq $64,%rdx + cmovcq %rax,%rsi + + + + + + + + + + + vpmuludq %xmm0,%xmm9,%xmm5 + vpmuludq %xmm1,%xmm9,%xmm6 + vpaddq %xmm5,%xmm10,%xmm10 + vpaddq %xmm6,%xmm11,%xmm11 + vmovdqa -128(%r11),%xmm7 + vpmuludq %xmm2,%xmm9,%xmm5 + vpmuludq %xmm3,%xmm9,%xmm6 + vpaddq %xmm5,%xmm12,%xmm12 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm4,%xmm9,%xmm9 + vpmuludq -112(%r11),%xmm4,%xmm5 + vpaddq %xmm9,%xmm14,%xmm14 + + vpaddq %xmm5,%xmm10,%xmm10 + vpmuludq %xmm2,%xmm7,%xmm6 + vpmuludq %xmm3,%xmm7,%xmm5 + vpaddq %xmm6,%xmm13,%xmm13 + vmovdqa -96(%r11),%xmm8 + vpaddq %xmm5,%xmm14,%xmm14 + vpmuludq %xmm1,%xmm7,%xmm6 + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm6,%xmm12,%xmm12 + vpaddq %xmm7,%xmm11,%xmm11 + + vmovdqa -80(%r11),%xmm9 + vpmuludq %xmm2,%xmm8,%xmm5 + vpmuludq %xmm1,%xmm8,%xmm6 + vpaddq %xmm5,%xmm14,%xmm14 + vpaddq %xmm6,%xmm13,%xmm13 + vmovdqa -64(%r11),%xmm7 + vpmuludq %xmm0,%xmm8,%xmm8 + vpmuludq %xmm4,%xmm9,%xmm5 + vpaddq %xmm8,%xmm12,%xmm12 + vpaddq %xmm5,%xmm11,%xmm11 + vmovdqa -48(%r11),%xmm8 + vpmuludq %xmm3,%xmm9,%xmm9 + vpmuludq %xmm1,%xmm7,%xmm6 + vpaddq %xmm9,%xmm10,%xmm10 + + vmovdqa -16(%r11),%xmm9 + vpaddq %xmm6,%xmm14,%xmm14 + vpmuludq %xmm0,%xmm7,%xmm7 + vpmuludq %xmm4,%xmm8,%xmm5 + vpaddq %xmm7,%xmm13,%xmm13 + vpaddq %xmm5,%xmm12,%xmm12 + vmovdqu 32(%rsi),%xmm5 + vpmuludq %xmm3,%xmm8,%xmm7 + vpmuludq %xmm2,%xmm8,%xmm8 + vpaddq %xmm7,%xmm11,%xmm11 + vmovdqu 48(%rsi),%xmm6 + vpaddq %xmm8,%xmm10,%xmm10 + + vpmuludq %xmm2,%xmm9,%xmm2 + vpmuludq %xmm3,%xmm9,%xmm3 + vpsrldq $6,%xmm5,%xmm7 + vpaddq %xmm2,%xmm11,%xmm11 + vpmuludq %xmm4,%xmm9,%xmm4 + vpsrldq $6,%xmm6,%xmm8 + vpaddq %xmm3,%xmm12,%xmm2 + vpaddq %xmm4,%xmm13,%xmm3 + vpmuludq -32(%r11),%xmm0,%xmm4 + vpmuludq %xmm1,%xmm9,%xmm0 + vpunpckhqdq %xmm6,%xmm5,%xmm9 + vpaddq %xmm4,%xmm14,%xmm4 + vpaddq %xmm0,%xmm10,%xmm0 + + vpunpcklqdq %xmm6,%xmm5,%xmm5 + vpunpcklqdq %xmm8,%xmm7,%xmm8 + + + vpsrldq $5,%xmm9,%xmm9 + vpsrlq $26,%xmm5,%xmm6 + vmovdqa 0(%rsp),%xmm14 + vpand %xmm15,%xmm5,%xmm5 + vpsrlq $4,%xmm8,%xmm7 + vpand %xmm15,%xmm6,%xmm6 + vpand 0(%rcx),%xmm9,%xmm9 + vpsrlq $30,%xmm8,%xmm8 + vpand %xmm15,%xmm7,%xmm7 + vpand %xmm15,%xmm8,%xmm8 + vpor 32(%rcx),%xmm9,%xmm9 + + + + + + vpsrlq $26,%xmm3,%xmm13 + vpand %xmm15,%xmm3,%xmm3 + vpaddq %xmm13,%xmm4,%xmm4 + + vpsrlq $26,%xmm0,%xmm10 + vpand %xmm15,%xmm0,%xmm0 + vpaddq %xmm10,%xmm11,%xmm1 + + vpsrlq $26,%xmm4,%xmm10 + vpand %xmm15,%xmm4,%xmm4 + + vpsrlq $26,%xmm1,%xmm11 + vpand %xmm15,%xmm1,%xmm1 + vpaddq %xmm11,%xmm2,%xmm2 + + vpaddq %xmm10,%xmm0,%xmm0 + vpsllq $2,%xmm10,%xmm10 + vpaddq %xmm10,%xmm0,%xmm0 + + vpsrlq $26,%xmm2,%xmm12 + vpand %xmm15,%xmm2,%xmm2 + vpaddq %xmm12,%xmm3,%xmm3 + + vpsrlq $26,%xmm0,%xmm10 + vpand %xmm15,%xmm0,%xmm0 + vpaddq %xmm10,%xmm1,%xmm1 + + vpsrlq $26,%xmm3,%xmm13 + vpand %xmm15,%xmm3,%xmm3 + vpaddq %xmm13,%xmm4,%xmm4 + + ja .Loop_avx + +.Lskip_loop_avx: + + + + vpshufd $0x10,%xmm14,%xmm14 + addq $32,%rdx + jnz .Long_tail_avx + + vpaddq %xmm2,%xmm7,%xmm7 + vpaddq %xmm0,%xmm5,%xmm5 + vpaddq %xmm1,%xmm6,%xmm6 + vpaddq %xmm3,%xmm8,%xmm8 + vpaddq %xmm4,%xmm9,%xmm9 + +.Long_tail_avx: + vmovdqa %xmm2,32(%r11) + vmovdqa %xmm0,0(%r11) + vmovdqa %xmm1,16(%r11) + vmovdqa %xmm3,48(%r11) + vmovdqa %xmm4,64(%r11) + + + + + + + + vpmuludq %xmm7,%xmm14,%xmm12 + vpmuludq %xmm5,%xmm14,%xmm10 + vpshufd $0x10,-48(%rdi),%xmm2 + vpmuludq %xmm6,%xmm14,%xmm11 + vpmuludq %xmm8,%xmm14,%xmm13 + vpmuludq %xmm9,%xmm14,%xmm14 + + vpmuludq %xmm8,%xmm2,%xmm0 + vpaddq %xmm0,%xmm14,%xmm14 + vpshufd $0x10,-32(%rdi),%xmm3 + vpmuludq %xmm7,%xmm2,%xmm1 + vpaddq %xmm1,%xmm13,%xmm13 + vpshufd $0x10,-16(%rdi),%xmm4 + vpmuludq %xmm6,%xmm2,%xmm0 + vpaddq %xmm0,%xmm12,%xmm12 + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm2,%xmm11,%xmm11 + vpmuludq %xmm9,%xmm3,%xmm3 + vpaddq %xmm3,%xmm10,%xmm10 + + vpshufd $0x10,0(%rdi),%xmm2 + vpmuludq %xmm7,%xmm4,%xmm1 + vpaddq %xmm1,%xmm14,%xmm14 + vpmuludq %xmm6,%xmm4,%xmm0 + vpaddq %xmm0,%xmm13,%xmm13 + vpshufd $0x10,16(%rdi),%xmm3 + vpmuludq %xmm5,%xmm4,%xmm4 + vpaddq %xmm4,%xmm12,%xmm12 + vpmuludq %xmm9,%xmm2,%xmm1 + vpaddq %xmm1,%xmm11,%xmm11 + vpshufd $0x10,32(%rdi),%xmm4 + vpmuludq %xmm8,%xmm2,%xmm2 + vpaddq %xmm2,%xmm10,%xmm10 + + vpmuludq %xmm6,%xmm3,%xmm0 + vpaddq %xmm0,%xmm14,%xmm14 + vpmuludq %xmm5,%xmm3,%xmm3 + vpaddq %xmm3,%xmm13,%xmm13 + vpshufd $0x10,48(%rdi),%xmm2 + vpmuludq %xmm9,%xmm4,%xmm1 + vpaddq %xmm1,%xmm12,%xmm12 + vpshufd $0x10,64(%rdi),%xmm3 + vpmuludq %xmm8,%xmm4,%xmm0 + vpaddq %xmm0,%xmm11,%xmm11 + vpmuludq %xmm7,%xmm4,%xmm4 + vpaddq %xmm4,%xmm10,%xmm10 + + vpmuludq %xmm5,%xmm2,%xmm2 + vpaddq %xmm2,%xmm14,%xmm14 + vpmuludq %xmm9,%xmm3,%xmm1 + vpaddq %xmm1,%xmm13,%xmm13 + vpmuludq %xmm8,%xmm3,%xmm0 + vpaddq %xmm0,%xmm12,%xmm12 + vpmuludq %xmm7,%xmm3,%xmm1 + vpaddq %xmm1,%xmm11,%xmm11 + vpmuludq %xmm6,%xmm3,%xmm3 + vpaddq %xmm3,%xmm10,%xmm10 + + jz .Lshort_tail_avx + + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + + vpsrldq $6,%xmm0,%xmm2 + vpsrldq $6,%xmm1,%xmm3 + vpunpckhqdq %xmm1,%xmm0,%xmm4 + vpunpcklqdq %xmm1,%xmm0,%xmm0 + vpunpcklqdq %xmm3,%xmm2,%xmm3 + + vpsrlq $40,%xmm4,%xmm4 + vpsrlq $26,%xmm0,%xmm1 + vpand %xmm15,%xmm0,%xmm0 + vpsrlq $4,%xmm3,%xmm2 + vpand %xmm15,%xmm1,%xmm1 + vpsrlq $30,%xmm3,%xmm3 + vpand %xmm15,%xmm2,%xmm2 + vpand %xmm15,%xmm3,%xmm3 + vpor 32(%rcx),%xmm4,%xmm4 + + vpshufd $0x32,-64(%rdi),%xmm9 + vpaddq 0(%r11),%xmm0,%xmm0 + vpaddq 16(%r11),%xmm1,%xmm1 + vpaddq 32(%r11),%xmm2,%xmm2 + vpaddq 48(%r11),%xmm3,%xmm3 + vpaddq 64(%r11),%xmm4,%xmm4 + + + + + vpmuludq %xmm0,%xmm9,%xmm5 + vpaddq %xmm5,%xmm10,%xmm10 + vpmuludq %xmm1,%xmm9,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpmuludq %xmm2,%xmm9,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpshufd $0x32,-48(%rdi),%xmm7 + vpmuludq %xmm3,%xmm9,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm4,%xmm9,%xmm9 + vpaddq %xmm9,%xmm14,%xmm14 + + vpmuludq %xmm3,%xmm7,%xmm5 + vpaddq %xmm5,%xmm14,%xmm14 + vpshufd $0x32,-32(%rdi),%xmm8 + vpmuludq %xmm2,%xmm7,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpshufd $0x32,-16(%rdi),%xmm9 + vpmuludq %xmm1,%xmm7,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm7,%xmm11,%xmm11 + vpmuludq %xmm4,%xmm8,%xmm8 + vpaddq %xmm8,%xmm10,%xmm10 + + vpshufd $0x32,0(%rdi),%xmm7 + vpmuludq %xmm2,%xmm9,%xmm6 + vpaddq %xmm6,%xmm14,%xmm14 + vpmuludq %xmm1,%xmm9,%xmm5 + vpaddq %xmm5,%xmm13,%xmm13 + vpshufd $0x32,16(%rdi),%xmm8 + vpmuludq %xmm0,%xmm9,%xmm9 + vpaddq %xmm9,%xmm12,%xmm12 + vpmuludq %xmm4,%xmm7,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpshufd $0x32,32(%rdi),%xmm9 + vpmuludq %xmm3,%xmm7,%xmm7 + vpaddq %xmm7,%xmm10,%xmm10 + + vpmuludq %xmm1,%xmm8,%xmm5 + vpaddq %xmm5,%xmm14,%xmm14 + vpmuludq %xmm0,%xmm8,%xmm8 + vpaddq %xmm8,%xmm13,%xmm13 + vpshufd $0x32,48(%rdi),%xmm7 + vpmuludq %xmm4,%xmm9,%xmm6 + vpaddq %xmm6,%xmm12,%xmm12 + vpshufd $0x32,64(%rdi),%xmm8 + vpmuludq %xmm3,%xmm9,%xmm5 + vpaddq %xmm5,%xmm11,%xmm11 + vpmuludq %xmm2,%xmm9,%xmm9 + vpaddq %xmm9,%xmm10,%xmm10 + + vpmuludq %xmm0,%xmm7,%xmm7 + vpaddq %xmm7,%xmm14,%xmm14 + vpmuludq %xmm4,%xmm8,%xmm6 + vpaddq %xmm6,%xmm13,%xmm13 + vpmuludq %xmm3,%xmm8,%xmm5 + vpaddq %xmm5,%xmm12,%xmm12 + vpmuludq %xmm2,%xmm8,%xmm6 + vpaddq %xmm6,%xmm11,%xmm11 + vpmuludq %xmm1,%xmm8,%xmm8 + vpaddq %xmm8,%xmm10,%xmm10 + +.Lshort_tail_avx: + + + + vpsrldq $8,%xmm14,%xmm9 + vpsrldq $8,%xmm13,%xmm8 + vpsrldq $8,%xmm11,%xmm6 + vpsrldq $8,%xmm10,%xmm5 + vpsrldq $8,%xmm12,%xmm7 + vpaddq %xmm8,%xmm13,%xmm13 + vpaddq %xmm9,%xmm14,%xmm14 + vpaddq %xmm5,%xmm10,%xmm10 + vpaddq %xmm6,%xmm11,%xmm11 + vpaddq %xmm7,%xmm12,%xmm12 + + + + + vpsrlq $26,%xmm13,%xmm3 + vpand %xmm15,%xmm13,%xmm13 + vpaddq %xmm3,%xmm14,%xmm14 + + vpsrlq $26,%xmm10,%xmm0 + vpand %xmm15,%xmm10,%xmm10 + vpaddq %xmm0,%xmm11,%xmm11 + + vpsrlq $26,%xmm14,%xmm4 + vpand %xmm15,%xmm14,%xmm14 + + vpsrlq $26,%xmm11,%xmm1 + vpand %xmm15,%xmm11,%xmm11 + vpaddq %xmm1,%xmm12,%xmm12 + + vpaddq %xmm4,%xmm10,%xmm10 + vpsllq $2,%xmm4,%xmm4 + vpaddq %xmm4,%xmm10,%xmm10 + + vpsrlq $26,%xmm12,%xmm2 + vpand %xmm15,%xmm12,%xmm12 + vpaddq %xmm2,%xmm13,%xmm13 + + vpsrlq $26,%xmm10,%xmm0 + vpand %xmm15,%xmm10,%xmm10 + vpaddq %xmm0,%xmm11,%xmm11 + + vpsrlq $26,%xmm13,%xmm3 + vpand %xmm15,%xmm13,%xmm13 + vpaddq %xmm3,%xmm14,%xmm14 + + vmovd %xmm10,-112(%rdi) + vmovd %xmm11,-108(%rdi) + vmovd %xmm12,-104(%rdi) + vmovd %xmm13,-100(%rdi) + vmovd %xmm14,-96(%rdi) + vmovdqa 80(%r11),%xmm6 + vmovdqa 96(%r11),%xmm7 + vmovdqa 112(%r11),%xmm8 + vmovdqa 128(%r11),%xmm9 + vmovdqa 144(%r11),%xmm10 + vmovdqa 160(%r11),%xmm11 + vmovdqa 176(%r11),%xmm12 + vmovdqa 192(%r11),%xmm13 + vmovdqa 208(%r11),%xmm14 + vmovdqa 224(%r11),%xmm15 + leaq 248(%r11),%rsp +.Ldo_avx_epilogue: + vzeroupper + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_poly1305_asm_blocks_avx: +.def crypton_poly1305_asm_blocks_avx2; .scl 3; .type 32; .endef +.p2align 5 +crypton_poly1305_asm_blocks_avx2: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%rax +.LSEH_begin_crypton_poly1305_asm_blocks_avx2: + + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + movq %r9,%rcx + movl 20(%rdi),%r8d + cmpq $128,%rdx + jb .Lblocks + + andq $-16,%rdx + + vzeroupper + + testl %r8d,%r8d + jz .Lbase2_64_avx2 + + testq $63,%rdx + jz .Leven_avx2 + + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -8(%rsp),%rsp + +.Lblocks_avx2_body: + + movq %rdx,%r15 + + movq 0(%rdi),%r8 + movq 8(%rdi),%r9 + movl 16(%rdi),%ebp + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + + movl %r8d,%r14d + andq $-2147483648,%r8 + movq %r9,%r12 + movl %r9d,%ebx + andq $-2147483648,%r9 + + shrq $6,%r8 + shlq $52,%r12 + addq %r8,%r14 + shrq $12,%rbx + shrq $18,%r9 + addq %r12,%r14 + adcq %r9,%rbx + + movq %rbp,%r8 + shlq $40,%r8 + shrq $24,%rbp + addq %r8,%rbx + adcq $0,%rbp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + +.Lbase2_26_pre_avx2: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + movq %r12,%rax + + testq $63,%r15 + jnz .Lbase2_26_pre_avx2 + + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r11 + movq %rbx,%r12 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r11 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r11,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r12 + andq $0x3ffffff,%rbx + orq %r12,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + + movq %r15,%rdx + + movq 8(%rsp),%r15 + + movq 16(%rsp),%r14 + + movq 24(%rsp),%r13 + + movq 32(%rsp),%r12 + + movq 40(%rsp),%rbp + + movq 48(%rsp),%rbx + + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp + +.Lblocks_avx2_epilogue: + jmp .Ldo_avx2 + + +.p2align 5 +.Lbase2_64_avx2: + + pushq %rbx + + pushq %rbp + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + leaq -8(%rsp),%rsp + +.Lbase2_64_avx2_body: + + movq %rdx,%r15 + + movq 24(%rdi),%r11 + movq 32(%rdi),%r13 + + movq 0(%rdi),%r14 + movq 8(%rdi),%rbx + movl 16(%rdi),%ebp + + movq %r13,%r12 + movq %r13,%rax + shrq $2,%r13 + addq %r12,%r13 + + testq $63,%rdx + jz .Linit_avx2 + +.Lbase2_64_pre_avx2: + addq 0(%rsi),%r14 + adcq 8(%rsi),%rbx + leaq 16(%rsi),%rsi + adcq %rcx,%rbp + subq $16,%r15 + + call __crypton_poly1305_asm_block + movq %r12,%rax + + testq $63,%r15 + jnz .Lbase2_64_pre_avx2 + +.Linit_avx2: + + movq %r14,%rax + movq %r14,%rdx + shrq $52,%r14 + movq %rbx,%r8 + movq %rbx,%r9 + shrq $26,%rdx + andq $0x3ffffff,%rax + shlq $12,%r8 + andq $0x3ffffff,%rdx + shrq $14,%rbx + orq %r8,%r14 + shlq $24,%rbp + andq $0x3ffffff,%r14 + shrq $40,%r9 + andq $0x3ffffff,%rbx + orq %r9,%rbp + + vmovd %eax,%xmm0 + vmovd %edx,%xmm1 + vmovd %r14d,%xmm2 + vmovd %ebx,%xmm3 + vmovd %ebp,%xmm4 + movl $1,20(%rdi) + + call __crypton_poly1305_asm_init_avx + + movq %r15,%rdx + + movq 8(%rsp),%r15 + + movq 16(%rsp),%r14 + + movq 24(%rsp),%r13 + + movq 32(%rsp),%r12 + + movq 40(%rsp),%rbp + + movq 48(%rsp),%rbx + + leaq 56(%rsp),%rax + leaq 56(%rsp),%rsp + +.Lbase2_64_avx2_epilogue: + jmp .Ldo_avx2 + + +.p2align 5 +.Leven_avx2: + + vmovd 0(%rdi),%xmm0 + vmovd 4(%rdi),%xmm1 + vmovd 8(%rdi),%xmm2 + vmovd 12(%rdi),%xmm3 + vmovd 16(%rdi),%xmm4 + +.Ldo_avx2: + leaq -248(%rsp),%r11 + subq $0x1c8,%rsp + vmovdqa %xmm6,80(%r11) + vmovdqa %xmm7,96(%r11) + vmovdqa %xmm8,112(%r11) + vmovdqa %xmm9,128(%r11) + vmovdqa %xmm10,144(%r11) + vmovdqa %xmm11,160(%r11) + vmovdqa %xmm12,176(%r11) + vmovdqa %xmm13,192(%r11) + vmovdqa %xmm14,208(%r11) + vmovdqa %xmm15,224(%r11) +.Ldo_avx2_body: + leaq .Lconst(%rip),%rcx + leaq 48+64(%rdi),%rdi + vmovdqa 96(%rcx),%ymm7 + + + vmovdqu -64(%rdi),%xmm9 + andq $-512,%rsp + vmovdqu -48(%rdi),%xmm10 + vmovdqu -32(%rdi),%xmm6 + vmovdqu -16(%rdi),%xmm11 + vmovdqu 0(%rdi),%xmm12 + vmovdqu 16(%rdi),%xmm13 + leaq 144(%rsp),%rax + vmovdqu 32(%rdi),%xmm14 + vpermd %ymm9,%ymm7,%ymm9 + vmovdqu 48(%rdi),%xmm15 + vpermd %ymm10,%ymm7,%ymm10 + vmovdqu 64(%rdi),%xmm5 + vpermd %ymm6,%ymm7,%ymm6 + vmovdqa %ymm9,0(%rsp) + vpermd %ymm11,%ymm7,%ymm11 + vmovdqa %ymm10,32-144(%rax) + vpermd %ymm12,%ymm7,%ymm12 + vmovdqa %ymm6,64-144(%rax) + vpermd %ymm13,%ymm7,%ymm13 + vmovdqa %ymm11,96-144(%rax) + vpermd %ymm14,%ymm7,%ymm14 + vmovdqa %ymm12,128-144(%rax) + vpermd %ymm15,%ymm7,%ymm15 + vmovdqa %ymm13,160-144(%rax) + vpermd %ymm5,%ymm7,%ymm5 + vmovdqa %ymm14,192-144(%rax) + vmovdqa %ymm15,224-144(%rax) + vmovdqa %ymm5,256-144(%rax) + vmovdqa 64(%rcx),%ymm5 + + + + vmovdqu 0(%rsi),%xmm7 + vmovdqu 16(%rsi),%xmm8 + vinserti128 $1,32(%rsi),%ymm7,%ymm7 + vinserti128 $1,48(%rsi),%ymm8,%ymm8 + leaq 64(%rsi),%rsi + + vpsrldq $6,%ymm7,%ymm9 + vpsrldq $6,%ymm8,%ymm10 + vpunpckhqdq %ymm8,%ymm7,%ymm6 + vpunpcklqdq %ymm10,%ymm9,%ymm9 + vpunpcklqdq %ymm8,%ymm7,%ymm7 + + vpsrlq $30,%ymm9,%ymm10 + vpsrlq $4,%ymm9,%ymm9 + vpsrlq $26,%ymm7,%ymm8 + vpsrlq $40,%ymm6,%ymm6 + vpand %ymm5,%ymm9,%ymm9 + vpand %ymm5,%ymm7,%ymm7 + vpand %ymm5,%ymm8,%ymm8 + vpand %ymm5,%ymm10,%ymm10 + vpor 32(%rcx),%ymm6,%ymm6 + + vpaddq %ymm2,%ymm9,%ymm2 + subq $64,%rdx + jz .Ltail_avx2 + jmp .Loop_avx2 + +.p2align 5 +.Loop_avx2: + + + + + + + + + vpaddq %ymm0,%ymm7,%ymm0 + vmovdqa 0(%rsp),%ymm7 + vpaddq %ymm1,%ymm8,%ymm1 + vmovdqa 32(%rsp),%ymm8 + vpaddq %ymm3,%ymm10,%ymm3 + vmovdqa 96(%rsp),%ymm9 + vpaddq %ymm4,%ymm6,%ymm4 + vmovdqa 48(%rax),%ymm10 + vmovdqa 112(%rax),%ymm5 + + + + + + + + + + + + + + + + + vpmuludq %ymm2,%ymm7,%ymm13 + vpmuludq %ymm2,%ymm8,%ymm14 + vpmuludq %ymm2,%ymm9,%ymm15 + vpmuludq %ymm2,%ymm10,%ymm11 + vpmuludq %ymm2,%ymm5,%ymm12 + + vpmuludq %ymm0,%ymm8,%ymm6 + vpmuludq %ymm1,%ymm8,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq 64(%rsp),%ymm4,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm11,%ymm11 + vmovdqa -16(%rax),%ymm8 + + vpmuludq %ymm0,%ymm7,%ymm6 + vpmuludq %ymm1,%ymm7,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vpmuludq %ymm3,%ymm7,%ymm6 + vpmuludq %ymm4,%ymm7,%ymm2 + vmovdqu 0(%rsi),%xmm7 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm2,%ymm15,%ymm15 + vinserti128 $1,32(%rsi),%ymm7,%ymm7 + + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq %ymm4,%ymm8,%ymm2 + vmovdqu 16(%rsi),%xmm8 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vmovdqa 16(%rax),%ymm2 + vpmuludq %ymm1,%ymm9,%ymm6 + vpmuludq %ymm0,%ymm9,%ymm9 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm9,%ymm13,%ymm13 + vinserti128 $1,48(%rsi),%ymm8,%ymm8 + leaq 64(%rsi),%rsi + + vpmuludq %ymm1,%ymm2,%ymm6 + vpmuludq %ymm0,%ymm2,%ymm2 + vpsrldq $6,%ymm7,%ymm9 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm14,%ymm14 + vpmuludq %ymm3,%ymm10,%ymm6 + vpmuludq %ymm4,%ymm10,%ymm2 + vpsrldq $6,%ymm8,%ymm10 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpunpckhqdq %ymm8,%ymm7,%ymm6 + + vpmuludq %ymm3,%ymm5,%ymm3 + vpmuludq %ymm4,%ymm5,%ymm4 + vpunpcklqdq %ymm8,%ymm7,%ymm7 + vpaddq %ymm3,%ymm13,%ymm2 + vpaddq %ymm4,%ymm14,%ymm3 + vpunpcklqdq %ymm10,%ymm9,%ymm10 + vpmuludq 80(%rax),%ymm0,%ymm4 + vpmuludq %ymm1,%ymm5,%ymm0 + vmovdqa 64(%rcx),%ymm5 + vpaddq %ymm4,%ymm15,%ymm4 + vpaddq %ymm0,%ymm11,%ymm0 + + + + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm12,%ymm1 + + vpsrlq $26,%ymm4,%ymm15 + vpand %ymm5,%ymm4,%ymm4 + + vpsrlq $4,%ymm10,%ymm9 + + vpsrlq $26,%ymm1,%ymm12 + vpand %ymm5,%ymm1,%ymm1 + vpaddq %ymm12,%ymm2,%ymm2 + + vpaddq %ymm15,%ymm0,%ymm0 + vpsllq $2,%ymm15,%ymm15 + vpaddq %ymm15,%ymm0,%ymm0 + + vpand %ymm5,%ymm9,%ymm9 + vpsrlq $26,%ymm7,%ymm8 + + vpsrlq $26,%ymm2,%ymm13 + vpand %ymm5,%ymm2,%ymm2 + vpaddq %ymm13,%ymm3,%ymm3 + + vpaddq %ymm9,%ymm2,%ymm2 + vpsrlq $30,%ymm10,%ymm10 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm1,%ymm1 + + vpsrlq $40,%ymm6,%ymm6 + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpand %ymm5,%ymm7,%ymm7 + vpand %ymm5,%ymm8,%ymm8 + vpand %ymm5,%ymm10,%ymm10 + vpor 32(%rcx),%ymm6,%ymm6 + + subq $64,%rdx + jnz .Loop_avx2 + +.byte 0x66,0x90 +.Ltail_avx2: + + + + + + + + vpaddq %ymm0,%ymm7,%ymm0 + vmovdqu 4(%rsp),%ymm7 + vpaddq %ymm1,%ymm8,%ymm1 + vmovdqu 36(%rsp),%ymm8 + vpaddq %ymm3,%ymm10,%ymm3 + vmovdqu 100(%rsp),%ymm9 + vpaddq %ymm4,%ymm6,%ymm4 + vmovdqu 52(%rax),%ymm10 + vmovdqu 116(%rax),%ymm5 + + vpmuludq %ymm2,%ymm7,%ymm13 + vpmuludq %ymm2,%ymm8,%ymm14 + vpmuludq %ymm2,%ymm9,%ymm15 + vpmuludq %ymm2,%ymm10,%ymm11 + vpmuludq %ymm2,%ymm5,%ymm12 + + vpmuludq %ymm0,%ymm8,%ymm6 + vpmuludq %ymm1,%ymm8,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq 68(%rsp),%ymm4,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm11,%ymm11 + + vpmuludq %ymm0,%ymm7,%ymm6 + vpmuludq %ymm1,%ymm7,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vmovdqu -12(%rax),%ymm8 + vpaddq %ymm2,%ymm12,%ymm12 + vpmuludq %ymm3,%ymm7,%ymm6 + vpmuludq %ymm4,%ymm7,%ymm2 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm2,%ymm15,%ymm15 + + vpmuludq %ymm3,%ymm8,%ymm6 + vpmuludq %ymm4,%ymm8,%ymm2 + vpaddq %ymm6,%ymm11,%ymm11 + vpaddq %ymm2,%ymm12,%ymm12 + vmovdqu 20(%rax),%ymm2 + vpmuludq %ymm1,%ymm9,%ymm6 + vpmuludq %ymm0,%ymm9,%ymm9 + vpaddq %ymm6,%ymm14,%ymm14 + vpaddq %ymm9,%ymm13,%ymm13 + + vpmuludq %ymm1,%ymm2,%ymm6 + vpmuludq %ymm0,%ymm2,%ymm2 + vpaddq %ymm6,%ymm15,%ymm15 + vpaddq %ymm2,%ymm14,%ymm14 + vpmuludq %ymm3,%ymm10,%ymm6 + vpmuludq %ymm4,%ymm10,%ymm2 + vpaddq %ymm6,%ymm12,%ymm12 + vpaddq %ymm2,%ymm13,%ymm13 + + vpmuludq %ymm3,%ymm5,%ymm3 + vpmuludq %ymm4,%ymm5,%ymm4 + vpaddq %ymm3,%ymm13,%ymm2 + vpaddq %ymm4,%ymm14,%ymm3 + vpmuludq 84(%rax),%ymm0,%ymm4 + vpmuludq %ymm1,%ymm5,%ymm0 + vmovdqa 64(%rcx),%ymm5 + vpaddq %ymm4,%ymm15,%ymm4 + vpaddq %ymm0,%ymm11,%ymm0 + + + + + vpsrldq $8,%ymm12,%ymm8 + vpsrldq $8,%ymm2,%ymm9 + vpsrldq $8,%ymm3,%ymm10 + vpsrldq $8,%ymm4,%ymm6 + vpsrldq $8,%ymm0,%ymm7 + vpaddq %ymm8,%ymm12,%ymm12 + vpaddq %ymm9,%ymm2,%ymm2 + vpaddq %ymm10,%ymm3,%ymm3 + vpaddq %ymm6,%ymm4,%ymm4 + vpaddq %ymm7,%ymm0,%ymm0 + + vpermq $0x2,%ymm3,%ymm10 + vpermq $0x2,%ymm4,%ymm6 + vpermq $0x2,%ymm0,%ymm7 + vpermq $0x2,%ymm12,%ymm8 + vpermq $0x2,%ymm2,%ymm9 + vpaddq %ymm10,%ymm3,%ymm3 + vpaddq %ymm6,%ymm4,%ymm4 + vpaddq %ymm7,%ymm0,%ymm0 + vpaddq %ymm8,%ymm12,%ymm12 + vpaddq %ymm9,%ymm2,%ymm2 + + + + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm12,%ymm1 + + vpsrlq $26,%ymm4,%ymm15 + vpand %ymm5,%ymm4,%ymm4 + + vpsrlq $26,%ymm1,%ymm12 + vpand %ymm5,%ymm1,%ymm1 + vpaddq %ymm12,%ymm2,%ymm2 + + vpaddq %ymm15,%ymm0,%ymm0 + vpsllq $2,%ymm15,%ymm15 + vpaddq %ymm15,%ymm0,%ymm0 + + vpsrlq $26,%ymm2,%ymm13 + vpand %ymm5,%ymm2,%ymm2 + vpaddq %ymm13,%ymm3,%ymm3 + + vpsrlq $26,%ymm0,%ymm11 + vpand %ymm5,%ymm0,%ymm0 + vpaddq %ymm11,%ymm1,%ymm1 + + vpsrlq $26,%ymm3,%ymm14 + vpand %ymm5,%ymm3,%ymm3 + vpaddq %ymm14,%ymm4,%ymm4 + + vmovd %xmm0,-112(%rdi) + vmovd %xmm1,-108(%rdi) + vmovd %xmm2,-104(%rdi) + vmovd %xmm3,-100(%rdi) + vmovd %xmm4,-96(%rdi) + vmovdqa 80(%r11),%xmm6 + vmovdqa 96(%r11),%xmm7 + vmovdqa 112(%r11),%xmm8 + vmovdqa 128(%r11),%xmm9 + vmovdqa 144(%r11),%xmm10 + vmovdqa 160(%r11),%xmm11 + vmovdqa 176(%r11),%xmm12 + vmovdqa 192(%r11),%xmm13 + vmovdqa 208(%r11),%xmm14 + vmovdqa 224(%r11),%xmm15 + leaq 248(%r11),%rsp +.Ldo_avx2_epilogue: + vzeroupper + movq 8(%rsp),%rdi + movq 16(%rsp),%rsi + .byte 0xf3,0xc3 + +.LSEH_end_crypton_poly1305_asm_blocks_avx2: +.p2align 6 +.Lconst: +.Lmask24: +.long 0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0 +.L129: +.long 16777216,0,16777216,0,16777216,0,16777216,0 +.Lmask26: +.long 0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0 +.Lpermd_avx2: +.long 2,2,2,3,2,0,2,1 +.Lpermd_avx512: +.long 0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7 + +.L2_44_inp_permd: +.long 0,1,1,2,2,3,7,7 +.L2_44_inp_shift: +.quad 0,12,24,64 +.L2_44_mask: +.quad 0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff +.L2_44_shift_rgt: +.quad 44,44,42,64 +.L2_44_shift_lft: +.quad 8,8,10,64 + +.p2align 6 +.Lx_mask44: +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.Lx_mask42: +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.byte 80,111,108,121,49,51,48,53,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.p2align 4 +.globl crypton_xor128_encrypt_n_pad +.def crypton_xor128_encrypt_n_pad; .scl 2; .type 32; .endef +.p2align 4 +crypton_xor128_encrypt_n_pad: + .byte 0xf3,0x0f,0x1e,0xfa + + subq %r8,%rdx + subq %r8,%rcx + movq %r9,%r10 + shrq $4,%r9 + jz .Ltail_enc + nop +.Loop_enc_xmm: + movdqu (%rdx,%r8,1),%xmm0 + pxor (%r8),%xmm0 + movdqu %xmm0,(%rcx,%r8,1) + movdqa %xmm0,(%r8) + leaq 16(%r8),%r8 + decq %r9 + jnz .Loop_enc_xmm + + andq $15,%r10 + jz .Ldone_enc + +.Ltail_enc: + movq $16,%r9 + subq %r10,%r9 + xorl %eax,%eax +.Loop_enc_byte: + movb (%rdx,%r8,1),%al + xorb (%r8),%al + movb %al,(%rcx,%r8,1) + movb %al,(%r8) + leaq 1(%r8),%r8 + decq %r10 + jnz .Loop_enc_byte + + xorl %eax,%eax +.Loop_enc_pad: + movb %al,(%r8) + leaq 1(%r8),%r8 + decq %r9 + jnz .Loop_enc_pad + +.Ldone_enc: + movq %r8,%rax + .byte 0xf3,0xc3 + + +.globl crypton_xor128_decrypt_n_pad +.def crypton_xor128_decrypt_n_pad; .scl 2; .type 32; .endef +.p2align 4 +crypton_xor128_decrypt_n_pad: + .byte 0xf3,0x0f,0x1e,0xfa + + subq %r8,%rdx + subq %r8,%rcx + movq %r9,%r10 + shrq $4,%r9 + jz .Ltail_dec + nop +.Loop_dec_xmm: + movdqu (%rdx,%r8,1),%xmm0 + movdqa (%r8),%xmm1 + pxor %xmm0,%xmm1 + movdqu %xmm1,(%rcx,%r8,1) + movdqa %xmm0,(%r8) + leaq 16(%r8),%r8 + decq %r9 + jnz .Loop_dec_xmm + + pxor %xmm1,%xmm1 + andq $15,%r10 + jz .Ldone_dec + +.Ltail_dec: + movq $16,%r9 + subq %r10,%r9 + xorl %eax,%eax + xorq %r11,%r11 +.Loop_dec_byte: + movb (%rdx,%r8,1),%r11b + movb (%r8),%al + xorb %r11b,%al + movb %al,(%rcx,%r8,1) + movb %r11b,(%r8) + leaq 1(%r8),%r8 + decq %r10 + jnz .Loop_dec_byte + + xorl %eax,%eax +.Loop_dec_pad: + movb %al,(%r8) + leaq 1(%r8),%r8 + decq %r9 + jnz .Loop_dec_pad + +.Ldone_dec: + movq %r8,%rax + .byte 0xf3,0xc3 + + +.def se_handler; .scl 3; .type 32; .endef +.p2align 4 +se_handler: + .byte 0xf3,0x0f,0x1e,0xfa + + pushq %rsi + pushq %rdi + pushq %rbx + pushq %rbp + pushq %r12 + pushq %r13 + pushq %r14 + pushq %r15 + pushfq + subq $64,%rsp + + movq 120(%r8),%rax + movq 248(%r8),%rbx + + movq 8(%r9),%rsi + movq 56(%r9),%r11 + + movl 0(%r11),%r10d + leaq (%rsi,%r10,1),%r10 + cmpq %r10,%rbx + jb .Lcommon_seh_tail + + movq 152(%r8),%rax + + movl 4(%r11),%r10d + leaq (%rsi,%r10,1),%r10 + cmpq %r10,%rbx + jae .Lcommon_seh_tail + + leaq 56(%rax),%rax + + movq -8(%rax),%rbx + movq -16(%rax),%rbp + movq -24(%rax),%r12 + movq -32(%rax),%r13 + movq -40(%rax),%r14 + movq -48(%rax),%r15 + movq %rbx,144(%r8) + movq %rbp,160(%r8) + movq %r12,216(%r8) + movq %r13,224(%r8) + movq %r14,232(%r8) + movq %r15,240(%r8) + + jmp .Lcommon_seh_tail + + +.def avx_handler; .scl 3; .type 32; .endef +.p2align 4 +avx_handler: + .byte 0xf3,0x0f,0x1e,0xfa + + pushq %rsi + pushq %rdi + pushq %rbx + pushq %rbp + pushq %r12 + pushq %r13 + pushq %r14 + pushq %r15 + pushfq + subq $64,%rsp + + movq 120(%r8),%rax + movq 248(%r8),%rbx + + movq 8(%r9),%rsi + movq 56(%r9),%r11 + + movl 0(%r11),%r10d + leaq (%rsi,%r10,1),%r10 + cmpq %r10,%rbx + jb .Lcommon_seh_tail + + movq 152(%r8),%rax + + movl 4(%r11),%r10d + leaq (%rsi,%r10,1),%r10 + cmpq %r10,%rbx + jae .Lcommon_seh_tail + + movq 208(%r8),%rax + + leaq 80(%rax),%rsi + leaq 248(%rax),%rax + leaq 512(%r8),%rdi + movl $20,%ecx +.long 0xa548f3fc + +.Lcommon_seh_tail: + movq 8(%rax),%rdi + movq 16(%rax),%rsi + movq %rax,152(%r8) + movq %rsi,168(%r8) + movq %rdi,176(%r8) + + movq 40(%r9),%rdi + movq %r8,%rsi + movl $154,%ecx +.long 0xa548f3fc + + movq %r9,%rsi + xorq %rcx,%rcx + movq 8(%rsi),%rdx + movq 0(%rsi),%r8 + movq 16(%rsi),%r9 + movq 40(%rsi),%r10 + leaq 56(%rsi),%r11 + leaq 24(%rsi),%r12 + movq %r10,32(%rsp) + movq %r11,40(%rsp) + movq %r12,48(%rsp) + movq %rcx,56(%rsp) + call *__imp_RtlVirtualUnwind(%rip) + + movl $1,%eax + addq $64,%rsp + popfq + popq %r15 + popq %r14 + popq %r13 + popq %r12 + popq %rbp + popq %rbx + popq %rdi + popq %rsi + .byte 0xf3,0xc3 + + +.section .pdata +.p2align 2 +.rva .LSEH_begin_crypton_poly1305_asm_init +.rva .LSEH_end_crypton_poly1305_asm_init +.rva .LSEH_info_crypton_poly1305_asm_init + +.rva .LSEH_begin_crypton_poly1305_asm_blocks +.rva .LSEH_end_crypton_poly1305_asm_blocks +.rva .LSEH_info_crypton_poly1305_asm_blocks + +.rva .LSEH_begin_crypton_poly1305_asm_emit +.rva .LSEH_end_crypton_poly1305_asm_emit +.rva .LSEH_info_crypton_poly1305_asm_emit +.rva .LSEH_begin_crypton_poly1305_asm_blocks_avx +.rva .Lbase2_64_avx +.rva .LSEH_info_crypton_poly1305_asm_blocks_avx_1 + +.rva .Lbase2_64_avx +.rva .Leven_avx +.rva .LSEH_info_crypton_poly1305_asm_blocks_avx_2 + +.rva .Leven_avx +.rva .LSEH_end_crypton_poly1305_asm_blocks_avx +.rva .LSEH_info_crypton_poly1305_asm_blocks_avx_3 +.rva .LSEH_begin_crypton_poly1305_asm_blocks_avx2 +.rva .Lbase2_64_avx2 +.rva .LSEH_info_crypton_poly1305_asm_blocks_avx2_1 + +.rva .Lbase2_64_avx2 +.rva .Leven_avx2 +.rva .LSEH_info_crypton_poly1305_asm_blocks_avx2_2 + +.rva .Leven_avx2 +.rva .LSEH_end_crypton_poly1305_asm_blocks_avx2 +.rva .LSEH_info_crypton_poly1305_asm_blocks_avx2_3 +.section .xdata +.p2align 3 +.LSEH_info_crypton_poly1305_asm_init: +.byte 9,0,0,0 +.rva se_handler +.long 0,0 + +.LSEH_info_crypton_poly1305_asm_blocks: +.byte 9,0,0,0 +.rva se_handler +.rva .Lblocks_body,.Lblocks_epilogue + +.LSEH_info_crypton_poly1305_asm_emit: +.byte 9,0,0,0 +.rva se_handler +.long 0,0 +.LSEH_info_crypton_poly1305_asm_blocks_avx_1: +.byte 9,0,0,0 +.rva se_handler +.rva .Lblocks_avx_body,.Lblocks_avx_epilogue + +.LSEH_info_crypton_poly1305_asm_blocks_avx_2: +.byte 9,0,0,0 +.rva se_handler +.rva .Lbase2_64_avx_body,.Lbase2_64_avx_epilogue + +.LSEH_info_crypton_poly1305_asm_blocks_avx_3: +.byte 9,0,0,0 +.rva avx_handler +.rva .Ldo_avx_body,.Ldo_avx_epilogue +.LSEH_info_crypton_poly1305_asm_blocks_avx2_1: +.byte 9,0,0,0 +.rva se_handler +.rva .Lblocks_avx2_body,.Lblocks_avx2_epilogue + +.LSEH_info_crypton_poly1305_asm_blocks_avx2_2: +.byte 9,0,0,0 +.rva se_handler +.rva .Lbase2_64_avx2_body,.Lbase2_64_avx2_epilogue + +.LSEH_info_crypton_poly1305_asm_blocks_avx2_3: +.byte 9,0,0,0 +.rva avx_handler +.rva .Ldo_avx2_body,.Ldo_avx2_epilogue diff --git a/cbits/asm/poly1305-x86_64.pl b/cbits/asm/poly1305-x86_64.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/poly1305-x86_64.pl @@ -0,0 +1,4333 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# This module implements Poly1305 hash for x86_64. +# +# March 2015 +# +# Initial release. +# +# December 2016 +# +# Add AVX512F+VL+BW code path. +# +# November 2017 +# +# Convert AVX512F+VL+BW code path to pure AVX512F, so that it can be +# executed even on Knights Landing. Trigger for modification was +# observation that AVX512 code paths can negatively affect overall +# Skylake-X system performance. Since we are likely to suppress +# AVX512F capability flag [at least on Skylake-X], conversion serves +# as kind of "investment protection". Note that next *lake processor, +# Cannonlake, has AVX512IFMA code path to execute... +# +# Numbers are cycles per processed byte with poly1305_blocks alone, +# most are measured with rdtsc at fixed clock frequency. +# +# IALU/gcc-4.8(i) AVX(ii) AVX2 AVX-512 +# P4 4.46/+120% - +# Core 2 2.41/+90% - +# Westmere 1.88/+120% - +# Sandy Bridge 1.39/+140% 1.10 +# Haswell 1.14/+175% 1.11 0.65 +# Skylake[-X] 1.13/+120% 0.96 0.51 [0.35] +# Cannon Lake 1.13/+120% 0.93 0.38(iv)0.24(iv) +# Rocket Lake 1.13/+120% 0.84 0.43(iv)0.24(iv) +# Silvermont 2.83/+95% - +# Knights L 3.60/? 1.65 1.10 0.41(iii) +# Goldmont 1.70/+180% - +# VIA Nano 1.82/+150% - +# Sledgehammer 1.38/+160% - +# Bulldozer 2.30/+130% 0.97 +# Ryzen 1.15/+200% 1.08 1.18 +# +# (i) improvement coefficients relative to clang are more modest and +# are ~50% on most processors, in both cases we are comparing to +# __int128 code; +# (ii) SSE2 implementation was attempted, but among non-AVX processors +# it was faster than integer-only code only on older Intel P4 and +# Core processors, 50-30%, less newer processor is, but slower on +# contemporary ones, for example almost 2x slower on Atom, and as +# former are naturally disappearing, SSE2 is deemed unnecessary; +# (iii) strangely enough performance seems to vary from core to core, +# listed result is best case; +# (iv) these are IFMA results, which in addition means that first IALU +# column does not reflect short-input performance; + +$flavour = shift; +$output = shift; +if ($flavour =~ /\./) { $output = $flavour; undef $flavour; } + +$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/); + +$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; +( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or +( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or +die "can't locate x86_64-xlate.pl"; + +$avx=undef; + +if (!defined($avx) && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) && + ($ENV{ASM} //= "nasm") && + `"$ENV{ASM}" -v 2>&1` =~ /NASM version ([0-9]+\.[0-9]+)(?:\.([0-9]+))?/) { + $avx = ($1>=2.09) + ($1>=2.10) + 2 * ($1>=2.12); + $avx += 2 if ($1==2.11 && $2>=8); +} + +if (!defined($avx) && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) && + ($ENV{ASM} //= "ml64") && + `"$ENV{ASM}" 2>&1` =~ /Version ([0-9]+)\./) { + $avx = ($1>=10) + ($1>=12) + 2 * ($1>=14); +} + +$ENV{CC} //= "cc"; +if (!defined($avx) && `$ENV{CC} -Wa,-v -c -o /dev/zero -x assembler /dev/null 2>&1` + =~ /GNU assembler version ([0-9]+)\.([0-9]+)/) { + my $ver = $1 + $2/100.0; # 3.1->3.01, 3.10->3.10 + $avx = ($ver>=2.19) + ($ver>=2.22) + ($ver>=2.25) + ($ver>=2.26); +} + +if (!defined($avx) && `$ENV{CC} -v 2>&1` + =~ /((?:^clang|LLVM) version|.*based on LLVM) ([0-9]+)\.([0-9]+)/) { + my $ver = $2 + $3/100.0; # 3.1->3.01, 3.10->3.10 + $avx = ($ver>=3.0) + ($ver>3.0); + $avx += 2*($ver>=7.0) if ($1 =~ /^clang/); +} + +open OUT,"| \"$^X\" \"$xlate\" $flavour \"$output\""; +*STDOUT=*OUT; + +my ($ctx,$inp,$len,$padbit)=("%rdi","%rsi","%rdx","%rcx"); +my ($mac,$nonce)=($inp,$len); # *_emit arguments +my ($d1,$d2,$d3, $r0,$r1,$s1)=map("%r$_",(8..13)); +my ($h0,$h1,$h2)=("%r14","%rbx","%rbp"); + +sub poly1305_iteration { +# input: copy of $r1 in %rax, $h0-$h2, $r0-$r1 +# output: $h0-$h2 *= $r0-$r1 +$code.=<<___; + mulq $h0 # h0*r1 + mov %rax,$d2 + mov $r0,%rax + mov %rdx,$d3 + + mulq $h0 # h0*r0 + mov %rax,$h0 # future $h0 + mov $r0,%rax + mov %rdx,$d1 + + mulq $h1 # h1*r0 + add %rax,$d2 + mov $s1,%rax + adc %rdx,$d3 + + mulq $h1 # h1*s1 + mov $h2,$h1 # borrow $h1 + add %rax,$h0 + adc %rdx,$d1 + + imulq $s1,$h1 # h2*s1 + add $h1,$d2 + mov $d1,$h1 + adc \$0,$d3 + + imulq $r0,$h2 # h2*r0 + add $d2,$h1 + mov \$-4,%rax # mask value + adc $h2,$d3 + + and $d3,%rax # last reduction step + mov $d3,$h2 + shr \$2,$d3 + and \$3,$h2 + add $d3,%rax + add %rax,$h0 + adc \$0,$h1 + adc \$0,$h2 +___ +} + +######################################################################## +# Layout of opaque area is following. +# +# unsigned __int64 h[3]; # current hash value base 2^64 +# unsigned __int64 r[2]; # key value base 2^64 + +if ($flavour =~ /kernel/) { +$code.=<<___ if ($avx); +.globl poly1305_blocks_avx +___ +$code.=<<___ if ($avx>1); +.globl poly1305_blocks_avx2 +___ +$code.=<<___ if ($avx>3); +.globl poly1305_init_base2_44 +.globl poly1305_blocks_base2_44 +.globl poly1305_emit_base2_44 +.globl poly1305_blocks_vpmadd52 +___ +} +$code.=<<___; +.text + +.extern OPENSSL_ia32cap_P + +.globl poly1305_init +.hidden poly1305_init +.globl poly1305_blocks +.hidden poly1305_blocks +.globl poly1305_emit +.hidden poly1305_emit + +.type poly1305_init,\@function,3 +.align 32 +poly1305_init: + xor %rax,%rax + mov %rax,0($ctx) # initialize hash value + mov %rax,8($ctx) + mov %rax,16($ctx) # [along with is_base2_26] + + cmp \$0,$inp + je .Lno_key + + mov \$0x0ffffffc0fffffff,%rax + lea -3(%rax),%rcx # $0x0ffffffc0ffffffc + and 0($inp),%rax + and 8($inp),%rcx + mov %rax,24($ctx) + mov %rcx,32($ctx) +___ +$code.=<<___ if ($avx); + movl \$-1,48($ctx) # write impossible value +___ + if ($flavour !~ /kernel/) { +$code.=<<___; + lea poly1305_blocks(%rip),%r10 + lea poly1305_emit(%rip),%r11 +___ +$code.=<<___ if ($avx); + mov OPENSSL_ia32cap_P+4(%rip),%r9 + lea poly1305_blocks_avx(%rip),%rax + bt \$`60-32`,%r9 # AVX? + cmovc %rax,%r10 +___ +$code.=<<___ if ($avx>1); + lea poly1305_blocks_avx2(%rip),%rax + bt \$`5+32`,%r9 # AVX2? + cmovc %rax,%r10 +___ +$code.=<<___ if ($avx>3); + mov \$`(1<<31|1<<21)`,%rax # AVX512VL|AVX512IFMA + shr \$32,%r9 + and %rax,%r9 + cmp %rax,%r9 + je .Linit_base2_44 +___ +$code.=<<___ if ($flavour !~ /elf32/); + mov %r10,0(%rdx) + mov %r11,8(%rdx) +___ +$code.=<<___ if ($flavour =~ /elf32/); + mov %r10d,0(%rdx) + mov %r11d,4(%rdx) +___ + } +$code.=<<___; + mov \$1,%eax +.Lno_key: + ret +.size poly1305_init,.-poly1305_init + +.type poly1305_blocks,\@function,4 +.align 32 +poly1305_blocks: +.cfi_startproc +.Lblocks: + shr \$4,$len + jz .Lno_data # too short + + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + lea -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lblocks_body: + + mov $len,%r15 # reassign $len + + mov 24($ctx),$r0 # load r + mov 32($ctx),$s1 + + mov 0($ctx),$h0 # load hash value base 2^64 + mov 8($ctx),$h1 + mov 16($ctx),$h2 # [along with is_base2_26] + + mov $h0#d,%eax # load hash value base 2^26 + mov 4($ctx),%edx + mov $h1#d,%r8d + mov 12($ctx),%r10d + mov $h2#d,%r12d + + shl \$26,%rdx # base 2^26 -> base 2^64 + mov %r8,%r9 + shl \$52,%r8 + add %rdx,%rax + shr \$12,%r9 + add %rax,%r8 # h0 + adc \$0,%r9 + + shl \$14,%r10 + mov %r12,%rax + shr \$24,%r12 + add %r10,%r9 + shl \$40,%rax + add %rax,%r9 # h1 + adc \$0,%r12 # h2 + + cmp \$4,$h2 # is_base2_26? [4 is as good as 2^32-1] + + cmova %r8,$h0 # choose between radixes + cmova %r9,$h1 + cmova %r12,$h2 + + mov $s1,$r1 + shr \$2,$s1 + mov $r1,%rax + add $r1,$s1 # s1 = r1 + (r1 >> 2) + jmp .Loop + +.align 32 +.Loop: + add 0($inp),$h0 # accumulate input + adc 8($inp),$h1 + lea 16($inp),$inp + adc $padbit,$h2 +___ + &poly1305_iteration(); +$code.=<<___; + mov $r1,%rax + dec %r15 # len-=16 + jnz .Loop + + mov $h0,0($ctx) # store hash value + mov $h1,8($ctx) + mov $h2,16($ctx) + + mov 8(%rsp),%r15 +.cfi_restore %r15 + mov 16(%rsp),%r14 +.cfi_restore %r14 + mov 24(%rsp),%r13 +.cfi_restore %r13 + mov 32(%rsp),%r12 +.cfi_restore %r12 + mov 40(%rsp),%rbp +.cfi_restore %rbp + mov 48(%rsp),%rbx +.cfi_restore %rbx + lea 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lno_data: +.Lblocks_epilogue: + ret +.cfi_endproc +.size poly1305_blocks,.-poly1305_blocks + +.type poly1305_emit,\@function,3 +.align 32 +poly1305_emit: + mov 0($ctx),%eax # load hash value base 2^26 + mov 4($ctx),%ecx + mov 8($ctx),%r8d + mov 12($ctx),%r11d + mov 16($ctx),%r10d + + shl \$26,%rcx # base 2^26 -> base 2^64 + mov %r8,%r9 + shl \$52,%r8 + add %rcx,%rax + shr \$12,%r9 + add %rax,%r8 # h0 + adc \$0,%r9 + + shl \$14,%r11 + mov %r10,%rax + shr \$24,%r10 + add %r11,%r9 + mov 0($ctx),%rcx # load hash value base 2^64 + shl \$40,%rax + mov 8($ctx),%r11 + add %rax,%r9 # h1 + mov 16($ctx),%rax # [along with is_base2_26] + adc \$0,%r10 # h2 + + cmp \$4,%rax # is_base2_26? [4 is as good as 2^32-1] + + cmovbe %rcx,%r8 # choose between radixes + cmovbe %r11,%r9 + cmovbe %rax,%r10 + + mov %r8,%rax + add \$5,%r8 # compare to modulus + mov %r9,%rcx + adc \$0,%r9 + adc \$0,%r10 + shr \$2,%r10 # did 130-bit value overflow? + cmovnz %r8,%rax + cmovnz %r9,%rcx + + add 0($nonce),%rax # accumulate nonce + adc 8($nonce),%rcx + mov %rax,0($mac) # write result + mov %rcx,8($mac) + + ret +.size poly1305_emit,.-poly1305_emit +___ +if ($avx) { + +######################################################################## +# Layout of opaque area is following. +# +# unsigned __int32 h[5]; # current hash value base 2^26 +# unsigned __int32 is_base2_26; +# unsigned __int64 r[2]; # key value base 2^64 +# unsigned __int64 pad; +# struct { unsigned __int32 r^2, r^1, r^4, r^3; } r[9]; +# +# where r^n are base 2^26 digits of degrees of multiplier key. There are +# 5 digits, but last four are interleaved with multiples of 5, totalling +# in 9 elements: r0, r1, 5*r1, r2, 5*r2, r3, 5*r3, r4, 5*r4. + +my ($H0,$H1,$H2,$H3,$H4, $T0,$T1,$T2,$T3,$T4, $D0,$D1,$D2,$D3,$D4, $MASK) = + map("%xmm$_",(0..15)); + +$code.=<<___; +.type __poly1305_block,\@abi-omnipotent +.align 32 +__poly1305_block: +___ + &poly1305_iteration(); +$code.=<<___; + ret +.size __poly1305_block,.-__poly1305_block + +.type __poly1305_init_avx,\@abi-omnipotent +.align 32 +__poly1305_init_avx: + cmpl \$-1,48($ctx) + jne .Ldone_init_avx + + mov $r0,$h0 + mov $r1,$h1 + xor $h2,$h2 + + lea 48+64($ctx),$ctx # size optimization + + mov $r1,%rax + call __poly1305_block # r^2 + + mov \$0x3ffffff,%eax # save interleaved r^2 and r base 2^26 + mov \$0x3ffffff,%edx + mov $h0,$d1 + and $h0#d,%eax + mov $r0,$d2 + and $r0#d,%edx + mov %eax,`16*0+0-64`($ctx) + shr \$26,$d1 + mov %edx,`16*0+4-64`($ctx) + shr \$26,$d2 + + mov \$0x3ffffff,%eax + mov \$0x3ffffff,%edx + and $d1#d,%eax + and $d2#d,%edx + mov %eax,`16*1+0-64`($ctx) + lea (%rax,%rax,4),%eax # *5 + mov %edx,`16*1+4-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + mov %eax,`16*2+0-64`($ctx) + shr \$26,$d1 + mov %edx,`16*2+4-64`($ctx) + shr \$26,$d2 + + mov $h1,%rax + mov $r1,%rdx + shl \$12,%rax + shl \$12,%rdx + or $d1,%rax + or $d2,%rdx + and \$0x3ffffff,%eax + and \$0x3ffffff,%edx + mov %eax,`16*3+0-64`($ctx) + lea (%rax,%rax,4),%eax # *5 + mov %edx,`16*3+4-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + mov %eax,`16*4+0-64`($ctx) + mov $h1,$d1 + mov %edx,`16*4+4-64`($ctx) + mov $r1,$d2 + + mov \$0x3ffffff,%eax + mov \$0x3ffffff,%edx + shr \$14,$d1 + shr \$14,$d2 + and $d1#d,%eax + and $d2#d,%edx + mov %eax,`16*5+0-64`($ctx) + lea (%rax,%rax,4),%eax # *5 + mov %edx,`16*5+4-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + mov %eax,`16*6+0-64`($ctx) + shr \$26,$d1 + mov %edx,`16*6+4-64`($ctx) + shr \$26,$d2 + + mov $h2,%rax + shl \$24,%rax + or %rax,$d1 + mov $d1#d,`16*7+0-64`($ctx) + lea ($d1,$d1,4),$d1 # *5 + mov $d2#d,`16*7+4-64`($ctx) + lea ($d2,$d2,4),$d2 # *5 + mov $d1#d,`16*8+0-64`($ctx) + mov $d2#d,`16*8+4-64`($ctx) + + mov $r1,%rax + call __poly1305_block # r^3 + + mov \$0x3ffffff,%eax # save r^3 base 2^26 + mov $h0,$d1 + and $h0#d,%eax + shr \$26,$d1 + mov %eax,`16*0+12-64`($ctx) + + mov \$0x3ffffff,%edx + and $d1#d,%edx + mov %edx,`16*1+12-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + shr \$26,$d1 + mov %edx,`16*2+12-64`($ctx) + + mov $h1,%rax + shl \$12,%rax + or $d1,%rax + and \$0x3ffffff,%eax + mov %eax,`16*3+12-64`($ctx) + lea (%rax,%rax,4),%eax # *5 + mov $h1,$d1 + mov %eax,`16*4+12-64`($ctx) + + mov \$0x3ffffff,%edx + shr \$14,$d1 + and $d1#d,%edx + mov %edx,`16*5+12-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + shr \$26,$d1 + mov %edx,`16*6+12-64`($ctx) + + mov $h2,%rax + shl \$24,%rax + or %rax,$d1 + mov $d1#d,`16*7+12-64`($ctx) + lea ($d1,$d1,4),$d1 # *5 + mov $d1#d,`16*8+12-64`($ctx) + + mov $r1,%rax + call __poly1305_block # r^4 + + mov \$0x3ffffff,%eax # save r^4 base 2^26 + mov $h0,$d1 + and $h0#d,%eax + shr \$26,$d1 + mov %eax,`16*0+8-64`($ctx) + + mov \$0x3ffffff,%edx + and $d1#d,%edx + mov %edx,`16*1+8-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + shr \$26,$d1 + mov %edx,`16*2+8-64`($ctx) + + mov $h1,%rax + shl \$12,%rax + or $d1,%rax + and \$0x3ffffff,%eax + mov %eax,`16*3+8-64`($ctx) + lea (%rax,%rax,4),%eax # *5 + mov $h1,$d1 + mov %eax,`16*4+8-64`($ctx) + + mov \$0x3ffffff,%edx + shr \$14,$d1 + and $d1#d,%edx + mov %edx,`16*5+8-64`($ctx) + lea (%rdx,%rdx,4),%edx # *5 + shr \$26,$d1 + mov %edx,`16*6+8-64`($ctx) + + mov $h2,%rax + shl \$24,%rax + or %rax,$d1 + mov $d1#d,`16*7+8-64`($ctx) + lea ($d1,$d1,4),$d1 # *5 + mov $d1#d,`16*8+8-64`($ctx) + + lea -48-64($ctx),$ctx # size [de-]optimization +.Ldone_init_avx: + ret +.size __poly1305_init_avx,.-__poly1305_init_avx + +.type poly1305_blocks_avx,\@function,4 +.align 32 +poly1305_blocks_avx: +.cfi_startproc + mov 20($ctx),%r8d # load is_base2_26 + cmp \$128,$len + jb .Lblocks + + and \$-16,$len + + vzeroupper + + test %r8d,%r8d # is_base2_26? + jz .Lbase2_64_avx + + test \$31,$len + jz .Leven_avx + + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + lea -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lblocks_avx_body: + + mov $len,%r15 # reassign $len + + mov 0($ctx),$d1 # load hash value + mov 8($ctx),$d2 + mov 16($ctx),$h2#d + + mov 24($ctx),$r0 # load r + mov 32($ctx),$s1 + + ################################# base 2^26 -> base 2^64 + mov $d1#d,$h0#d + and \$`-1*(1<<31)`,$d1 + mov $d2,$r1 # borrow $r1 + mov $d2#d,$h1#d + and \$`-1*(1<<31)`,$d2 + + shr \$6,$d1 + shl \$52,$r1 + add $d1,$h0 + shr \$12,$h1 + shr \$18,$d2 + add $r1,$h0 + adc $d2,$h1 + + mov $h2,$d1 + shl \$40,$d1 + shr \$24,$h2 + add $d1,$h1 + adc \$0,$h2 # can be partially reduced... + + mov $s1,$r1 + mov $s1,%rax + shr \$2,$s1 + add $r1,$s1 # s1 = r1 + (r1 >> 2) + + add 0($inp),$h0 # accumulate input + adc 8($inp),$h1 + lea 16($inp),$inp + adc $padbit,$h2 + + call __poly1305_block + + ################################# base 2^64 -> base 2^26 + mov $h0,%rax + mov $h0,%rdx + shr \$52,$h0 + mov $h1,$r0 + mov $h1,$r1 + shr \$26,%rdx + and \$0x3ffffff,%rax # h[0] + shl \$12,$r0 + and \$0x3ffffff,%rdx # h[1] + shr \$14,$h1 + or $r0,$h0 + shl \$24,$h2 + and \$0x3ffffff,$h0 # h[2] + shr \$40,$r1 + and \$0x3ffffff,$h1 # h[3] + or $r1,$h2 # h[4] + + vmovd %rax#d,$H0 + vmovd %rdx#d,$H1 + vmovd $h0#d,$H2 + vmovd $h1#d,$H3 + vmovd $h2#d,$H4 + + lea -16(%r15),$len + + mov 8(%rsp),%r15 +.cfi_restore %r15 + mov 16(%rsp),%r14 +.cfi_restore %r14 + mov 24(%rsp),%r13 +.cfi_restore %r13 + mov 32(%rsp),%r12 +.cfi_restore %r12 + mov 40(%rsp),%rbp +.cfi_restore %rbp + mov 48(%rsp),%rbx +.cfi_restore %rbx + lea 56(%rsp),%rax # for win64 + lea 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lblocks_avx_epilogue: + jmp .Ldo_avx +.cfi_endproc + +.align 32 +.Lbase2_64_avx: +.cfi_startproc + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + lea -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lbase2_64_avx_body: + + mov $len,%r15 # reassign $len + + mov 24($ctx),$r0 # load r + mov 32($ctx),$s1 + + mov 0($ctx),$h0 # load hash value + mov 8($ctx),$h1 + mov 16($ctx),$h2#d + + mov $s1,$r1 + mov $s1,%rax + shr \$2,$s1 + add $r1,$s1 # s1 = r1 + (r1 >> 2) + + test \$31,$len + jz .Linit_avx + + add 0($inp),$h0 # accumulate input + adc 8($inp),$h1 + lea 16($inp),$inp + adc $padbit,$h2 + sub \$16,%r15 + + call __poly1305_block + +.Linit_avx: + ################################# base 2^64 -> base 2^26 + mov $h0,%rax + mov $h0,%rdx + shr \$52,$h0 + mov $h1,$d1 + mov $h1,$d2 + shr \$26,%rdx + and \$0x3ffffff,%rax # h[0] + shl \$12,$d1 + and \$0x3ffffff,%rdx # h[1] + shr \$14,$h1 + or $d1,$h0 + shl \$24,$h2 + and \$0x3ffffff,$h0 # h[2] + shr \$40,$d2 + and \$0x3ffffff,$h1 # h[3] + or $d2,$h2 # h[4] + + vmovd %rax#d,$H0 + vmovd %rdx#d,$H1 + vmovd $h0#d,$H2 + vmovd $h1#d,$H3 + vmovd $h2#d,$H4 + movl \$1,20($ctx) # set is_base2_26 + + call __poly1305_init_avx + + mov %r15,$len + + mov 8(%rsp),%r15 +.cfi_restore %r15 + mov 16(%rsp),%r14 +.cfi_restore %r14 + mov 24(%rsp),%r13 +.cfi_restore %r13 + mov 32(%rsp),%r12 +.cfi_restore %r12 + mov 40(%rsp),%rbp +.cfi_restore %rbp + mov 48(%rsp),%rbx +.cfi_restore %rbx + lea 56(%rsp),%rax # for win64 + lea 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lbase2_64_avx_epilogue: + jmp .Ldo_avx +.cfi_endproc + +.align 32 +.Leven_avx: +.cfi_startproc + vmovd 4*0($ctx),$H0 # load hash value + vmovd 4*1($ctx),$H1 + vmovd 4*2($ctx),$H2 + vmovd 4*3($ctx),$H3 + vmovd 4*4($ctx),$H4 + +.Ldo_avx: +___ +$code.=<<___ if (!$win64); + lea -0x58(%rsp),%r11 +.cfi_def_cfa %r11,0x60 + sub \$0x178,%rsp +___ +$code.=<<___ if ($win64); + lea -0xf8(%rsp),%r11 + sub \$0x218,%rsp + vmovdqa %xmm6,0x50(%r11) + vmovdqa %xmm7,0x60(%r11) + vmovdqa %xmm8,0x70(%r11) + vmovdqa %xmm9,0x80(%r11) + vmovdqa %xmm10,0x90(%r11) + vmovdqa %xmm11,0xa0(%r11) + vmovdqa %xmm12,0xb0(%r11) + vmovdqa %xmm13,0xc0(%r11) + vmovdqa %xmm14,0xd0(%r11) + vmovdqa %xmm15,0xe0(%r11) +.Ldo_avx_body: +___ +$code.=<<___; + sub \$64,$len + lea -32($inp),%rax + cmovc %rax,$inp + + vmovdqu `16*3`($ctx),$D4 # preload r0^2 + lea `16*3+64`($ctx),$ctx # size optimization + lea .Lconst(%rip),%rcx + + ################################################################ + # load input + vmovdqu 16*2($inp),$T0 + vmovdqu 16*3($inp),$T1 + vmovdqa 64(%rcx),$MASK # .Lmask26 + + vpsrldq \$6,$T0,$T2 # splat input + vpsrldq \$6,$T1,$T3 + vpunpckhqdq $T1,$T0,$T4 # 4 + vpunpcklqdq $T1,$T0,$T0 # 0:1 + vpunpcklqdq $T3,$T2,$T3 # 2:3 + + vpsrlq \$40,$T4,$T4 # 4 + vpsrlq \$26,$T0,$T1 + vpand $MASK,$T0,$T0 # 0 + vpsrlq \$4,$T3,$T2 + vpand $MASK,$T1,$T1 # 1 + vpsrlq \$30,$T3,$T3 + vpand $MASK,$T2,$T2 # 2 + vpand $MASK,$T3,$T3 # 3 + vpor 32(%rcx),$T4,$T4 # padbit, yes, always + + jbe .Lskip_loop_avx + + # expand and copy pre-calculated table to stack + vmovdqu `16*1-64`($ctx),$D1 + vmovdqu `16*2-64`($ctx),$D2 + vpshufd \$0xEE,$D4,$D3 # 34xx -> 3434 + vpshufd \$0x44,$D4,$D0 # xx12 -> 1212 + vmovdqa $D3,-0x90(%r11) + vmovdqa $D0,0x00(%rsp) + vpshufd \$0xEE,$D1,$D4 + vmovdqu `16*3-64`($ctx),$D0 + vpshufd \$0x44,$D1,$D1 + vmovdqa $D4,-0x80(%r11) + vmovdqa $D1,0x10(%rsp) + vpshufd \$0xEE,$D2,$D3 + vmovdqu `16*4-64`($ctx),$D1 + vpshufd \$0x44,$D2,$D2 + vmovdqa $D3,-0x70(%r11) + vmovdqa $D2,0x20(%rsp) + vpshufd \$0xEE,$D0,$D4 + vmovdqu `16*5-64`($ctx),$D2 + vpshufd \$0x44,$D0,$D0 + vmovdqa $D4,-0x60(%r11) + vmovdqa $D0,0x30(%rsp) + vpshufd \$0xEE,$D1,$D3 + vmovdqu `16*6-64`($ctx),$D0 + vpshufd \$0x44,$D1,$D1 + vmovdqa $D3,-0x50(%r11) + vmovdqa $D1,0x40(%rsp) + vpshufd \$0xEE,$D2,$D4 + vmovdqu `16*7-64`($ctx),$D1 + vpshufd \$0x44,$D2,$D2 + vmovdqa $D4,-0x40(%r11) + vmovdqa $D2,0x50(%rsp) + vpshufd \$0xEE,$D0,$D3 + vmovdqu `16*8-64`($ctx),$D2 + vpshufd \$0x44,$D0,$D0 + vmovdqa $D3,-0x30(%r11) + vmovdqa $D0,0x60(%rsp) + vpshufd \$0xEE,$D1,$D4 + vpshufd \$0x44,$D1,$D1 + vmovdqa $D4,-0x20(%r11) + vmovdqa $D1,0x70(%rsp) + vpshufd \$0xEE,$D2,$D3 + vmovdqa 0x00(%rsp),$D4 # preload r0^2 + vpshufd \$0x44,$D2,$D2 + vmovdqa $D3,-0x10(%r11) + vmovdqa $D2,0x80(%rsp) + + jmp .Loop_avx + +.align 32 +.Loop_avx: + ################################################################ + # ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2 + # ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^3+inp[7]*r + # \___________________/ + # ((inp[0]*r^4+inp[2]*r^2+inp[4])*r^4+inp[6]*r^2+inp[8])*r^2 + # ((inp[1]*r^4+inp[3]*r^2+inp[5])*r^4+inp[7]*r^2+inp[9])*r + # \___________________/ \____________________/ + # + # Note that we start with inp[2:3]*r^2. This is because it + # doesn't depend on reduction in previous iteration. + ################################################################ + # d4 = h4*r0 + h3*r1 + h2*r2 + h1*r3 + h0*r4 + # d3 = h3*r0 + h2*r1 + h1*r2 + h0*r3 + h4*5*r4 + # d2 = h2*r0 + h1*r1 + h0*r2 + h4*5*r3 + h3*5*r4 + # d1 = h1*r0 + h0*r1 + h4*5*r2 + h3*5*r3 + h2*5*r4 + # d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4 + # + # though note that $Tx and $Hx are "reversed" in this section, + # and $D4 is preloaded with r0^2... + + vpmuludq $T0,$D4,$D0 # d0 = h0*r0 + vpmuludq $T1,$D4,$D1 # d1 = h1*r0 + vmovdqa $H2,0x20(%r11) # offload hash + vpmuludq $T2,$D4,$D2 # d3 = h2*r0 + vmovdqa 0x10(%rsp),$H2 # r1^2 + vpmuludq $T3,$D4,$D3 # d3 = h3*r0 + vpmuludq $T4,$D4,$D4 # d4 = h4*r0 + + vmovdqa $H0,0x00(%r11) # + vpmuludq 0x20(%rsp),$T4,$H0 # h4*s1 + vmovdqa $H1,0x10(%r11) # + vpmuludq $T3,$H2,$H1 # h3*r1 + vpaddq $H0,$D0,$D0 # d0 += h4*s1 + vpaddq $H1,$D4,$D4 # d4 += h3*r1 + vmovdqa $H3,0x30(%r11) # + vpmuludq $T2,$H2,$H0 # h2*r1 + vpmuludq $T1,$H2,$H1 # h1*r1 + vpaddq $H0,$D3,$D3 # d3 += h2*r1 + vmovdqa 0x30(%rsp),$H3 # r2^2 + vpaddq $H1,$D2,$D2 # d2 += h1*r1 + vmovdqa $H4,0x40(%r11) # + vpmuludq $T0,$H2,$H2 # h0*r1 + vpmuludq $T2,$H3,$H0 # h2*r2 + vpaddq $H2,$D1,$D1 # d1 += h0*r1 + + vmovdqa 0x40(%rsp),$H4 # s2^2 + vpaddq $H0,$D4,$D4 # d4 += h2*r2 + vpmuludq $T1,$H3,$H1 # h1*r2 + vpmuludq $T0,$H3,$H3 # h0*r2 + vpaddq $H1,$D3,$D3 # d3 += h1*r2 + vmovdqa 0x50(%rsp),$H2 # r3^2 + vpaddq $H3,$D2,$D2 # d2 += h0*r2 + vpmuludq $T4,$H4,$H0 # h4*s2 + vpmuludq $T3,$H4,$H4 # h3*s2 + vpaddq $H0,$D1,$D1 # d1 += h4*s2 + vmovdqa 0x60(%rsp),$H3 # s3^2 + vpaddq $H4,$D0,$D0 # d0 += h3*s2 + + vmovdqa 0x80(%rsp),$H4 # s4^2 + vpmuludq $T1,$H2,$H1 # h1*r3 + vpmuludq $T0,$H2,$H2 # h0*r3 + vpaddq $H1,$D4,$D4 # d4 += h1*r3 + vpaddq $H2,$D3,$D3 # d3 += h0*r3 + vpmuludq $T4,$H3,$H0 # h4*s3 + vpmuludq $T3,$H3,$H1 # h3*s3 + vpaddq $H0,$D2,$D2 # d2 += h4*s3 + vmovdqu 16*0($inp),$H0 # load input + vpaddq $H1,$D1,$D1 # d1 += h3*s3 + vpmuludq $T2,$H3,$H3 # h2*s3 + vpmuludq $T2,$H4,$T2 # h2*s4 + vpaddq $H3,$D0,$D0 # d0 += h2*s3 + + vmovdqu 16*1($inp),$H1 # + vpaddq $T2,$D1,$D1 # d1 += h2*s4 + vpmuludq $T3,$H4,$T3 # h3*s4 + vpmuludq $T4,$H4,$T4 # h4*s4 + vpsrldq \$6,$H0,$H2 # splat input + vpaddq $T3,$D2,$D2 # d2 += h3*s4 + vpaddq $T4,$D3,$D3 # d3 += h4*s4 + vpsrldq \$6,$H1,$H3 # + vpmuludq 0x70(%rsp),$T0,$T4 # h0*r4 + vpmuludq $T1,$H4,$T0 # h1*s4 + vpunpckhqdq $H1,$H0,$H4 # 4 + vpaddq $T4,$D4,$D4 # d4 += h0*r4 + vmovdqa -0x90(%r11),$T4 # r0^4 + vpaddq $T0,$D0,$D0 # d0 += h1*s4 + + vpunpcklqdq $H1,$H0,$H0 # 0:1 + vpunpcklqdq $H3,$H2,$H3 # 2:3 + + #vpsrlq \$40,$H4,$H4 # 4 + vpsrldq \$`40/8`,$H4,$H4 # 4 + vpsrlq \$26,$H0,$H1 + vpand $MASK,$H0,$H0 # 0 + vpsrlq \$4,$H3,$H2 + vpand $MASK,$H1,$H1 # 1 + vpand 0(%rcx),$H4,$H4 # .Lmask24 + vpsrlq \$30,$H3,$H3 + vpand $MASK,$H2,$H2 # 2 + vpand $MASK,$H3,$H3 # 3 + vpor 32(%rcx),$H4,$H4 # padbit, yes, always + + vpaddq 0x00(%r11),$H0,$H0 # add hash value + vpaddq 0x10(%r11),$H1,$H1 + vpaddq 0x20(%r11),$H2,$H2 + vpaddq 0x30(%r11),$H3,$H3 + vpaddq 0x40(%r11),$H4,$H4 + + lea 16*2($inp),%rax + lea 16*4($inp),$inp + sub \$64,$len + cmovc %rax,$inp + + ################################################################ + # Now we accumulate (inp[0:1]+hash)*r^4 + ################################################################ + # d4 = h4*r0 + h3*r1 + h2*r2 + h1*r3 + h0*r4 + # d3 = h3*r0 + h2*r1 + h1*r2 + h0*r3 + h4*5*r4 + # d2 = h2*r0 + h1*r1 + h0*r2 + h4*5*r3 + h3*5*r4 + # d1 = h1*r0 + h0*r1 + h4*5*r2 + h3*5*r3 + h2*5*r4 + # d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4 + + vpmuludq $H0,$T4,$T0 # h0*r0 + vpmuludq $H1,$T4,$T1 # h1*r0 + vpaddq $T0,$D0,$D0 + vpaddq $T1,$D1,$D1 + vmovdqa -0x80(%r11),$T2 # r1^4 + vpmuludq $H2,$T4,$T0 # h2*r0 + vpmuludq $H3,$T4,$T1 # h3*r0 + vpaddq $T0,$D2,$D2 + vpaddq $T1,$D3,$D3 + vpmuludq $H4,$T4,$T4 # h4*r0 + vpmuludq -0x70(%r11),$H4,$T0 # h4*s1 + vpaddq $T4,$D4,$D4 + + vpaddq $T0,$D0,$D0 # d0 += h4*s1 + vpmuludq $H2,$T2,$T1 # h2*r1 + vpmuludq $H3,$T2,$T0 # h3*r1 + vpaddq $T1,$D3,$D3 # d3 += h2*r1 + vmovdqa -0x60(%r11),$T3 # r2^4 + vpaddq $T0,$D4,$D4 # d4 += h3*r1 + vpmuludq $H1,$T2,$T1 # h1*r1 + vpmuludq $H0,$T2,$T2 # h0*r1 + vpaddq $T1,$D2,$D2 # d2 += h1*r1 + vpaddq $T2,$D1,$D1 # d1 += h0*r1 + + vmovdqa -0x50(%r11),$T4 # s2^4 + vpmuludq $H2,$T3,$T0 # h2*r2 + vpmuludq $H1,$T3,$T1 # h1*r2 + vpaddq $T0,$D4,$D4 # d4 += h2*r2 + vpaddq $T1,$D3,$D3 # d3 += h1*r2 + vmovdqa -0x40(%r11),$T2 # r3^4 + vpmuludq $H0,$T3,$T3 # h0*r2 + vpmuludq $H4,$T4,$T0 # h4*s2 + vpaddq $T3,$D2,$D2 # d2 += h0*r2 + vpaddq $T0,$D1,$D1 # d1 += h4*s2 + vmovdqa -0x30(%r11),$T3 # s3^4 + vpmuludq $H3,$T4,$T4 # h3*s2 + vpmuludq $H1,$T2,$T1 # h1*r3 + vpaddq $T4,$D0,$D0 # d0 += h3*s2 + + vmovdqa -0x10(%r11),$T4 # s4^4 + vpaddq $T1,$D4,$D4 # d4 += h1*r3 + vpmuludq $H0,$T2,$T2 # h0*r3 + vpmuludq $H4,$T3,$T0 # h4*s3 + vpaddq $T2,$D3,$D3 # d3 += h0*r3 + vpaddq $T0,$D2,$D2 # d2 += h4*s3 + vmovdqu 16*2($inp),$T0 # load input + vpmuludq $H3,$T3,$T2 # h3*s3 + vpmuludq $H2,$T3,$T3 # h2*s3 + vpaddq $T2,$D1,$D1 # d1 += h3*s3 + vmovdqu 16*3($inp),$T1 # + vpaddq $T3,$D0,$D0 # d0 += h2*s3 + + vpmuludq $H2,$T4,$H2 # h2*s4 + vpmuludq $H3,$T4,$H3 # h3*s4 + vpsrldq \$6,$T0,$T2 # splat input + vpaddq $H2,$D1,$D1 # d1 += h2*s4 + vpmuludq $H4,$T4,$H4 # h4*s4 + vpsrldq \$6,$T1,$T3 # + vpaddq $H3,$D2,$H2 # h2 = d2 + h3*s4 + vpaddq $H4,$D3,$H3 # h3 = d3 + h4*s4 + vpmuludq -0x20(%r11),$H0,$H4 # h0*r4 + vpmuludq $H1,$T4,$H0 + vpunpckhqdq $T1,$T0,$T4 # 4 + vpaddq $H4,$D4,$H4 # h4 = d4 + h0*r4 + vpaddq $H0,$D0,$H0 # h0 = d0 + h1*s4 + + vpunpcklqdq $T1,$T0,$T0 # 0:1 + vpunpcklqdq $T3,$T2,$T3 # 2:3 + + #vpsrlq \$40,$T4,$T4 # 4 + vpsrldq \$`40/8`,$T4,$T4 # 4 + vpsrlq \$26,$T0,$T1 + vmovdqa 0x00(%rsp),$D4 # preload r0^2 + vpand $MASK,$T0,$T0 # 0 + vpsrlq \$4,$T3,$T2 + vpand $MASK,$T1,$T1 # 1 + vpand 0(%rcx),$T4,$T4 # .Lmask24 + vpsrlq \$30,$T3,$T3 + vpand $MASK,$T2,$T2 # 2 + vpand $MASK,$T3,$T3 # 3 + vpor 32(%rcx),$T4,$T4 # padbit, yes, always + + ################################################################ + # lazy reduction as discussed in "NEON crypto" by D.J. Bernstein + # and P. Schwabe + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $D0,$D1,$H1 # h0 -> h1 + + vpsrlq \$26,$H4,$D0 + vpand $MASK,$H4,$H4 + + vpsrlq \$26,$H1,$D1 + vpand $MASK,$H1,$H1 + vpaddq $D1,$H2,$H2 # h1 -> h2 + + vpaddq $D0,$H0,$H0 + vpsllq \$2,$D0,$D0 + vpaddq $D0,$H0,$H0 # h4 -> h0 + + vpsrlq \$26,$H2,$D2 + vpand $MASK,$H2,$H2 + vpaddq $D2,$H3,$H3 # h2 -> h3 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + ja .Loop_avx + +.Lskip_loop_avx: + ################################################################ + # multiply (inp[0:1]+hash) or inp[2:3] by r^2:r^1 + + vpshufd \$0x10,$D4,$D4 # r0^n, xx12 -> x1x2 + add \$32,$len + jnz .Long_tail_avx + + vpaddq $H2,$T2,$T2 + vpaddq $H0,$T0,$T0 + vpaddq $H1,$T1,$T1 + vpaddq $H3,$T3,$T3 + vpaddq $H4,$T4,$T4 + +.Long_tail_avx: + vmovdqa $H2,0x20(%r11) + vmovdqa $H0,0x00(%r11) + vmovdqa $H1,0x10(%r11) + vmovdqa $H3,0x30(%r11) + vmovdqa $H4,0x40(%r11) + + # d4 = h4*r0 + h3*r1 + h2*r2 + h1*r3 + h0*r4 + # d3 = h3*r0 + h2*r1 + h1*r2 + h0*r3 + h4*5*r4 + # d2 = h2*r0 + h1*r1 + h0*r2 + h4*5*r3 + h3*5*r4 + # d1 = h1*r0 + h0*r1 + h4*5*r2 + h3*5*r3 + h2*5*r4 + # d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4 + + vpmuludq $T2,$D4,$D2 # d2 = h2*r0 + vpmuludq $T0,$D4,$D0 # d0 = h0*r0 + vpshufd \$0x10,`16*1-64`($ctx),$H2 # r1^n + vpmuludq $T1,$D4,$D1 # d1 = h1*r0 + vpmuludq $T3,$D4,$D3 # d3 = h3*r0 + vpmuludq $T4,$D4,$D4 # d4 = h4*r0 + + vpmuludq $T3,$H2,$H0 # h3*r1 + vpaddq $H0,$D4,$D4 # d4 += h3*r1 + vpshufd \$0x10,`16*2-64`($ctx),$H3 # s1^n + vpmuludq $T2,$H2,$H1 # h2*r1 + vpaddq $H1,$D3,$D3 # d3 += h2*r1 + vpshufd \$0x10,`16*3-64`($ctx),$H4 # r2^n + vpmuludq $T1,$H2,$H0 # h1*r1 + vpaddq $H0,$D2,$D2 # d2 += h1*r1 + vpmuludq $T0,$H2,$H2 # h0*r1 + vpaddq $H2,$D1,$D1 # d1 += h0*r1 + vpmuludq $T4,$H3,$H3 # h4*s1 + vpaddq $H3,$D0,$D0 # d0 += h4*s1 + + vpshufd \$0x10,`16*4-64`($ctx),$H2 # s2^n + vpmuludq $T2,$H4,$H1 # h2*r2 + vpaddq $H1,$D4,$D4 # d4 += h2*r2 + vpmuludq $T1,$H4,$H0 # h1*r2 + vpaddq $H0,$D3,$D3 # d3 += h1*r2 + vpshufd \$0x10,`16*5-64`($ctx),$H3 # r3^n + vpmuludq $T0,$H4,$H4 # h0*r2 + vpaddq $H4,$D2,$D2 # d2 += h0*r2 + vpmuludq $T4,$H2,$H1 # h4*s2 + vpaddq $H1,$D1,$D1 # d1 += h4*s2 + vpshufd \$0x10,`16*6-64`($ctx),$H4 # s3^n + vpmuludq $T3,$H2,$H2 # h3*s2 + vpaddq $H2,$D0,$D0 # d0 += h3*s2 + + vpmuludq $T1,$H3,$H0 # h1*r3 + vpaddq $H0,$D4,$D4 # d4 += h1*r3 + vpmuludq $T0,$H3,$H3 # h0*r3 + vpaddq $H3,$D3,$D3 # d3 += h0*r3 + vpshufd \$0x10,`16*7-64`($ctx),$H2 # r4^n + vpmuludq $T4,$H4,$H1 # h4*s3 + vpaddq $H1,$D2,$D2 # d2 += h4*s3 + vpshufd \$0x10,`16*8-64`($ctx),$H3 # s4^n + vpmuludq $T3,$H4,$H0 # h3*s3 + vpaddq $H0,$D1,$D1 # d1 += h3*s3 + vpmuludq $T2,$H4,$H4 # h2*s3 + vpaddq $H4,$D0,$D0 # d0 += h2*s3 + + vpmuludq $T0,$H2,$H2 # h0*r4 + vpaddq $H2,$D4,$D4 # h4 = d4 + h0*r4 + vpmuludq $T4,$H3,$H1 # h4*s4 + vpaddq $H1,$D3,$D3 # h3 = d3 + h4*s4 + vpmuludq $T3,$H3,$H0 # h3*s4 + vpaddq $H0,$D2,$D2 # h2 = d2 + h3*s4 + vpmuludq $T2,$H3,$H1 # h2*s4 + vpaddq $H1,$D1,$D1 # h1 = d1 + h2*s4 + vpmuludq $T1,$H3,$H3 # h1*s4 + vpaddq $H3,$D0,$D0 # h0 = d0 + h1*s4 + + jz .Lshort_tail_avx + + vmovdqu 16*0($inp),$H0 # load input + vmovdqu 16*1($inp),$H1 + + vpsrldq \$6,$H0,$H2 # splat input + vpsrldq \$6,$H1,$H3 + vpunpckhqdq $H1,$H0,$H4 # 4 + vpunpcklqdq $H1,$H0,$H0 # 0:1 + vpunpcklqdq $H3,$H2,$H3 # 2:3 + + vpsrlq \$40,$H4,$H4 # 4 + vpsrlq \$26,$H0,$H1 + vpand $MASK,$H0,$H0 # 0 + vpsrlq \$4,$H3,$H2 + vpand $MASK,$H1,$H1 # 1 + vpsrlq \$30,$H3,$H3 + vpand $MASK,$H2,$H2 # 2 + vpand $MASK,$H3,$H3 # 3 + vpor 32(%rcx),$H4,$H4 # padbit, yes, always + + vpshufd \$0x32,`16*0-64`($ctx),$T4 # r0^n, 34xx -> x3x4 + vpaddq 0x00(%r11),$H0,$H0 + vpaddq 0x10(%r11),$H1,$H1 + vpaddq 0x20(%r11),$H2,$H2 + vpaddq 0x30(%r11),$H3,$H3 + vpaddq 0x40(%r11),$H4,$H4 + + ################################################################ + # multiply (inp[0:1]+hash) by r^4:r^3 and accumulate + + vpmuludq $H0,$T4,$T0 # h0*r0 + vpaddq $T0,$D0,$D0 # d0 += h0*r0 + vpmuludq $H1,$T4,$T1 # h1*r0 + vpaddq $T1,$D1,$D1 # d1 += h1*r0 + vpmuludq $H2,$T4,$T0 # h2*r0 + vpaddq $T0,$D2,$D2 # d2 += h2*r0 + vpshufd \$0x32,`16*1-64`($ctx),$T2 # r1^n + vpmuludq $H3,$T4,$T1 # h3*r0 + vpaddq $T1,$D3,$D3 # d3 += h3*r0 + vpmuludq $H4,$T4,$T4 # h4*r0 + vpaddq $T4,$D4,$D4 # d4 += h4*r0 + + vpmuludq $H3,$T2,$T0 # h3*r1 + vpaddq $T0,$D4,$D4 # d4 += h3*r1 + vpshufd \$0x32,`16*2-64`($ctx),$T3 # s1 + vpmuludq $H2,$T2,$T1 # h2*r1 + vpaddq $T1,$D3,$D3 # d3 += h2*r1 + vpshufd \$0x32,`16*3-64`($ctx),$T4 # r2 + vpmuludq $H1,$T2,$T0 # h1*r1 + vpaddq $T0,$D2,$D2 # d2 += h1*r1 + vpmuludq $H0,$T2,$T2 # h0*r1 + vpaddq $T2,$D1,$D1 # d1 += h0*r1 + vpmuludq $H4,$T3,$T3 # h4*s1 + vpaddq $T3,$D0,$D0 # d0 += h4*s1 + + vpshufd \$0x32,`16*4-64`($ctx),$T2 # s2 + vpmuludq $H2,$T4,$T1 # h2*r2 + vpaddq $T1,$D4,$D4 # d4 += h2*r2 + vpmuludq $H1,$T4,$T0 # h1*r2 + vpaddq $T0,$D3,$D3 # d3 += h1*r2 + vpshufd \$0x32,`16*5-64`($ctx),$T3 # r3 + vpmuludq $H0,$T4,$T4 # h0*r2 + vpaddq $T4,$D2,$D2 # d2 += h0*r2 + vpmuludq $H4,$T2,$T1 # h4*s2 + vpaddq $T1,$D1,$D1 # d1 += h4*s2 + vpshufd \$0x32,`16*6-64`($ctx),$T4 # s3 + vpmuludq $H3,$T2,$T2 # h3*s2 + vpaddq $T2,$D0,$D0 # d0 += h3*s2 + + vpmuludq $H1,$T3,$T0 # h1*r3 + vpaddq $T0,$D4,$D4 # d4 += h1*r3 + vpmuludq $H0,$T3,$T3 # h0*r3 + vpaddq $T3,$D3,$D3 # d3 += h0*r3 + vpshufd \$0x32,`16*7-64`($ctx),$T2 # r4 + vpmuludq $H4,$T4,$T1 # h4*s3 + vpaddq $T1,$D2,$D2 # d2 += h4*s3 + vpshufd \$0x32,`16*8-64`($ctx),$T3 # s4 + vpmuludq $H3,$T4,$T0 # h3*s3 + vpaddq $T0,$D1,$D1 # d1 += h3*s3 + vpmuludq $H2,$T4,$T4 # h2*s3 + vpaddq $T4,$D0,$D0 # d0 += h2*s3 + + vpmuludq $H0,$T2,$T2 # h0*r4 + vpaddq $T2,$D4,$D4 # d4 += h0*r4 + vpmuludq $H4,$T3,$T1 # h4*s4 + vpaddq $T1,$D3,$D3 # d3 += h4*s4 + vpmuludq $H3,$T3,$T0 # h3*s4 + vpaddq $T0,$D2,$D2 # d2 += h3*s4 + vpmuludq $H2,$T3,$T1 # h2*s4 + vpaddq $T1,$D1,$D1 # d1 += h2*s4 + vpmuludq $H1,$T3,$T3 # h1*s4 + vpaddq $T3,$D0,$D0 # d0 += h1*s4 + +.Lshort_tail_avx: + ################################################################ + # horizontal addition + + vpsrldq \$8,$D4,$T4 + vpsrldq \$8,$D3,$T3 + vpsrldq \$8,$D1,$T1 + vpsrldq \$8,$D0,$T0 + vpsrldq \$8,$D2,$T2 + vpaddq $T3,$D3,$D3 + vpaddq $T4,$D4,$D4 + vpaddq $T0,$D0,$D0 + vpaddq $T1,$D1,$D1 + vpaddq $T2,$D2,$D2 + + ################################################################ + # lazy reduction + + vpsrlq \$26,$D3,$H3 + vpand $MASK,$D3,$D3 + vpaddq $H3,$D4,$D4 # h3 -> h4 + + vpsrlq \$26,$D0,$H0 + vpand $MASK,$D0,$D0 + vpaddq $H0,$D1,$D1 # h0 -> h1 + + vpsrlq \$26,$D4,$H4 + vpand $MASK,$D4,$D4 + + vpsrlq \$26,$D1,$H1 + vpand $MASK,$D1,$D1 + vpaddq $H1,$D2,$D2 # h1 -> h2 + + vpaddq $H4,$D0,$D0 + vpsllq \$2,$H4,$H4 + vpaddq $H4,$D0,$D0 # h4 -> h0 + + vpsrlq \$26,$D2,$H2 + vpand $MASK,$D2,$D2 + vpaddq $H2,$D3,$D3 # h2 -> h3 + + vpsrlq \$26,$D0,$H0 + vpand $MASK,$D0,$D0 + vpaddq $H0,$D1,$D1 # h0 -> h1 + + vpsrlq \$26,$D3,$H3 + vpand $MASK,$D3,$D3 + vpaddq $H3,$D4,$D4 # h3 -> h4 + + vmovd $D0,`4*0-48-64`($ctx) # save partially reduced + vmovd $D1,`4*1-48-64`($ctx) + vmovd $D2,`4*2-48-64`($ctx) + vmovd $D3,`4*3-48-64`($ctx) + vmovd $D4,`4*4-48-64`($ctx) +___ +$code.=<<___ if ($win64); + vmovdqa 0x50(%r11),%xmm6 + vmovdqa 0x60(%r11),%xmm7 + vmovdqa 0x70(%r11),%xmm8 + vmovdqa 0x80(%r11),%xmm9 + vmovdqa 0x90(%r11),%xmm10 + vmovdqa 0xa0(%r11),%xmm11 + vmovdqa 0xb0(%r11),%xmm12 + vmovdqa 0xc0(%r11),%xmm13 + vmovdqa 0xd0(%r11),%xmm14 + vmovdqa 0xe0(%r11),%xmm15 + lea 0xf8(%r11),%rsp +.Ldo_avx_epilogue: +___ +$code.=<<___ if (!$win64); + lea 0x58(%r11),%rsp +.cfi_def_cfa %rsp,8 +___ +$code.=<<___; + vzeroupper + ret +.cfi_endproc +.size poly1305_blocks_avx,.-poly1305_blocks_avx +___ + +if ($avx>1) { +my ($H0,$H1,$H2,$H3,$H4, $MASK, $T4,$T0,$T1,$T2,$T3, $D0,$D1,$D2,$D3,$D4) = + map("%ymm$_",(0..15)); +my $S4=$MASK; + +$code.=<<___; +.type poly1305_blocks_avx2,\@function,4 +.align 32 +poly1305_blocks_avx2: +.cfi_startproc + mov 20($ctx),%r8d # load is_base2_26 + cmp \$128,$len + jb .Lblocks + + and \$-16,$len + + vzeroupper + + test %r8d,%r8d # is_base2_26? + jz .Lbase2_64_avx2 + + test \$63,$len + jz .Leven_avx2 + + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + lea -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lblocks_avx2_body: + + mov $len,%r15 # reassign $len + + mov 0($ctx),$d1 # load hash value + mov 8($ctx),$d2 + mov 16($ctx),$h2#d + + mov 24($ctx),$r0 # load r + mov 32($ctx),$s1 + + ################################# base 2^26 -> base 2^64 + mov $d1#d,$h0#d + and \$`-1*(1<<31)`,$d1 + mov $d2,$r1 # borrow $r1 + mov $d2#d,$h1#d + and \$`-1*(1<<31)`,$d2 + + shr \$6,$d1 + shl \$52,$r1 + add $d1,$h0 + shr \$12,$h1 + shr \$18,$d2 + add $r1,$h0 + adc $d2,$h1 + + mov $h2,$d1 + shl \$40,$d1 + shr \$24,$h2 + add $d1,$h1 + adc \$0,$h2 # can be partially reduced... + + mov $s1,$r1 + mov $s1,%rax + shr \$2,$s1 + add $r1,$s1 # s1 = r1 + (r1 >> 2) + +.Lbase2_26_pre_avx2: + add 0($inp),$h0 # accumulate input + adc 8($inp),$h1 + lea 16($inp),$inp + adc $padbit,$h2 + sub \$16,%r15 + + call __poly1305_block + mov $r1,%rax + + test \$63,%r15 + jnz .Lbase2_26_pre_avx2 + + ################################# base 2^64 -> base 2^26 + mov $h0,%rax + mov $h0,%rdx + shr \$52,$h0 + mov $h1,$r0 + mov $h1,$r1 + shr \$26,%rdx + and \$0x3ffffff,%rax # h[0] + shl \$12,$r0 + and \$0x3ffffff,%rdx # h[1] + shr \$14,$h1 + or $r0,$h0 + shl \$24,$h2 + and \$0x3ffffff,$h0 # h[2] + shr \$40,$r1 + and \$0x3ffffff,$h1 # h[3] + or $r1,$h2 # h[4] + + vmovd %rax#d,%x#$H0 + vmovd %rdx#d,%x#$H1 + vmovd $h0#d,%x#$H2 + vmovd $h1#d,%x#$H3 + vmovd $h2#d,%x#$H4 + + mov %r15,$len # restore $len + + mov 8(%rsp),%r15 +.cfi_restore %r15 + mov 16(%rsp),%r14 +.cfi_restore %r14 + mov 24(%rsp),%r13 +.cfi_restore %r13 + mov 32(%rsp),%r12 +.cfi_restore %r12 + mov 40(%rsp),%rbp +.cfi_restore %rbp + mov 48(%rsp),%rbx +.cfi_restore %rbx + lea 56(%rsp),%rax # for win64 + lea 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lblocks_avx2_epilogue: + jmp .Ldo_avx2 +.cfi_endproc + +.align 32 +.Lbase2_64_avx2: +.cfi_startproc + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + lea -8(%rsp),%rsp +.cfi_adjust_cfa_offset 8 +.Lbase2_64_avx2_body: + + mov $len,%r15 # reassign $len + + mov 24($ctx),$r0 # load r + mov 32($ctx),$s1 + + mov 0($ctx),$h0 # load hash value + mov 8($ctx),$h1 + mov 16($ctx),$h2#d + + mov $s1,$r1 + mov $s1,%rax + shr \$2,$s1 + add $r1,$s1 # s1 = r1 + (r1 >> 2) + + test \$63,$len + jz .Linit_avx2 + +.Lbase2_64_pre_avx2: + add 0($inp),$h0 # accumulate input + adc 8($inp),$h1 + lea 16($inp),$inp + adc $padbit,$h2 + sub \$16,%r15 + + call __poly1305_block + mov $r1,%rax + + test \$63,%r15 + jnz .Lbase2_64_pre_avx2 + +.Linit_avx2: + ################################# base 2^64 -> base 2^26 + mov $h0,%rax + mov $h0,%rdx + shr \$52,$h0 + mov $h1,$d1 + mov $h1,$d2 + shr \$26,%rdx + and \$0x3ffffff,%rax # h[0] + shl \$12,$d1 + and \$0x3ffffff,%rdx # h[1] + shr \$14,$h1 + or $d1,$h0 + shl \$24,$h2 + and \$0x3ffffff,$h0 # h[2] + shr \$40,$d2 + and \$0x3ffffff,$h1 # h[3] + or $d2,$h2 # h[4] + + vmovd %rax#d,%x#$H0 + vmovd %rdx#d,%x#$H1 + vmovd $h0#d,%x#$H2 + vmovd $h1#d,%x#$H3 + vmovd $h2#d,%x#$H4 + movl \$1,20($ctx) # set is_base2_26 + + call __poly1305_init_avx + + mov %r15,$len # restore $len + + mov 8(%rsp),%r15 +.cfi_restore %r15 + mov 16(%rsp),%r14 +.cfi_restore %r14 + mov 24(%rsp),%r13 +.cfi_restore %r13 + mov 32(%rsp),%r12 +.cfi_restore %r12 + mov 40(%rsp),%rbp +.cfi_restore %rbp + mov 48(%rsp),%rbx +.cfi_restore %rbx + lea 56(%rsp),%rax # for inw64 + lea 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lbase2_64_avx2_epilogue: + jmp .Ldo_avx2 +.cfi_endproc + +.align 32 +.Leven_avx2: +.cfi_startproc + vmovd 4*0($ctx),%x#$H0 # load hash value base 2^26 + vmovd 4*1($ctx),%x#$H1 + vmovd 4*2($ctx),%x#$H2 + vmovd 4*3($ctx),%x#$H3 + vmovd 4*4($ctx),%x#$H4 + +.Ldo_avx2: +___ +$code.=<<___ if ($avx>2 && $flavour !~ /kernel/); + mov OPENSSL_ia32cap_P+8(%rip),%r10d + cmp \$512,$len + jb .Lskip_avx512 + test \$`1<<16`,%r10d # check for AVX512F + jnz .Lblocks_avx512 +.Lskip_avx512: +___ +$code.=<<___ if (!$win64); + lea -8(%rsp),%r11 +.cfi_def_cfa %r11,16 + sub \$0x128,%rsp +___ +$code.=<<___ if ($win64); + lea -0xf8(%rsp),%r11 + sub \$0x1c8,%rsp + vmovdqa %xmm6,0x50(%r11) + vmovdqa %xmm7,0x60(%r11) + vmovdqa %xmm8,0x70(%r11) + vmovdqa %xmm9,0x80(%r11) + vmovdqa %xmm10,0x90(%r11) + vmovdqa %xmm11,0xa0(%r11) + vmovdqa %xmm12,0xb0(%r11) + vmovdqa %xmm13,0xc0(%r11) + vmovdqa %xmm14,0xd0(%r11) + vmovdqa %xmm15,0xe0(%r11) +.Ldo_avx2_body: +___ +$code.=<<___; + lea .Lconst(%rip),%rcx + lea 48+64($ctx),$ctx # size optimization + vmovdqa 96(%rcx),$T0 # .Lpermd_avx2 + + # expand and copy pre-calculated table to stack + vmovdqu `16*0-64`($ctx),%x#$T2 + and \$-512,%rsp + vmovdqu `16*1-64`($ctx),%x#$T3 + vmovdqu `16*2-64`($ctx),%x#$T4 + vmovdqu `16*3-64`($ctx),%x#$D0 + vmovdqu `16*4-64`($ctx),%x#$D1 + vmovdqu `16*5-64`($ctx),%x#$D2 + lea 0x90(%rsp),%rax # size optimization + vmovdqu `16*6-64`($ctx),%x#$D3 + vpermd $T2,$T0,$T2 # 00003412 -> 14243444 + vmovdqu `16*7-64`($ctx),%x#$D4 + vpermd $T3,$T0,$T3 + vmovdqu `16*8-64`($ctx),%x#$MASK + vpermd $T4,$T0,$T4 + vmovdqa $T2,0x00(%rsp) + vpermd $D0,$T0,$D0 + vmovdqa $T3,0x20-0x90(%rax) + vpermd $D1,$T0,$D1 + vmovdqa $T4,0x40-0x90(%rax) + vpermd $D2,$T0,$D2 + vmovdqa $D0,0x60-0x90(%rax) + vpermd $D3,$T0,$D3 + vmovdqa $D1,0x80-0x90(%rax) + vpermd $D4,$T0,$D4 + vmovdqa $D2,0xa0-0x90(%rax) + vpermd $MASK,$T0,$MASK + vmovdqa $D3,0xc0-0x90(%rax) + vmovdqa $D4,0xe0-0x90(%rax) + vmovdqa $MASK,0x100-0x90(%rax) + vmovdqa 64(%rcx),$MASK # .Lmask26 + + ################################################################ + # load input + vmovdqu 16*0($inp),%x#$T0 + vmovdqu 16*1($inp),%x#$T1 + vinserti128 \$1,16*2($inp),$T0,$T0 + vinserti128 \$1,16*3($inp),$T1,$T1 + lea 16*4($inp),$inp + + vpsrldq \$6,$T0,$T2 # splat input + vpsrldq \$6,$T1,$T3 + vpunpckhqdq $T1,$T0,$T4 # 4 + vpunpcklqdq $T3,$T2,$T2 # 2:3 + vpunpcklqdq $T1,$T0,$T0 # 0:1 + + vpsrlq \$30,$T2,$T3 + vpsrlq \$4,$T2,$T2 + vpsrlq \$26,$T0,$T1 + vpsrlq \$40,$T4,$T4 # 4 + vpand $MASK,$T2,$T2 # 2 + vpand $MASK,$T0,$T0 # 0 + vpand $MASK,$T1,$T1 # 1 + vpand $MASK,$T3,$T3 # 3 + vpor 32(%rcx),$T4,$T4 # padbit, yes, always + + vpaddq $H2,$T2,$H2 # accumulate input + sub \$64,$len + jz .Ltail_avx2 + jmp .Loop_avx2 + +.align 32 +.Loop_avx2: + ################################################################ + # ((inp[0]*r^4+inp[4])*r^4+inp[ 8])*r^4 + # ((inp[1]*r^4+inp[5])*r^4+inp[ 9])*r^3 + # ((inp[2]*r^4+inp[6])*r^4+inp[10])*r^2 + # ((inp[3]*r^4+inp[7])*r^4+inp[11])*r^1 + # \________/\__________/ + ################################################################ + #vpaddq $H2,$T2,$H2 # accumulate input + vpaddq $H0,$T0,$H0 + vmovdqa `32*0`(%rsp),$T0 # r0^4 + vpaddq $H1,$T1,$H1 + vmovdqa `32*1`(%rsp),$T1 # r1^4 + vpaddq $H3,$T3,$H3 + vmovdqa `32*3`(%rsp),$T2 # r2^4 + vpaddq $H4,$T4,$H4 + vmovdqa `32*6-0x90`(%rax),$T3 # s3^4 + vmovdqa `32*8-0x90`(%rax),$S4 # s4^4 + + # d4 = h4*r0 + h3*r1 + h2*r2 + h1*r3 + h0*r4 + # d3 = h3*r0 + h2*r1 + h1*r2 + h0*r3 + h4*5*r4 + # d2 = h2*r0 + h1*r1 + h0*r2 + h4*5*r3 + h3*5*r4 + # d1 = h1*r0 + h0*r1 + h4*5*r2 + h3*5*r3 + h2*5*r4 + # d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4 + # + # however, as h2 is "chronologically" first one available pull + # corresponding operations up, so it's + # + # d4 = h2*r2 + h4*r0 + h3*r1 + h1*r3 + h0*r4 + # d3 = h2*r1 + h3*r0 + h1*r2 + h0*r3 + h4*5*r4 + # d2 = h2*r0 + h1*r1 + h0*r2 + h4*5*r3 + h3*5*r4 + # d1 = h2*5*r4 + h1*r0 + h0*r1 + h4*5*r2 + h3*5*r3 + # d0 = h2*5*r3 + h0*r0 + h4*5*r1 + h3*5*r2 + h1*5*r4 + + vpmuludq $H2,$T0,$D2 # d2 = h2*r0 + vpmuludq $H2,$T1,$D3 # d3 = h2*r1 + vpmuludq $H2,$T2,$D4 # d4 = h2*r2 + vpmuludq $H2,$T3,$D0 # d0 = h2*s3 + vpmuludq $H2,$S4,$D1 # d1 = h2*s4 + + vpmuludq $H0,$T1,$T4 # h0*r1 + vpmuludq $H1,$T1,$H2 # h1*r1, borrow $H2 as temp + vpaddq $T4,$D1,$D1 # d1 += h0*r1 + vpaddq $H2,$D2,$D2 # d2 += h1*r1 + vpmuludq $H3,$T1,$T4 # h3*r1 + vpmuludq `32*2`(%rsp),$H4,$H2 # h4*s1 + vpaddq $T4,$D4,$D4 # d4 += h3*r1 + vpaddq $H2,$D0,$D0 # d0 += h4*s1 + vmovdqa `32*4-0x90`(%rax),$T1 # s2 + + vpmuludq $H0,$T0,$T4 # h0*r0 + vpmuludq $H1,$T0,$H2 # h1*r0 + vpaddq $T4,$D0,$D0 # d0 += h0*r0 + vpaddq $H2,$D1,$D1 # d1 += h1*r0 + vpmuludq $H3,$T0,$T4 # h3*r0 + vpmuludq $H4,$T0,$H2 # h4*r0 + vmovdqu 16*0($inp),%x#$T0 # load input + vpaddq $T4,$D3,$D3 # d3 += h3*r0 + vpaddq $H2,$D4,$D4 # d4 += h4*r0 + vinserti128 \$1,16*2($inp),$T0,$T0 + + vpmuludq $H3,$T1,$T4 # h3*s2 + vpmuludq $H4,$T1,$H2 # h4*s2 + vmovdqu 16*1($inp),%x#$T1 + vpaddq $T4,$D0,$D0 # d0 += h3*s2 + vpaddq $H2,$D1,$D1 # d1 += h4*s2 + vmovdqa `32*5-0x90`(%rax),$H2 # r3 + vpmuludq $H1,$T2,$T4 # h1*r2 + vpmuludq $H0,$T2,$T2 # h0*r2 + vpaddq $T4,$D3,$D3 # d3 += h1*r2 + vpaddq $T2,$D2,$D2 # d2 += h0*r2 + vinserti128 \$1,16*3($inp),$T1,$T1 + lea 16*4($inp),$inp + + vpmuludq $H1,$H2,$T4 # h1*r3 + vpmuludq $H0,$H2,$H2 # h0*r3 + vpsrldq \$6,$T0,$T2 # splat input + vpaddq $T4,$D4,$D4 # d4 += h1*r3 + vpaddq $H2,$D3,$D3 # d3 += h0*r3 + vpmuludq $H3,$T3,$T4 # h3*s3 + vpmuludq $H4,$T3,$H2 # h4*s3 + vpsrldq \$6,$T1,$T3 + vpaddq $T4,$D1,$D1 # d1 += h3*s3 + vpaddq $H2,$D2,$D2 # d2 += h4*s3 + vpunpckhqdq $T1,$T0,$T4 # 4 + + vpmuludq $H3,$S4,$H3 # h3*s4 + vpmuludq $H4,$S4,$H4 # h4*s4 + vpunpcklqdq $T1,$T0,$T0 # 0:1 + vpaddq $H3,$D2,$H2 # h2 = d2 + h3*r4 + vpaddq $H4,$D3,$H3 # h3 = d3 + h4*r4 + vpunpcklqdq $T3,$T2,$T3 # 2:3 + vpmuludq `32*7-0x90`(%rax),$H0,$H4 # h0*r4 + vpmuludq $H1,$S4,$H0 # h1*s4 + vmovdqa 64(%rcx),$MASK # .Lmask26 + vpaddq $H4,$D4,$H4 # h4 = d4 + h0*r4 + vpaddq $H0,$D0,$H0 # h0 = d0 + h1*s4 + + ################################################################ + # lazy reduction (interleaved with tail of input splat) + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $D0,$D1,$H1 # h0 -> h1 + + vpsrlq \$26,$H4,$D4 + vpand $MASK,$H4,$H4 + + vpsrlq \$4,$T3,$T2 + + vpsrlq \$26,$H1,$D1 + vpand $MASK,$H1,$H1 + vpaddq $D1,$H2,$H2 # h1 -> h2 + + vpaddq $D4,$H0,$H0 + vpsllq \$2,$D4,$D4 + vpaddq $D4,$H0,$H0 # h4 -> h0 + + vpand $MASK,$T2,$T2 # 2 + vpsrlq \$26,$T0,$T1 + + vpsrlq \$26,$H2,$D2 + vpand $MASK,$H2,$H2 + vpaddq $D2,$H3,$H3 # h2 -> h3 + + vpaddq $T2,$H2,$H2 # modulo-scheduled + vpsrlq \$30,$T3,$T3 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpsrlq \$40,$T4,$T4 # 4 + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vpand $MASK,$T0,$T0 # 0 + vpand $MASK,$T1,$T1 # 1 + vpand $MASK,$T3,$T3 # 3 + vpor 32(%rcx),$T4,$T4 # padbit, yes, always + + sub \$64,$len + jnz .Loop_avx2 + + .byte 0x66,0x90 +.Ltail_avx2: + ################################################################ + # while above multiplications were by r^4 in all lanes, in last + # iteration we multiply least significant lane by r^4 and most + # significant one by r, so copy of above except that references + # to the precomputed table are displaced by 4... + + #vpaddq $H2,$T2,$H2 # accumulate input + vpaddq $H0,$T0,$H0 + vmovdqu `32*0+4`(%rsp),$T0 # r0^4 + vpaddq $H1,$T1,$H1 + vmovdqu `32*1+4`(%rsp),$T1 # r1^4 + vpaddq $H3,$T3,$H3 + vmovdqu `32*3+4`(%rsp),$T2 # r2^4 + vpaddq $H4,$T4,$H4 + vmovdqu `32*6+4-0x90`(%rax),$T3 # s3^4 + vmovdqu `32*8+4-0x90`(%rax),$S4 # s4^4 + + vpmuludq $H2,$T0,$D2 # d2 = h2*r0 + vpmuludq $H2,$T1,$D3 # d3 = h2*r1 + vpmuludq $H2,$T2,$D4 # d4 = h2*r2 + vpmuludq $H2,$T3,$D0 # d0 = h2*s3 + vpmuludq $H2,$S4,$D1 # d1 = h2*s4 + + vpmuludq $H0,$T1,$T4 # h0*r1 + vpmuludq $H1,$T1,$H2 # h1*r1 + vpaddq $T4,$D1,$D1 # d1 += h0*r1 + vpaddq $H2,$D2,$D2 # d2 += h1*r1 + vpmuludq $H3,$T1,$T4 # h3*r1 + vpmuludq `32*2+4`(%rsp),$H4,$H2 # h4*s1 + vpaddq $T4,$D4,$D4 # d4 += h3*r1 + vpaddq $H2,$D0,$D0 # d0 += h4*s1 + + vpmuludq $H0,$T0,$T4 # h0*r0 + vpmuludq $H1,$T0,$H2 # h1*r0 + vpaddq $T4,$D0,$D0 # d0 += h0*r0 + vmovdqu `32*4+4-0x90`(%rax),$T1 # s2 + vpaddq $H2,$D1,$D1 # d1 += h1*r0 + vpmuludq $H3,$T0,$T4 # h3*r0 + vpmuludq $H4,$T0,$H2 # h4*r0 + vpaddq $T4,$D3,$D3 # d3 += h3*r0 + vpaddq $H2,$D4,$D4 # d4 += h4*r0 + + vpmuludq $H3,$T1,$T4 # h3*s2 + vpmuludq $H4,$T1,$H2 # h4*s2 + vpaddq $T4,$D0,$D0 # d0 += h3*s2 + vpaddq $H2,$D1,$D1 # d1 += h4*s2 + vmovdqu `32*5+4-0x90`(%rax),$H2 # r3 + vpmuludq $H1,$T2,$T4 # h1*r2 + vpmuludq $H0,$T2,$T2 # h0*r2 + vpaddq $T4,$D3,$D3 # d3 += h1*r2 + vpaddq $T2,$D2,$D2 # d2 += h0*r2 + + vpmuludq $H1,$H2,$T4 # h1*r3 + vpmuludq $H0,$H2,$H2 # h0*r3 + vpaddq $T4,$D4,$D4 # d4 += h1*r3 + vpaddq $H2,$D3,$D3 # d3 += h0*r3 + vpmuludq $H3,$T3,$T4 # h3*s3 + vpmuludq $H4,$T3,$H2 # h4*s3 + vpaddq $T4,$D1,$D1 # d1 += h3*s3 + vpaddq $H2,$D2,$D2 # d2 += h4*s3 + + vpmuludq $H3,$S4,$H3 # h3*s4 + vpmuludq $H4,$S4,$H4 # h4*s4 + vpaddq $H3,$D2,$H2 # h2 = d2 + h3*r4 + vpaddq $H4,$D3,$H3 # h3 = d3 + h4*r4 + vpmuludq `32*7+4-0x90`(%rax),$H0,$H4 # h0*r4 + vpmuludq $H1,$S4,$H0 # h1*s4 + vmovdqa 64(%rcx),$MASK # .Lmask26 + vpaddq $H4,$D4,$H4 # h4 = d4 + h0*r4 + vpaddq $H0,$D0,$H0 # h0 = d0 + h1*s4 + + ################################################################ + # horizontal addition + + vpsrldq \$8,$D1,$T1 + vpsrldq \$8,$H2,$T2 + vpsrldq \$8,$H3,$T3 + vpsrldq \$8,$H4,$T4 + vpsrldq \$8,$H0,$T0 + vpaddq $T1,$D1,$D1 + vpaddq $T2,$H2,$H2 + vpaddq $T3,$H3,$H3 + vpaddq $T4,$H4,$H4 + vpaddq $T0,$H0,$H0 + + vpermq \$0x2,$H3,$T3 + vpermq \$0x2,$H4,$T4 + vpermq \$0x2,$H0,$T0 + vpermq \$0x2,$D1,$T1 + vpermq \$0x2,$H2,$T2 + vpaddq $T3,$H3,$H3 + vpaddq $T4,$H4,$H4 + vpaddq $T0,$H0,$H0 + vpaddq $T1,$D1,$D1 + vpaddq $T2,$H2,$H2 + + ################################################################ + # lazy reduction + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $D0,$D1,$H1 # h0 -> h1 + + vpsrlq \$26,$H4,$D4 + vpand $MASK,$H4,$H4 + + vpsrlq \$26,$H1,$D1 + vpand $MASK,$H1,$H1 + vpaddq $D1,$H2,$H2 # h1 -> h2 + + vpaddq $D4,$H0,$H0 + vpsllq \$2,$D4,$D4 + vpaddq $D4,$H0,$H0 # h4 -> h0 + + vpsrlq \$26,$H2,$D2 + vpand $MASK,$H2,$H2 + vpaddq $D2,$H3,$H3 # h2 -> h3 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vmovd %x#$H0,`4*0-48-64`($ctx)# save partially reduced + vmovd %x#$H1,`4*1-48-64`($ctx) + vmovd %x#$H2,`4*2-48-64`($ctx) + vmovd %x#$H3,`4*3-48-64`($ctx) + vmovd %x#$H4,`4*4-48-64`($ctx) +___ +$code.=<<___ if ($win64); + vmovdqa 0x50(%r11),%xmm6 + vmovdqa 0x60(%r11),%xmm7 + vmovdqa 0x70(%r11),%xmm8 + vmovdqa 0x80(%r11),%xmm9 + vmovdqa 0x90(%r11),%xmm10 + vmovdqa 0xa0(%r11),%xmm11 + vmovdqa 0xb0(%r11),%xmm12 + vmovdqa 0xc0(%r11),%xmm13 + vmovdqa 0xd0(%r11),%xmm14 + vmovdqa 0xe0(%r11),%xmm15 + lea 0xf8(%r11),%rsp +.Ldo_avx2_epilogue: +___ +$code.=<<___ if (!$win64); + lea 8(%r11),%rsp +.cfi_def_cfa %rsp,8 +___ +$code.=<<___; + vzeroupper + ret +.cfi_endproc +.size poly1305_blocks_avx2,.-poly1305_blocks_avx2 +___ +####################################################################### +if ($avx>2 && $flavour !~ /kernel/) { +# On entry we have input length divisible by 64. But since inner loop +# processes 128 bytes per iteration, cases when length is not divisible +# by 128 are handled by passing tail 64 bytes to .Ltail_avx2. For this +# reason stack layout is kept identical to poly1305_blocks_avx2. If not +# for this tail, we wouldn't have to even allocate stack frame... + +my ($R0,$R1,$R2,$R3,$R4, $S1,$S2,$S3,$S4) = map("%zmm$_",(16..24)); +my ($M0,$M1,$M2,$M3,$M4) = map("%zmm$_",(25..29)); +my $PADBIT="%zmm30"; + +map(s/%y/%z/,($T4,$T0,$T1,$T2,$T3)); # switch to %zmm domain +map(s/%y/%z/,($D0,$D1,$D2,$D3,$D4)); +map(s/%y/%z/,($H0,$H1,$H2,$H3,$H4)); +map(s/%y/%z/,($MASK)); + +$code.=<<___; +.type poly1305_blocks_avx512,\@function,4 +.align 32 +poly1305_blocks_avx512: +.cfi_startproc +.Lblocks_avx512: + mov \$15,%eax + kmovw %eax,%k2 +___ +$code.=<<___ if (!$win64); + lea -8(%rsp),%r11 +.cfi_def_cfa %r11,16 + sub \$0x128,%rsp +___ +$code.=<<___ if ($win64); + lea -0xf8(%rsp),%r11 + sub \$0x1c8,%rsp + vmovdqa %xmm6,0x50(%r11) + vmovdqa %xmm7,0x60(%r11) + vmovdqa %xmm8,0x70(%r11) + vmovdqa %xmm9,0x80(%r11) + vmovdqa %xmm10,0x90(%r11) + vmovdqa %xmm11,0xa0(%r11) + vmovdqa %xmm12,0xb0(%r11) + vmovdqa %xmm13,0xc0(%r11) + vmovdqa %xmm14,0xd0(%r11) + vmovdqa %xmm15,0xe0(%r11) +.Ldo_avx512_body: +___ +$code.=<<___; + lea .Lconst(%rip),%rcx + lea 48+64($ctx),$ctx # size optimization + vmovdqa 96(%rcx),%y#$T2 # .Lpermd_avx2 + + # expand pre-calculated table + vmovdqu `16*0-64`($ctx),%x#$D0 # will become expanded ${R0} + and \$-512,%rsp + vmovdqu `16*1-64`($ctx),%x#$D1 # will become ... ${R1} + mov \$0x20,%rax + vmovdqu `16*2-64`($ctx),%x#$T0 # ... ${S1} + vmovdqu `16*3-64`($ctx),%x#$D2 # ... ${R2} + vmovdqu `16*4-64`($ctx),%x#$T1 # ... ${S2} + vmovdqu `16*5-64`($ctx),%x#$D3 # ... ${R3} + vmovdqu `16*6-64`($ctx),%x#$T3 # ... ${S3} + vmovdqu `16*7-64`($ctx),%x#$D4 # ... ${R4} + vmovdqu `16*8-64`($ctx),%x#$T4 # ... ${S4} + vpermd $D0,$T2,$R0 # 00003412 -> 14243444 + vpbroadcastq 64(%rcx),$MASK # .Lmask26 + vpermd $D1,$T2,$R1 + vpermd $T0,$T2,$S1 + vpermd $D2,$T2,$R2 + vmovdqa64 $R0,0x00(%rsp){%k2} # save in case $len%128 != 0 + vpsrlq \$32,$R0,$T0 # 14243444 -> 01020304 + vpermd $T1,$T2,$S2 + vmovdqu64 $R1,0x00(%rsp,%rax){%k2} + vpsrlq \$32,$R1,$T1 + vpermd $D3,$T2,$R3 + vmovdqa64 $S1,0x40(%rsp){%k2} + vpermd $T3,$T2,$S3 + vpermd $D4,$T2,$R4 + vmovdqu64 $R2,0x40(%rsp,%rax){%k2} + vpermd $T4,$T2,$S4 + vmovdqa64 $S2,0x80(%rsp){%k2} + vmovdqu64 $R3,0x80(%rsp,%rax){%k2} + vmovdqa64 $S3,0xc0(%rsp){%k2} + vmovdqu64 $R4,0xc0(%rsp,%rax){%k2} + vmovdqa64 $S4,0x100(%rsp){%k2} + + ################################################################ + # calculate 5th through 8th powers of the key + # + # d0 = r0'*r0 + r1'*5*r4 + r2'*5*r3 + r3'*5*r2 + r4'*5*r1 + # d1 = r0'*r1 + r1'*r0 + r2'*5*r4 + r3'*5*r3 + r4'*5*r2 + # d2 = r0'*r2 + r1'*r1 + r2'*r0 + r3'*5*r4 + r4'*5*r3 + # d3 = r0'*r3 + r1'*r2 + r2'*r1 + r3'*r0 + r4'*5*r4 + # d4 = r0'*r4 + r1'*r3 + r2'*r2 + r3'*r1 + r4'*r0 + + vpmuludq $T0,$R0,$D0 # d0 = r0'*r0 + vpmuludq $T0,$R1,$D1 # d1 = r0'*r1 + vpmuludq $T0,$R2,$D2 # d2 = r0'*r2 + vpmuludq $T0,$R3,$D3 # d3 = r0'*r3 + vpmuludq $T0,$R4,$D4 # d4 = r0'*r4 + vpsrlq \$32,$R2,$T2 + + vpmuludq $T1,$S4,$M0 + vpmuludq $T1,$R0,$M1 + vpmuludq $T1,$R1,$M2 + vpmuludq $T1,$R2,$M3 + vpmuludq $T1,$R3,$M4 + vpsrlq \$32,$R3,$T3 + vpaddq $M0,$D0,$D0 # d0 += r1'*5*r4 + vpaddq $M1,$D1,$D1 # d1 += r1'*r0 + vpaddq $M2,$D2,$D2 # d2 += r1'*r1 + vpaddq $M3,$D3,$D3 # d3 += r1'*r2 + vpaddq $M4,$D4,$D4 # d4 += r1'*r3 + + vpmuludq $T2,$S3,$M0 + vpmuludq $T2,$S4,$M1 + vpmuludq $T2,$R1,$M3 + vpmuludq $T2,$R2,$M4 + vpmuludq $T2,$R0,$M2 + vpsrlq \$32,$R4,$T4 + vpaddq $M0,$D0,$D0 # d0 += r2'*5*r3 + vpaddq $M1,$D1,$D1 # d1 += r2'*5*r4 + vpaddq $M3,$D3,$D3 # d3 += r2'*r1 + vpaddq $M4,$D4,$D4 # d4 += r2'*r2 + vpaddq $M2,$D2,$D2 # d2 += r2'*r0 + + vpmuludq $T3,$S2,$M0 + vpmuludq $T3,$R0,$M3 + vpmuludq $T3,$R1,$M4 + vpmuludq $T3,$S3,$M1 + vpmuludq $T3,$S4,$M2 + vpaddq $M0,$D0,$D0 # d0 += r3'*5*r2 + vpaddq $M3,$D3,$D3 # d3 += r3'*r0 + vpaddq $M4,$D4,$D4 # d4 += r3'*r1 + vpaddq $M1,$D1,$D1 # d1 += r3'*5*r3 + vpaddq $M2,$D2,$D2 # d2 += r3'*5*r4 + + vpmuludq $T4,$S4,$M3 + vpmuludq $T4,$R0,$M4 + vpmuludq $T4,$S1,$M0 + vpmuludq $T4,$S2,$M1 + vpmuludq $T4,$S3,$M2 + vpaddq $M3,$D3,$D3 # d3 += r2'*5*r4 + vpaddq $M4,$D4,$D4 # d4 += r2'*r0 + vpaddq $M0,$D0,$D0 # d0 += r2'*5*r1 + vpaddq $M1,$D1,$D1 # d1 += r2'*5*r2 + vpaddq $M2,$D2,$D2 # d2 += r2'*5*r3 + + ################################################################ + # load input + vmovdqu64 16*0($inp),%z#$T3 + vmovdqu64 16*4($inp),%z#$T4 + lea 16*8($inp),$inp + + ################################################################ + # lazy reduction + + vpsrlq \$26,$D3,$M3 + vpandq $MASK,$D3,$D3 + vpaddq $M3,$D4,$D4 # d3 -> d4 + + vpsrlq \$26,$D0,$M0 + vpandq $MASK,$D0,$D0 + vpaddq $M0,$D1,$D1 # d0 -> d1 + + vpsrlq \$26,$D4,$M4 + vpandq $MASK,$D4,$D4 + + vpsrlq \$26,$D1,$M1 + vpandq $MASK,$D1,$D1 + vpaddq $M1,$D2,$D2 # d1 -> d2 + + vpaddq $M4,$D0,$D0 + vpsllq \$2,$M4,$M4 + vpaddq $M4,$D0,$D0 # d4 -> d0 + + vpsrlq \$26,$D2,$M2 + vpandq $MASK,$D2,$D2 + vpaddq $M2,$D3,$D3 # d2 -> d3 + + vpsrlq \$26,$D0,$M0 + vpandq $MASK,$D0,$D0 + vpaddq $M0,$D1,$D1 # d0 -> d1 + + vpsrlq \$26,$D3,$M3 + vpandq $MASK,$D3,$D3 + vpaddq $M3,$D4,$D4 # d3 -> d4 + + ################################################################ + # at this point we have 14243444 in $R0-$S4 and 05060708 in + # $D0-$D4, ... + + vpunpcklqdq $T4,$T3,$T0 # transpose input + vpunpckhqdq $T4,$T3,$T4 + + # ... since input 64-bit lanes are ordered as 73625140, we could + # "vperm" it to 76543210 (here and in each loop iteration), *or* + # we could just flow along, hence the goal for $R0-$S4 is + # 1858286838784888 ... + + vmovdqa32 128(%rcx),$M0 # .Lpermd_avx512: + mov \$0x7777,%eax + kmovw %eax,%k1 + + vpermd $R0,$M0,$R0 # 14243444 -> 1---2---3---4--- + vpermd $R1,$M0,$R1 + vpermd $R2,$M0,$R2 + vpermd $R3,$M0,$R3 + vpermd $R4,$M0,$R4 + + vpermd $D0,$M0,${R0}{%k1} # 05060708 -> 1858286838784888 + vpermd $D1,$M0,${R1}{%k1} + vpermd $D2,$M0,${R2}{%k1} + vpermd $D3,$M0,${R3}{%k1} + vpermd $D4,$M0,${R4}{%k1} + + vpslld \$2,$R1,$S1 # *5 + vpslld \$2,$R2,$S2 + vpslld \$2,$R3,$S3 + vpslld \$2,$R4,$S4 + vpaddd $R1,$S1,$S1 + vpaddd $R2,$S2,$S2 + vpaddd $R3,$S3,$S3 + vpaddd $R4,$S4,$S4 + + vpbroadcastq 32(%rcx),$PADBIT # .L129 + + vpsrlq \$52,$T0,$T2 # splat input + vpsllq \$12,$T4,$T3 + vporq $T3,$T2,$T2 + vpsrlq \$26,$T0,$T1 + vpsrlq \$14,$T4,$T3 + vpsrlq \$40,$T4,$T4 # 4 + vpandq $MASK,$T2,$T2 # 2 + vpandq $MASK,$T0,$T0 # 0 + #vpandq $MASK,$T1,$T1 # 1 + #vpandq $MASK,$T3,$T3 # 3 + #vporq $PADBIT,$T4,$T4 # padbit, yes, always + + vpaddq $H2,$T2,$H2 # accumulate input + sub \$192,$len + jbe .Ltail_avx512 + jmp .Loop_avx512 + +.align 32 +.Loop_avx512: + ################################################################ + # ((inp[0]*r^8+inp[ 8])*r^8+inp[16])*r^8 + # ((inp[1]*r^8+inp[ 9])*r^8+inp[17])*r^7 + # ((inp[2]*r^8+inp[10])*r^8+inp[18])*r^6 + # ((inp[3]*r^8+inp[11])*r^8+inp[19])*r^5 + # ((inp[4]*r^8+inp[12])*r^8+inp[20])*r^4 + # ((inp[5]*r^8+inp[13])*r^8+inp[21])*r^3 + # ((inp[6]*r^8+inp[14])*r^8+inp[22])*r^2 + # ((inp[7]*r^8+inp[15])*r^8+inp[23])*r^1 + # \________/\___________/ + ################################################################ + #vpaddq $H2,$T2,$H2 # accumulate input + + # d4 = h4*r0 + h3*r1 + h2*r2 + h1*r3 + h0*r4 + # d3 = h3*r0 + h2*r1 + h1*r2 + h0*r3 + h4*5*r4 + # d2 = h2*r0 + h1*r1 + h0*r2 + h4*5*r3 + h3*5*r4 + # d1 = h1*r0 + h0*r1 + h4*5*r2 + h3*5*r3 + h2*5*r4 + # d0 = h0*r0 + h4*5*r1 + h3*5*r2 + h2*5*r3 + h1*5*r4 + # + # however, as h2 is "chronologically" first one available pull + # corresponding operations up, so it's + # + # d3 = h2*r1 + h0*r3 + h1*r2 + h3*r0 + h4*5*r4 + # d4 = h2*r2 + h0*r4 + h1*r3 + h3*r1 + h4*r0 + # d0 = h2*5*r3 + h0*r0 + h1*5*r4 + h3*5*r2 + h4*5*r1 + # d1 = h2*5*r4 + h0*r1 + h1*r0 + h3*5*r3 + h4*5*r2 + # d2 = h2*r0 + h0*r2 + h1*r1 + h3*5*r4 + h4*5*r3 + + vpmuludq $H2,$R1,$D3 # d3 = h2*r1 + vpaddq $H0,$T0,$H0 + vpmuludq $H2,$R2,$D4 # d4 = h2*r2 + vpandq $MASK,$T1,$T1 # 1 + vpmuludq $H2,$S3,$D0 # d0 = h2*s3 + vpandq $MASK,$T3,$T3 # 3 + vpmuludq $H2,$S4,$D1 # d1 = h2*s4 + vporq $PADBIT,$T4,$T4 # padbit, yes, always + vpmuludq $H2,$R0,$D2 # d2 = h2*r0 + vpaddq $H1,$T1,$H1 # accumulate input + vpaddq $H3,$T3,$H3 + vpaddq $H4,$T4,$H4 + + vmovdqu64 16*0($inp),$T3 # load input + vmovdqu64 16*4($inp),$T4 + lea 16*8($inp),$inp + vpmuludq $H0,$R3,$M3 + vpmuludq $H0,$R4,$M4 + vpmuludq $H0,$R0,$M0 + vpmuludq $H0,$R1,$M1 + vpaddq $M3,$D3,$D3 # d3 += h0*r3 + vpaddq $M4,$D4,$D4 # d4 += h0*r4 + vpaddq $M0,$D0,$D0 # d0 += h0*r0 + vpaddq $M1,$D1,$D1 # d1 += h0*r1 + + vpmuludq $H1,$R2,$M3 + vpmuludq $H1,$R3,$M4 + vpmuludq $H1,$S4,$M0 + vpmuludq $H0,$R2,$M2 + vpaddq $M3,$D3,$D3 # d3 += h1*r2 + vpaddq $M4,$D4,$D4 # d4 += h1*r3 + vpaddq $M0,$D0,$D0 # d0 += h1*s4 + vpaddq $M2,$D2,$D2 # d2 += h0*r2 + + vpunpcklqdq $T4,$T3,$T0 # transpose input + vpunpckhqdq $T4,$T3,$T4 + + vpmuludq $H3,$R0,$M3 + vpmuludq $H3,$R1,$M4 + vpmuludq $H1,$R0,$M1 + vpmuludq $H1,$R1,$M2 + vpaddq $M3,$D3,$D3 # d3 += h3*r0 + vpaddq $M4,$D4,$D4 # d4 += h3*r1 + vpaddq $M1,$D1,$D1 # d1 += h1*r0 + vpaddq $M2,$D2,$D2 # d2 += h1*r1 + + vpmuludq $H4,$S4,$M3 + vpmuludq $H4,$R0,$M4 + vpmuludq $H3,$S2,$M0 + vpmuludq $H3,$S3,$M1 + vpaddq $M3,$D3,$D3 # d3 += h4*s4 + vpmuludq $H3,$S4,$M2 + vpaddq $M4,$D4,$D4 # d4 += h4*r0 + vpaddq $M0,$D0,$D0 # d0 += h3*s2 + vpaddq $M1,$D1,$D1 # d1 += h3*s3 + vpaddq $M2,$D2,$D2 # d2 += h3*s4 + + vpmuludq $H4,$S1,$M0 + vpmuludq $H4,$S2,$M1 + vpmuludq $H4,$S3,$M2 + vpaddq $M0,$D0,$H0 # h0 = d0 + h4*s1 + vpaddq $M1,$D1,$H1 # h1 = d2 + h4*s2 + vpaddq $M2,$D2,$H2 # h2 = d3 + h4*s3 + + ################################################################ + # lazy reduction (interleaved with input splat) + + vpsrlq \$52,$T0,$T2 # splat input + vpsllq \$12,$T4,$T3 + + vpsrlq \$26,$D3,$H3 + vpandq $MASK,$D3,$D3 + vpaddq $H3,$D4,$H4 # h3 -> h4 + + vporq $T3,$T2,$T2 + + vpsrlq \$26,$H0,$D0 + vpandq $MASK,$H0,$H0 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpandq $MASK,$T2,$T2 # 2 + + vpsrlq \$26,$H4,$D4 + vpandq $MASK,$H4,$H4 + + vpsrlq \$26,$H1,$D1 + vpandq $MASK,$H1,$H1 + vpaddq $D1,$H2,$H2 # h1 -> h2 + + vpaddq $D4,$H0,$H0 + vpsllq \$2,$D4,$D4 + vpaddq $D4,$H0,$H0 # h4 -> h0 + + vpaddq $T2,$H2,$H2 # modulo-scheduled + vpsrlq \$26,$T0,$T1 + + vpsrlq \$26,$H2,$D2 + vpandq $MASK,$H2,$H2 + vpaddq $D2,$D3,$H3 # h2 -> h3 + + vpsrlq \$14,$T4,$T3 + + vpsrlq \$26,$H0,$D0 + vpandq $MASK,$H0,$H0 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpsrlq \$40,$T4,$T4 # 4 + + vpsrlq \$26,$H3,$D3 + vpandq $MASK,$H3,$H3 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vpandq $MASK,$T0,$T0 # 0 + #vpandq $MASK,$T1,$T1 # 1 + #vpandq $MASK,$T3,$T3 # 3 + #vporq $PADBIT,$T4,$T4 # padbit, yes, always + + sub \$128,$len + ja .Loop_avx512 + +.Ltail_avx512: + ################################################################ + # while above multiplications were by r^8 in all lanes, in last + # iteration we multiply least significant lane by r^8 and most + # significant one by r, that's why table gets shifted... + + vpsrlq \$32,$R0,$R0 # 0105020603070408 + vpsrlq \$32,$R1,$R1 + vpsrlq \$32,$R2,$R2 + vpsrlq \$32,$S3,$S3 + vpsrlq \$32,$S4,$S4 + vpsrlq \$32,$R3,$R3 + vpsrlq \$32,$R4,$R4 + vpsrlq \$32,$S1,$S1 + vpsrlq \$32,$S2,$S2 + + ################################################################ + # load either next or last 64 byte of input + lea ($inp,$len),$inp + + #vpaddq $H2,$T2,$H2 # accumulate input + vpaddq $H0,$T0,$H0 + + vpmuludq $H2,$R1,$D3 # d3 = h2*r1 + vpmuludq $H2,$R2,$D4 # d4 = h2*r2 + vpmuludq $H2,$S3,$D0 # d0 = h2*s3 + vpandq $MASK,$T1,$T1 # 1 + vpmuludq $H2,$S4,$D1 # d1 = h2*s4 + vpandq $MASK,$T3,$T3 # 3 + vpmuludq $H2,$R0,$D2 # d2 = h2*r0 + vporq $PADBIT,$T4,$T4 # padbit, yes, always + vpaddq $H1,$T1,$H1 # accumulate input + vpaddq $H3,$T3,$H3 + vpaddq $H4,$T4,$H4 + + vmovdqu 16*0($inp),%x#$T0 + vpmuludq $H0,$R3,$M3 + vpmuludq $H0,$R4,$M4 + vpmuludq $H0,$R0,$M0 + vpmuludq $H0,$R1,$M1 + vpaddq $M3,$D3,$D3 # d3 += h0*r3 + vpaddq $M4,$D4,$D4 # d4 += h0*r4 + vpaddq $M0,$D0,$D0 # d0 += h0*r0 + vpaddq $M1,$D1,$D1 # d1 += h0*r1 + + vmovdqu 16*1($inp),%x#$T1 + vpmuludq $H1,$R2,$M3 + vpmuludq $H1,$R3,$M4 + vpmuludq $H1,$S4,$M0 + vpmuludq $H0,$R2,$M2 + vpaddq $M3,$D3,$D3 # d3 += h1*r2 + vpaddq $M4,$D4,$D4 # d4 += h1*r3 + vpaddq $M0,$D0,$D0 # d0 += h1*s4 + vpaddq $M2,$D2,$D2 # d2 += h0*r2 + + vinserti128 \$1,16*2($inp),%y#$T0,%y#$T0 + vpmuludq $H3,$R0,$M3 + vpmuludq $H3,$R1,$M4 + vpmuludq $H1,$R0,$M1 + vpmuludq $H1,$R1,$M2 + vpaddq $M3,$D3,$D3 # d3 += h3*r0 + vpaddq $M4,$D4,$D4 # d4 += h3*r1 + vpaddq $M1,$D1,$D1 # d1 += h1*r0 + vpaddq $M2,$D2,$D2 # d2 += h1*r1 + + vinserti128 \$1,16*3($inp),%y#$T1,%y#$T1 + vpmuludq $H4,$S4,$M3 + vpmuludq $H4,$R0,$M4 + vpmuludq $H3,$S2,$M0 + vpmuludq $H3,$S3,$M1 + vpmuludq $H3,$S4,$M2 + vpaddq $M3,$D3,$H3 # h3 = d3 + h4*s4 + vpaddq $M4,$D4,$D4 # d4 += h4*r0 + vpaddq $M0,$D0,$D0 # d0 += h3*s2 + vpaddq $M1,$D1,$D1 # d1 += h3*s3 + vpaddq $M2,$D2,$D2 # d2 += h3*s4 + + vpmuludq $H4,$S1,$M0 + vpmuludq $H4,$S2,$M1 + vpmuludq $H4,$S3,$M2 + vpaddq $M0,$D0,$H0 # h0 = d0 + h4*s1 + vpaddq $M1,$D1,$H1 # h1 = d2 + h4*s2 + vpaddq $M2,$D2,$H2 # h2 = d3 + h4*s3 + + ################################################################ + # horizontal addition + + mov \$1,%eax + vpermq \$0xb1,$H3,$D3 + vpermq \$0xb1,$D4,$H4 + vpermq \$0xb1,$H0,$D0 + vpermq \$0xb1,$H1,$D1 + vpermq \$0xb1,$H2,$D2 + vpaddq $D3,$H3,$H3 + vpaddq $D4,$H4,$H4 + vpaddq $D0,$H0,$H0 + vpaddq $D1,$H1,$H1 + vpaddq $D2,$H2,$H2 + + kmovw %eax,%k3 + vpermq \$0x2,$H3,$D3 + vpermq \$0x2,$H4,$D4 + vpermq \$0x2,$H0,$D0 + vpermq \$0x2,$H1,$D1 + vpermq \$0x2,$H2,$D2 + vpaddq $D3,$H3,$H3 + vpaddq $D4,$H4,$H4 + vpaddq $D0,$H0,$H0 + vpaddq $D1,$H1,$H1 + vpaddq $D2,$H2,$H2 + + vextracti64x4 \$0x1,$H3,%y#$D3 + vextracti64x4 \$0x1,$H4,%y#$D4 + vextracti64x4 \$0x1,$H0,%y#$D0 + vextracti64x4 \$0x1,$H1,%y#$D1 + vextracti64x4 \$0x1,$H2,%y#$D2 + vpaddq $D3,$H3,${H3}{%k3}{z} # keep single qword in case + vpaddq $D4,$H4,${H4}{%k3}{z} # it's passed to .Ltail_avx2 + vpaddq $D0,$H0,${H0}{%k3}{z} + vpaddq $D1,$H1,${H1}{%k3}{z} + vpaddq $D2,$H2,${H2}{%k3}{z} +___ +map(s/%z/%y/,($T0,$T1,$T2,$T3,$T4, $PADBIT)); +map(s/%z/%y/,($H0,$H1,$H2,$H3,$H4, $D0,$D1,$D2,$D3,$D4, $MASK)); +$code.=<<___; + ################################################################ + # lazy reduction (interleaved with input splat) + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpsrldq \$6,$T0,$T2 # splat input + vpsrldq \$6,$T1,$T3 + vpunpckhqdq $T1,$T0,$T4 # 4 + vpaddq $D3,$H4,$H4 # h3 -> h4 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpunpcklqdq $T3,$T2,$T2 # 2:3 + vpunpcklqdq $T1,$T0,$T0 # 0:1 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpsrlq \$26,$H4,$D4 + vpand $MASK,$H4,$H4 + + vpsrlq \$26,$H1,$D1 + vpand $MASK,$H1,$H1 + vpsrlq \$30,$T2,$T3 + vpsrlq \$4,$T2,$T2 + vpaddq $D1,$H2,$H2 # h1 -> h2 + + vpaddq $D4,$H0,$H0 + vpsllq \$2,$D4,$D4 + vpsrlq \$26,$T0,$T1 + vpsrlq \$40,$T4,$T4 # 4 + vpaddq $D4,$H0,$H0 # h4 -> h0 + + vpsrlq \$26,$H2,$D2 + vpand $MASK,$H2,$H2 + vpand $MASK,$T2,$T2 # 2 + vpand $MASK,$T0,$T0 # 0 + vpaddq $D2,$H3,$H3 # h2 -> h3 + + vpsrlq \$26,$H0,$D0 + vpand $MASK,$H0,$H0 + vpaddq $H2,$T2,$H2 # accumulate input for .Ltail_avx2 + vpand $MASK,$T1,$T1 # 1 + vpaddq $D0,$H1,$H1 # h0 -> h1 + + vpsrlq \$26,$H3,$D3 + vpand $MASK,$H3,$H3 + vpand $MASK,$T3,$T3 # 3 + vpor 32(%rcx),$T4,$T4 # padbit, yes, always + vpaddq $D3,$H4,$H4 # h3 -> h4 + + lea 0x90(%rsp),%rax # size optimization for .Ltail_avx2 + add \$64,$len + jnz .Ltail_avx2 + + vpsubq $T2,$H2,$H2 # undo input accumulation + vmovd %x#$H0,`4*0-48-64`($ctx)# save partially reduced + vmovd %x#$H1,`4*1-48-64`($ctx) + vmovd %x#$H2,`4*2-48-64`($ctx) + vmovd %x#$H3,`4*3-48-64`($ctx) + vmovd %x#$H4,`4*4-48-64`($ctx) + vzeroall +___ +$code.=<<___ if ($win64); + movdqa 0x50(%r11),%xmm6 + movdqa 0x60(%r11),%xmm7 + movdqa 0x70(%r11),%xmm8 + movdqa 0x80(%r11),%xmm9 + movdqa 0x90(%r11),%xmm10 + movdqa 0xa0(%r11),%xmm11 + movdqa 0xb0(%r11),%xmm12 + movdqa 0xc0(%r11),%xmm13 + movdqa 0xd0(%r11),%xmm14 + movdqa 0xe0(%r11),%xmm15 + lea 0xf8(%r11),%rsp +.Ldo_avx512_epilogue: +___ +$code.=<<___ if (!$win64); + lea 8(%r11),%rsp +.cfi_def_cfa %rsp,8 +___ +$code.=<<___; + ret +.cfi_endproc +.size poly1305_blocks_avx512,.-poly1305_blocks_avx512 +___ +} +if ($avx>3) { +######################################################################## +# VPMADD52 version using 2^44 radix. +# +# One can argue that base 2^52 would be more natural. Well, even though +# some operations would be more natural, one has to recognize couple of +# things. Base 2^52 doesn't provide advantage over base 2^44 if you look +# at amount of multiply-n-accumulate operations. Secondly, it makes it +# impossible to pre-compute multiples of 5 [referred to as s[]/sN in +# reference implementations], which means that more such operations +# would have to be performed in inner loop, which in turn makes critical +# path longer. In other words, even though base 2^44 reduction might +# look less elegant, overall critical path is actually shorter... + +######################################################################## +# Layout of opaque area is following. +# +# unsigned __int64 h[3]; # current hash value base 2^44 +# unsigned __int64 s[2]; # key value*20 base 2^44 +# unsigned __int64 r[3]; # key value base 2^44 +# struct { unsigned __int64 r^1, r^3, r^2, r^4; } R[4]; +# # r^n positions reflect +# # placement in register, not +# # memory, R[3] is R[1]*20 + +$code.=<<___; +.type poly1305_init_base2_44,\@function,3 +.align 32 +poly1305_init_base2_44: + xor %rax,%rax + mov %rax,0($ctx) # initialize hash value + mov %rax,8($ctx) + mov %rax,16($ctx) + + cmp \$0,$inp + je .Lno_key_base2_44 + +.Linit_base2_44: + mov \$0x0ffffffc0fffffff,%rax + mov \$0x0ffffffc0ffffffc,%rcx + and 0($inp),%rax + mov \$0x00000fffffffffff,%r8 + and 8($inp),%rcx + mov \$0x00000fffffffffff,%r9 + and %rax,%r8 # base 2^64 -> base 2^44 + shrd \$44,%rcx,%rax + mov %r8,40($ctx) # r0 + and %r9,%rax + shr \$24,%rcx + mov %rax,48($ctx) # r1 + lea (%rax,%rax,4),%rax # *5 + mov %rcx,56($ctx) # r2 + shl \$2,%rax # magic <<2 + lea (%rcx,%rcx,4),%rcx # *5 + shl \$2,%rcx # magic <<2 + mov %rax,24($ctx) # s1 + mov %rcx,32($ctx) # s2 + movq \$-1,64($ctx) # write impossible value +___ + if ($flavour !~ /kernel/) { +$code.=<<___; + lea poly1305_blocks_vpmadd52(%rip),%r10 + lea poly1305_emit_base2_44(%rip),%r11 +___ +$code.=<<___ if ($flavour !~ /elf32/); + mov %r10,0(%rdx) + mov %r11,8(%rdx) +___ +$code.=<<___ if ($flavour =~ /elf32/); + mov %r10d,0(%rdx) + mov %r11d,4(%rdx) +___ + } +$code.=<<___; + mov \$1,%eax +.Lno_key_base2_44: + ret +.size poly1305_init_base2_44,.-poly1305_init_base2_44 +___ +{ +my ($h0,$h1,$h2, $d1,$d2,$d3, $r0,$r1,$s2) = map("%r$_",("dx",8..15)); +$code.=<<___; +.type poly1305_blocks_base2_44,\@function,4 +.align 32 +poly1305_blocks_base2_44: +.cfi_startproc +.Lblocks_base2_44: + push %rbx +.cfi_push %rbx + push %rbp +.cfi_push %rbp + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + + and \$-16,$len + add $inp,$len # end of buffer + shl \$40,$padbit + push $len +.cfi_adjust_cfa_offset 8 +.Lblocks_base2_44_body: + + mov 0($ctx),$h0 # load hash value + mov 8($ctx),$h1 + mov 16($ctx),$h2 + + mov 40($ctx),$r0 # load key + mov 48($ctx),$r1 + mov 32($ctx),$s2 + mov \$0xfffff00000000000,%rax + jmp .Loop_base2_44 + ud2 + +.align 32 +.Loop_base2_44: + mov 0($inp),$d2 # load input + mov 8($inp),$d3 + lea 16($inp),$inp + + andn $d2,%rax,$d1 # base 2^64 -> base 2^44 + shrd \$44,$d3,$d2 + add $d1,$h0 # accumulate input + shr \$24,$d3 + andn $d2,%rax,$d2 + add $padbit,$h2 + + add $d2,$h1 + add $d3,$h2 + + #mov $h0,%rdx # h0 is %rdx + mulx $r0,$d1,%rbx # h0*r0 + mulx $r1,$d2,%rcx # h0*r1 + mulx 56($ctx),$d3,%rbp # h0*r2 + + mov $h1,%rdx + mulx $s2,%rax,$h1 # h1*s2 + add %rax,$d1 + adc %rbx,$h1 + mulx $r0,%rax,%rbx # h1*r0 + add %rax,$d2 + adc %rbx,%rcx + mulx $r1,%rax,%rbx # h1*r1 + mov $h2,%rdx + add %rax,$d3 + adc %rbx,%rbp + + mulx 24($ctx),%rax,%rbx # h2*s1 + add %rax,$d1 + adc %rbx,$h1 + mulx $s2,%rax,$h2 # h2*s2 + add %rax,$d2 + adc %rcx,$h2 + mulx $r0,%rax,%rbx # h2*r0 + add %rax,$d3 + adc %rbx,%rbp + + mov \$0xfffff00000000000,%rax + andn $d1,%rax,$h0 + shrd \$44,$h1,$d1 + add $d1,$d2 + adc \$0,$h2 + andn $d2,%rax,$h1 + shrd \$44,$h2,$d2 + mov \$0x03ffffffffff,$h2 + add $d2,$d3 + adc \$0,%rbp + and $d3,$h2 + shrd \$42,%rbp,$d3 + + mov \$0x10000000000,$padbit + lea ($d3,$d3,4),$d3 # *=5 + add $d3,$h0 + + cmp 0(%rsp),$inp + jb .Loop_base2_44 + + mov $h0,0($ctx) # store hash value + mov $h1,8($ctx) + mov $h2,16($ctx) + + mov 8(%rsp),%r15 +.cfi_restore %r15 + mov 16(%rsp),%r14 +.cfi_restore %r14 + mov 24(%rsp),%r13 +.cfi_restore %r13 + mov 32(%rsp),%r12 +.cfi_restore %r12 + mov 40(%rsp),%rbp +.cfi_restore %rbp + mov 48(%rsp),%rbx +.cfi_restore %rbx + lea 56(%rsp),%rsp +.cfi_adjust_cfa_offset -56 +.Lblocks_base2_44_epilogue: + ret +.cfi_endproc +.size poly1305_blocks_base2_44,.-poly1305_blocks_base2_44 +___ +} +{ +my ($H0,$H1,$H2,$r2r1r0,$r1r0s2,$r0s2s1,$Dlo,$Dhi) = map("%ymm$_",(0..5,16,17)); +my ($T0,$inp_permd,$inp_shift,$PAD) = map("%ymm$_",(18..21)); +my ($reduc_mask,$reduc_rght,$reduc_left) = map("%ymm$_",(22..25)); +my ($T1,$T2,$T3) = map("%ymm$_",(26..28)); + +$code.=<<___; +.type poly1305_blocks_vpmadd52,\@function,4 +.align 32 +poly1305_blocks_vpmadd52: + and \$-16,$len + jz .Lno_data_vpmadd52 # too short + + mov 64($ctx),%r8 # peek on power of the key + + # if powers of the key are not calculated yet, process up to 3 + # blocks with scalar single-block subroutine above, otherwise + # ensure that input length is divisible by 2 blocks and pass + # the rest down to next subroutine... + + mov \$0x30,%r9 + mov \$0x10,%r10 + cmp \$0x40,$len # is input long + cmovae %r10,%r9 + test %r8,%r8 # is power value impossible? + cmovns %r10,%r9 + + and $len,%r9 # is input of favourable length? + jz .Lblocks_vpmadd52_4x + + sub %r9,$len + cmovz %r9,$len + jz .Lblocks_base2_44 + + ######################################### + mov \$7,%r10d + mov \$1,%r11d + shl \$40,$padbit + kmovw %r10d,%k7 + lea .L2_44_inp_permd(%rip),%r10 + kmovw %r11d,%k1 + + vmovq $padbit,%x#$PAD + shr \$40,$padbit # restore original value + vmovdqa64 0(%r10),$inp_permd # .L2_44_inp_permd + vmovdqa64 32(%r10),$inp_shift # .L2_44_inp_shift + vpermq \$0xcf,$PAD,$PAD + vmovdqa64 64(%r10),$reduc_mask # .L2_44_mask + + vmovdqu64 0($ctx),${Dlo}{%k7}{z} # load hash value + vmovdqu64 40($ctx),${r2r1r0}{%k7}{z} # load keys + vmovdqu64 32($ctx),${r1r0s2}{%k7}{z} + vmovdqu64 24($ctx),${r0s2s1}{%k7}{z} + + vmovdqa64 96(%r10),$reduc_rght # .L2_44_shift_rgt + vmovdqa64 128(%r10),$reduc_left # .L2_44_shift_lft + + vmovdqu32 0($inp),%x#$T0 # load input as ----3210 + lea 16($inp),$inp + + vpermd $T0,$inp_permd,$T0 # ----3210 -> --322110 + vpsrlvq $inp_shift,$T0,$T0 + vpandq $reduc_mask,$T0,$T0 + vporq $PAD,$T0,$T0 + + vpaddq $T0,$Dlo,$Dlo # accumulate input + vpxord $T2,$T2,$T2 + vpxord $T3,$T3,$T3 + + vpermq \$0,$Dlo,${H0}{%k7}{z} # smash hash value + vpermq \$0b01010101,$Dlo,${H1}{%k7}{z} + vpermq \$0b10101010,$Dlo,${H2}{%k7}{z} + + vpxord $T0,$T0,$T0 + vpxord $T1,$T1,$T1 + vpmadd52luq $r2r1r0,$H0,$T2 + vpmadd52huq $r2r1r0,$H0,$T3 + + vpxord $Dlo,$Dlo,$Dlo + vpxord $Dhi,$Dhi,$Dhi + vpmadd52luq $r1r0s2,$H1,$T0 + vpmadd52huq $r1r0s2,$H1,$T1 + + vpmadd52luq $r0s2s1,$H2,$Dlo + vpmadd52huq $r0s2s1,$H2,$Dhi + + vpaddq $T0,$T2,$T2 + vpaddq $T1,$T3,$T3 + vpaddq $T2,$Dlo,$Dlo + vpaddq $T3,$Dhi,$Dhi + + vpsrlvq $reduc_rght,$Dlo,$T0 # 0 in topmost qword + vpsllvq $reduc_left,$Dhi,$Dhi # 0 in topmost qword + vpandq $reduc_mask,$Dlo,$Dlo + + vpaddq $T0,$Dhi,$Dhi + + vpermq \$0b10010011,$Dhi,$Dhi # 0 in lowest qword + + vpaddq $Dhi,$Dlo,$Dlo # note topmost qword :-) + + vpsrlvq $reduc_rght,$Dlo,$T0 # 0 in topmost word + vpandq $reduc_mask,$Dlo,$Dlo + + vpermq \$0b10010011,$T0,$T0 + + vpaddq $T0,$Dlo,$Dlo + + vpermq \$0b10010011,$Dlo,${T0}{%k1}{z} + + vpaddq $T0,$Dlo,$Dlo + vpsllq \$2,$T0,$T0 + + vpaddq $T0,$Dlo,$Dlo + + vmovdqu64 $Dlo,0($ctx){%k7} # store hash value + + jmp .Lblocks_vpmadd52_4x + +.Lno_data_vpmadd52: + ret +.size poly1305_blocks_vpmadd52,.-poly1305_blocks_vpmadd52 +___ +} +{ +######################################################################## +# As implied by its name 4x subroutine processes 4 blocks in parallel +# (but handles even 4*n+2 blocks lengths). It takes up to 4th key power +# and is handled in 256-bit %ymm registers. + +my ($H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2) = map("%ymm$_",(0..5,16,17)); +my ($D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi) = map("%ymm$_",(18..23)); +my ($T0,$T1,$T2,$T3,$T4,$tmp,$mask44,$PAD) = map("%ymm$_",(24..31)); + +$code.=<<___; +.type poly1305_blocks_vpmadd52_4x,\@function,4 +.align 32 +poly1305_blocks_vpmadd52_4x: + and \$-16,$len + jz .Lno_data_vpmadd52_4x # too short + + mov 64($ctx),%r8 # peek on power of the key + +.Lblocks_vpmadd52_4x: + shl \$40,$padbit + shr \$4,$len + vpbroadcastq $padbit,$PAD + + vmovdqa64 .Lx_mask44(%rip),$mask44 + mov \$5,%eax + kmovw %eax,%k1 # used in 2x path + + test %r8,%r8 # is power value impossible? + js .Linit_vpmadd52 # if it is, then init R[4] + + vmovq 0($ctx),%x#$H0 # load current hash value + vmovq 8($ctx),%x#$H1 + vmovq 16($ctx),%x#$H2 + + test \$3,$len # is length 4*n+2? + jnz .Lblocks_vpmadd52_2x_do + +.Lblocks_vpmadd52_4x_do: + vpbroadcastq 64($ctx),$R0 # load 4th power of the key + vpbroadcastq 96($ctx),$R1 + vpbroadcastq 128($ctx),$R2 + vpbroadcastq 160($ctx),$S1 + +.Lblocks_vpmadd52_4x_key_loaded: + vpsllq \$2,$R2,$S2 # S2 = R2*5*4 + vpaddq $R2,$S2,$S2 + vpsllq \$2,$S2,$S2 + + #test \$7,$len # is len 8*n? + #jz .Lblocks_vpmadd52_8x + + vmovdqu64 16*0($inp),$T2 # load data + vmovdqu64 16*2($inp),$T3 + lea 16*4($inp),$inp + + vpunpcklqdq $T3,$T2,$T1 # transpose data + vpunpckhqdq $T3,$T2,$T3 + + # at this point 64-bit lanes are ordered as 3-1-2-0 + + vpsrlq \$24,$T3,$T2 # splat the data + vporq $PAD,$T2,$T2 + vpaddq $T2,$H2,$H2 # accumulate input + vpandq $mask44,$T1,$T0 + vpsrlq \$44,$T1,$T1 + vpsllq \$20,$T3,$T3 + vporq $T3,$T1,$T1 + vpandq $mask44,$T1,$T1 + + sub \$4,$len + jz .Ltail_vpmadd52_4x + jmp .Loop_vpmadd52_4x + ud2 + +.align 32 +.Linit_vpmadd52: + vmovq 24($ctx),%x#$S1 # load key + vmovq 56($ctx),%x#$H2 + vmovq 32($ctx),%x#$S2 + vmovq 40($ctx),%x#$R0 + vmovq 48($ctx),%x#$R1 + + vmovdqa $R0,$H0 + vmovdqa $R1,$H1 + vmovdqa $H2,$R2 + + mov \$2,%eax + +.Lmul_init_vpmadd52: + vpxorq $D0lo,$D0lo,$D0lo + vpxorq $D0hi,$D0hi,$D0hi + vpxorq $D1lo,$D1lo,$D1lo + vpxorq $D1hi,$D1hi,$D1hi + vpxorq $D2lo,$D2lo,$D2lo + vpxorq $D2hi,$D2hi,$D2hi + vpmadd52luq $H2,$S1,$D0lo + vpxorq $T0,$T0,$T0 + vpxorq $T1,$T1,$T1 + vpmadd52huq $H2,$S1,$D0hi + vpxorq $T2,$T2,$T2 + vpxorq $T3,$T3,$T3 + vpmadd52luq $H2,$S2,$D1lo + vpxorq $T4,$T4,$T4 + vpxorq $tmp,$tmp,$tmp + vpmadd52huq $H2,$S2,$D1hi + vpmadd52luq $H2,$R0,$D2lo + vpmadd52huq $H2,$R0,$D2hi + + vpmadd52luq $H0,$R0,$T0 + vpmadd52huq $H0,$R0,$T1 + vpmadd52luq $H0,$R1,$T2 + vpmadd52huq $H0,$R1,$T3 + vpmadd52luq $H0,$R2,$T4 + vpmadd52huq $H0,$R2,$tmp + + vpmadd52luq $H1,$S2,$D0lo + vpmadd52huq $H1,$S2,$D0hi + vpmadd52luq $H1,$R0,$D1lo + vpmadd52huq $H1,$R0,$D1hi + vpaddq $T0,$D0lo,$D0lo + vpaddq $T1,$D0hi,$D0hi + vpmadd52luq $H1,$R1,$D2lo + vpaddq $T2,$D1lo,$D1lo + vpaddq $T3,$D1hi,$D1hi + vpmadd52huq $H1,$R1,$D2hi + vpaddq $T4,$D2lo,$D2lo + vpaddq $tmp,$D2hi,$D2hi + + ################################################################ + # partial reduction + vpsrlq \$44,$D0lo,$tmp + vpsllq \$8,$D0hi,$D0hi + vpandq $mask44,$D0lo,$H0 + vpaddq $tmp,$D0hi,$D0hi + + vpaddq $D0hi,$D1lo,$D1lo + + vpsrlq \$44,$D1lo,$tmp + vpsllq \$8,$D1hi,$D1hi + vpandq $mask44,$D1lo,$H1 + vpaddq $tmp,$D1hi,$D1hi + + vpaddq $D1hi,$D2lo,$D2lo + + vpsrlq \$42,$D2lo,$tmp + vpsllq \$10,$D2hi,$D2hi + vpandq .Lx_mask42(%rip),$D2lo,$H2 + vpaddq $tmp,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + vpsllq \$2,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + + vpsrlq \$44,$H0,$tmp # additional step + vpandq $mask44,$H0,$H0 + + vpaddq $tmp,$H1,$H1 + + dec %eax + jz .Ldone_init_vpmadd52 + + vpunpcklqdq $R1,$H1,$R1 # 1,2 + vpbroadcastq %x#$H1,%x#$H1 # 2,2 + vpunpcklqdq $R2,$H2,$R2 + vpbroadcastq %x#$H2,%x#$H2 + vpunpcklqdq $R0,$H0,$R0 + vpbroadcastq %x#$H0,%x#$H0 + + vpsllq \$2,$R1,$S1 # S1 = R1*5*4 + vpsllq \$2,$R2,$S2 # S2 = R2*5*4 + vpaddq $R1,$S1,$S1 + vpaddq $R2,$S2,$S2 + vpsllq \$2,$S1,$S1 + vpsllq \$2,$S2,$S2 + + jmp .Lmul_init_vpmadd52 + ud2 + +.align 32 +.Ldone_init_vpmadd52: + vinserti128 \$1,%x#$R1,$H1,$R1 # 1,2,3,4 + vinserti128 \$1,%x#$R2,$H2,$R2 + vinserti128 \$1,%x#$R0,$H0,$R0 + + vpermq \$0b11011000,$R1,$R1 # 1,3,2,4 + vpermq \$0b11011000,$R2,$R2 + vpermq \$0b11011000,$R0,$R0 + + vpsllq \$2,$R1,$S1 # S1 = R1*5*4 + vpaddq $R1,$S1,$S1 + vpsllq \$2,$S1,$S1 + + vmovq 0($ctx),%x#$H0 # load current hash value + vmovq 8($ctx),%x#$H1 + vmovq 16($ctx),%x#$H2 + + test \$3,$len # is length 4*n+2? + jnz .Ldone_init_vpmadd52_2x + + vmovdqu64 $R0,64($ctx) # save key powers + vpbroadcastq %x#$R0,$R0 # broadcast 4th power + vmovdqu64 $R1,96($ctx) + vpbroadcastq %x#$R1,$R1 + vmovdqu64 $R2,128($ctx) + vpbroadcastq %x#$R2,$R2 + vmovdqu64 $S1,160($ctx) + vpbroadcastq %x#$S1,$S1 + + jmp .Lblocks_vpmadd52_4x_key_loaded + ud2 + +.align 32 +.Ldone_init_vpmadd52_2x: + vmovdqu64 $R0,64($ctx) # save key powers + vpsrldq \$8,$R0,$R0 # 0-1-0-2 + vmovdqu64 $R1,96($ctx) + vpsrldq \$8,$R1,$R1 + vmovdqu64 $R2,128($ctx) + vpsrldq \$8,$R2,$R2 + vmovdqu64 $S1,160($ctx) + vpsrldq \$8,$S1,$S1 + jmp .Lblocks_vpmadd52_2x_key_loaded + ud2 + +.align 32 +.Lblocks_vpmadd52_2x_do: + vmovdqu64 128+8($ctx),${R2}{%k1}{z}# load 2nd and 1st key powers + vmovdqu64 160+8($ctx),${S1}{%k1}{z} + vmovdqu64 64+8($ctx),${R0}{%k1}{z} + vmovdqu64 96+8($ctx),${R1}{%k1}{z} + +.Lblocks_vpmadd52_2x_key_loaded: + vmovdqu64 16*0($inp),$T2 # load data + vpxorq $T3,$T3,$T3 + lea 16*2($inp),$inp + + vpunpcklqdq $T3,$T2,$T1 # transpose data + vpunpckhqdq $T3,$T2,$T3 + + # at this point 64-bit lanes are ordered as x-1-x-0 + + vpsrlq \$24,$T3,$T2 # splat the data + vporq $PAD,$T2,$T2 + vpaddq $T2,$H2,$H2 # accumulate input + vpandq $mask44,$T1,$T0 + vpsrlq \$44,$T1,$T1 + vpsllq \$20,$T3,$T3 + vporq $T3,$T1,$T1 + vpandq $mask44,$T1,$T1 + + jmp .Ltail_vpmadd52_2x + ud2 + +.align 32 +.Loop_vpmadd52_4x: + #vpaddq $T2,$H2,$H2 # accumulate input + vpaddq $T0,$H0,$H0 + vpaddq $T1,$H1,$H1 + + vpxorq $D0lo,$D0lo,$D0lo + vpxorq $D0hi,$D0hi,$D0hi + vpxorq $D1lo,$D1lo,$D1lo + vpxorq $D1hi,$D1hi,$D1hi + vpxorq $D2lo,$D2lo,$D2lo + vpxorq $D2hi,$D2hi,$D2hi + vpmadd52luq $H2,$S1,$D0lo + vpxorq $T0,$T0,$T0 + vpxorq $T1,$T1,$T1 + vpmadd52huq $H2,$S1,$D0hi + vpxorq $T2,$T2,$T2 + vpxorq $T3,$T3,$T3 + vpmadd52luq $H2,$S2,$D1lo + vpxorq $T4,$T4,$T4 + vpxorq $tmp,$tmp,$tmp + vpmadd52huq $H2,$S2,$D1hi + vpmadd52luq $H2,$R0,$D2lo + vpmadd52huq $H2,$R0,$D2hi + + vpmadd52luq $H0,$R0,$T0 + vpmadd52huq $H0,$R0,$T1 + vpmadd52luq $H0,$R1,$T2 + vpmadd52huq $H0,$R1,$T3 + vpmadd52luq $H0,$R2,$T4 + vpmadd52huq $H0,$R2,$tmp + + vpmadd52luq $H1,$S2,$D0lo + vpmadd52huq $H1,$S2,$D0hi + vpmadd52luq $H1,$R0,$D1lo + vpmadd52huq $H1,$R0,$D1hi + vpaddq $T0,$D0lo,$D0lo + vpaddq $T1,$D0hi,$D0hi + vpmadd52luq $H1,$R1,$D2lo + vpaddq $T2,$D1lo,$D1lo + vpaddq $T3,$D1hi,$D1hi + vpmadd52huq $H1,$R1,$D2hi + vpaddq $T4,$D2lo,$D2lo + vpaddq $tmp,$D2hi,$D2hi + + vmovdqu64 16*0($inp),$T2 # load data + vmovdqu64 16*2($inp),$T3 + lea 16*4($inp),$inp + vpunpcklqdq $T3,$T2,$T1 # transpose data + vpunpckhqdq $T3,$T2,$T3 + + ################################################################ + # partial reduction (interleaved with data splat) + vpsrlq \$44,$D0lo,$tmp + vpsllq \$8,$D0hi,$D0hi + vpandq $mask44,$D0lo,$H0 + vpaddq $tmp,$D0hi,$D0hi + + vpsrlq \$24,$T3,$T2 + vporq $PAD,$T2,$T2 + vpaddq $D0hi,$D1lo,$D1lo + + vpsrlq \$44,$D1lo,$tmp + vpsllq \$8,$D1hi,$D1hi + vpandq $mask44,$D1lo,$H1 + vpaddq $tmp,$D1hi,$D1hi + + vpandq $mask44,$T1,$T0 + vpsrlq \$44,$T1,$T1 + vpsllq \$20,$T3,$T3 + vpaddq $D1hi,$D2lo,$D2lo + + vpsrlq \$42,$D2lo,$tmp + vpsllq \$10,$D2hi,$D2hi + vpandq .Lx_mask42(%rip),$D2lo,$H2 + vpaddq $tmp,$D2hi,$D2hi + + vpaddq $T2,$H2,$H2 # accumulate input + vpaddq $D2hi,$H0,$H0 + vpsllq \$2,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + vporq $T3,$T1,$T1 + vpandq $mask44,$T1,$T1 + + vpsrlq \$44,$H0,$tmp # additional step + vpandq $mask44,$H0,$H0 + + vpaddq $tmp,$H1,$H1 + + sub \$4,$len # len-=64 + jnz .Loop_vpmadd52_4x + +.Ltail_vpmadd52_4x: + vmovdqu64 128($ctx),$R2 # load all key powers + vmovdqu64 160($ctx),$S1 + vmovdqu64 64($ctx),$R0 + vmovdqu64 96($ctx),$R1 + +.Ltail_vpmadd52_2x: + vpsllq \$2,$R2,$S2 # S2 = R2*5*4 + vpaddq $R2,$S2,$S2 + vpsllq \$2,$S2,$S2 + + #vpaddq $T2,$H2,$H2 # accumulate input + vpaddq $T0,$H0,$H0 + vpaddq $T1,$H1,$H1 + + vpxorq $D0lo,$D0lo,$D0lo + vpxorq $D0hi,$D0hi,$D0hi + vpxorq $D1lo,$D1lo,$D1lo + vpxorq $D1hi,$D1hi,$D1hi + vpxorq $D2lo,$D2lo,$D2lo + vpxorq $D2hi,$D2hi,$D2hi + vpmadd52luq $H2,$S1,$D0lo + vpxorq $T0,$T0,$T0 + vpxorq $T1,$T1,$T1 + vpmadd52huq $H2,$S1,$D0hi + vpxorq $T2,$T2,$T2 + vpxorq $T3,$T3,$T3 + vpmadd52luq $H2,$S2,$D1lo + vpxorq $T4,$T4,$T4 + vpxorq $tmp,$tmp,$tmp + vpmadd52huq $H2,$S2,$D1hi + vpmadd52luq $H2,$R0,$D2lo + vpmadd52huq $H2,$R0,$D2hi + + vpmadd52luq $H0,$R0,$T0 + vpmadd52huq $H0,$R0,$T1 + vpmadd52luq $H0,$R1,$T2 + vpmadd52huq $H0,$R1,$T3 + vpmadd52luq $H0,$R2,$T4 + vpmadd52huq $H0,$R2,$tmp + + vpmadd52luq $H1,$S2,$D0lo + vpmadd52huq $H1,$S2,$D0hi + vpmadd52luq $H1,$R0,$D1lo + vpmadd52huq $H1,$R0,$D1hi + vpaddq $T0,$D0lo,$D0lo + vpaddq $T1,$D0hi,$D0hi + vpmadd52luq $H1,$R1,$D2lo + vpaddq $T2,$D1lo,$D1lo + vpaddq $T3,$D1hi,$D1hi + vpmadd52huq $H1,$R1,$D2hi + vpaddq $T4,$D2lo,$D2lo + vpaddq $tmp,$D2hi,$D2hi + + ################################################################ + # horizontal addition + + mov \$1,%eax + kmovw %eax,%k1 + vpsrldq \$8,$D0lo,$T0 + vpsrldq \$8,$D0hi,$H0 + vpsrldq \$8,$D1lo,$T1 + vpsrldq \$8,$D1hi,$H1 + vpaddq $T0,$D0lo,$D0lo + vpaddq $H0,$D0hi,$D0hi + vpsrldq \$8,$D2lo,$T2 + vpsrldq \$8,$D2hi,$H2 + vpaddq $T1,$D1lo,$D1lo + vpaddq $H1,$D1hi,$D1hi + vpermq \$0x2,$D0lo,$T0 + vpermq \$0x2,$D0hi,$H0 + vpaddq $T2,$D2lo,$D2lo + vpaddq $H2,$D2hi,$D2hi + + vpermq \$0x2,$D1lo,$T1 + vpermq \$0x2,$D1hi,$H1 + vpaddq $T0,$D0lo,${D0lo}{%k1}{z} + vpaddq $H0,$D0hi,${D0hi}{%k1}{z} + vpermq \$0x2,$D2lo,$T2 + vpermq \$0x2,$D2hi,$H2 + vpaddq $T1,$D1lo,${D1lo}{%k1}{z} + vpaddq $H1,$D1hi,${D1hi}{%k1}{z} + vpaddq $T2,$D2lo,${D2lo}{%k1}{z} + vpaddq $H2,$D2hi,${D2hi}{%k1}{z} + + ################################################################ + # partial reduction + vpsrlq \$44,$D0lo,$tmp + vpsllq \$8,$D0hi,$D0hi + vpandq $mask44,$D0lo,$H0 + vpaddq $tmp,$D0hi,$D0hi + + vpaddq $D0hi,$D1lo,$D1lo + + vpsrlq \$44,$D1lo,$tmp + vpsllq \$8,$D1hi,$D1hi + vpandq $mask44,$D1lo,$H1 + vpaddq $tmp,$D1hi,$D1hi + + vpaddq $D1hi,$D2lo,$D2lo + + vpsrlq \$42,$D2lo,$tmp + vpsllq \$10,$D2hi,$D2hi + vpandq .Lx_mask42(%rip),$D2lo,$H2 + vpaddq $tmp,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + vpsllq \$2,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + + vpsrlq \$44,$H0,$tmp # additional step + vpandq $mask44,$H0,$H0 + + vpaddq $tmp,$H1,$H1 + # at this point $len is + # either 4*n+2 or 0... + sub \$2,$len # len-=32 + ja .Lblocks_vpmadd52_4x_do + + vmovq %x#$H0,0($ctx) + vmovq %x#$H1,8($ctx) + vmovq %x#$H2,16($ctx) + vzeroall + +.Lno_data_vpmadd52_4x: + ret +.size poly1305_blocks_vpmadd52_4x,.-poly1305_blocks_vpmadd52_4x +___ +} +if (0) { +######################################################################## +# As implied by its name 8x subroutine processes 8 blocks in parallel... +# This is intermediate version, as it's used only in cases when input +# length is either 8*n, 8*n+1 or 8*n+2... + +my ($H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2) = map("%ymm$_",(0..5,16,17)); +my ($D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi) = map("%ymm$_",(18..23)); +my ($T0,$T1,$T2,$T3,$mask44,$mask42,$tmp,$PAD) = map("%ymm$_",(24..31)); +my ($RR0,$RR1,$RR2,$SS1,$SS2) = map("%ymm$_",(6..10)); + +$code.=<<___; +.type poly1305_blocks_vpmadd52_8x,\@function,4 +.align 32 +poly1305_blocks_vpmadd52_8x: + shr \$4,$len + jz .Lno_data_vpmadd52_8x # too short + + shl \$40,$padbit + mov 64($ctx),%r8 # peek on power of the key + + vmovdqa64 .Lx_mask44(%rip),$mask44 + vmovdqa64 .Lx_mask42(%rip),$mask42 + + test %r8,%r8 # is power value impossible? + js .Linit_vpmadd52 # if it is, then init R[4] + + vmovq 0($ctx),%x#$H0 # load current hash value + vmovq 8($ctx),%x#$H1 + vmovq 16($ctx),%x#$H2 + +.Lblocks_vpmadd52_8x: + ################################################################ + # fist we calculate more key powers + + vmovdqu64 128($ctx),$R2 # load 1-3-2-4 powers + vmovdqu64 160($ctx),$S1 + vmovdqu64 64($ctx),$R0 + vmovdqu64 96($ctx),$R1 + + vpsllq \$2,$R2,$S2 # S2 = R2*5*4 + vpaddq $R2,$S2,$S2 + vpsllq \$2,$S2,$S2 + + vpbroadcastq %x#$R2,$RR2 # broadcast 4th power + vpbroadcastq %x#$R0,$RR0 + vpbroadcastq %x#$R1,$RR1 + + vpxorq $D0lo,$D0lo,$D0lo + vpmadd52luq $RR2,$S1,$D0lo + vpxorq $D0hi,$D0hi,$D0hi + vpmadd52huq $RR2,$S1,$D0hi + vpxorq $D1lo,$D1lo,$D1lo + vpmadd52luq $RR2,$S2,$D1lo + vpxorq $D1hi,$D1hi,$D1hi + vpmadd52huq $RR2,$S2,$D1hi + vpxorq $D2lo,$D2lo,$D2lo + vpmadd52luq $RR2,$R0,$D2lo + vpxorq $D2hi,$D2hi,$D2hi + vpmadd52huq $RR2,$R0,$D2hi + + vpmadd52luq $RR0,$R0,$D0lo + vpmadd52huq $RR0,$R0,$D0hi + vpmadd52luq $RR0,$R1,$D1lo + vpmadd52huq $RR0,$R1,$D1hi + vpmadd52luq $RR0,$R2,$D2lo + vpmadd52huq $RR0,$R2,$D2hi + + vpmadd52luq $RR1,$S2,$D0lo + vpmadd52huq $RR1,$S2,$D0hi + vpmadd52luq $RR1,$R0,$D1lo + vpmadd52huq $RR1,$R0,$D1hi + vpmadd52luq $RR1,$R1,$D2lo + vpmadd52huq $RR1,$R1,$D2hi + + ################################################################ + # partial reduction + vpsrlq \$44,$D0lo,$tmp + vpsllq \$8,$D0hi,$D0hi + vpandq $mask44,$D0lo,$RR0 + vpaddq $tmp,$D0hi,$D0hi + + vpaddq $D0hi,$D1lo,$D1lo + + vpsrlq \$44,$D1lo,$tmp + vpsllq \$8,$D1hi,$D1hi + vpandq $mask44,$D1lo,$RR1 + vpaddq $tmp,$D1hi,$D1hi + + vpaddq $D1hi,$D2lo,$D2lo + + vpsrlq \$42,$D2lo,$tmp + vpsllq \$10,$D2hi,$D2hi + vpandq $mask42,$D2lo,$RR2 + vpaddq $tmp,$D2hi,$D2hi + + vpaddq $D2hi,$RR0,$RR0 + vpsllq \$2,$D2hi,$D2hi + + vpaddq $D2hi,$RR0,$RR0 + + vpsrlq \$44,$RR0,$tmp # additional step + vpandq $mask44,$RR0,$RR0 + + vpaddq $tmp,$RR1,$RR1 + + ################################################################ + # At this point Rx holds 1324 powers, RRx - 5768, and the goal + # is 15263748, which reflects how data is loaded... + + vpunpcklqdq $R2,$RR2,$T2 # 3748 + vpunpckhqdq $R2,$RR2,$R2 # 1526 + vpunpcklqdq $R0,$RR0,$T0 + vpunpckhqdq $R0,$RR0,$R0 + vpunpcklqdq $R1,$RR1,$T1 + vpunpckhqdq $R1,$RR1,$R1 +___ +######## switch to %zmm +map(s/%y/%z/, $H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2); +map(s/%y/%z/, $D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi); +map(s/%y/%z/, $T0,$T1,$T2,$T3,$mask44,$mask42,$tmp,$PAD); +map(s/%y/%z/, $RR0,$RR1,$RR2,$SS1,$SS2); + +$code.=<<___; + vshufi64x2 \$0x44,$R2,$T2,$RR2 # 15263748 + vshufi64x2 \$0x44,$R0,$T0,$RR0 + vshufi64x2 \$0x44,$R1,$T1,$RR1 + + vmovdqu64 16*0($inp),$T2 # load data + vmovdqu64 16*4($inp),$T3 + lea 16*8($inp),$inp + + vpsllq \$2,$RR2,$SS2 # S2 = R2*5*4 + vpsllq \$2,$RR1,$SS1 # S1 = R1*5*4 + vpaddq $RR2,$SS2,$SS2 + vpaddq $RR1,$SS1,$SS1 + vpsllq \$2,$SS2,$SS2 + vpsllq \$2,$SS1,$SS1 + + vpbroadcastq $padbit,$PAD + vpbroadcastq %x#$mask44,$mask44 + vpbroadcastq %x#$mask42,$mask42 + + vpbroadcastq %x#$SS1,$S1 # broadcast 8th power + vpbroadcastq %x#$SS2,$S2 + vpbroadcastq %x#$RR0,$R0 + vpbroadcastq %x#$RR1,$R1 + vpbroadcastq %x#$RR2,$R2 + + vpunpcklqdq $T3,$T2,$T1 # transpose data + vpunpckhqdq $T3,$T2,$T3 + + # at this point 64-bit lanes are ordered as 73625140 + + vpsrlq \$24,$T3,$T2 # splat the data + vporq $PAD,$T2,$T2 + vpaddq $T2,$H2,$H2 # accumulate input + vpandq $mask44,$T1,$T0 + vpsrlq \$44,$T1,$T1 + vpsllq \$20,$T3,$T3 + vporq $T3,$T1,$T1 + vpandq $mask44,$T1,$T1 + + sub \$8,$len + jz .Ltail_vpmadd52_8x + jmp .Loop_vpmadd52_8x + +.align 32 +.Loop_vpmadd52_8x: + #vpaddq $T2,$H2,$H2 # accumulate input + vpaddq $T0,$H0,$H0 + vpaddq $T1,$H1,$H1 + + vpxorq $D0lo,$D0lo,$D0lo + vpmadd52luq $H2,$S1,$D0lo + vpxorq $D0hi,$D0hi,$D0hi + vpmadd52huq $H2,$S1,$D0hi + vpxorq $D1lo,$D1lo,$D1lo + vpmadd52luq $H2,$S2,$D1lo + vpxorq $D1hi,$D1hi,$D1hi + vpmadd52huq $H2,$S2,$D1hi + vpxorq $D2lo,$D2lo,$D2lo + vpmadd52luq $H2,$R0,$D2lo + vpxorq $D2hi,$D2hi,$D2hi + vpmadd52huq $H2,$R0,$D2hi + + vmovdqu64 16*0($inp),$T2 # load data + vmovdqu64 16*4($inp),$T3 + lea 16*8($inp),$inp + vpmadd52luq $H0,$R0,$D0lo + vpmadd52huq $H0,$R0,$D0hi + vpmadd52luq $H0,$R1,$D1lo + vpmadd52huq $H0,$R1,$D1hi + vpmadd52luq $H0,$R2,$D2lo + vpmadd52huq $H0,$R2,$D2hi + + vpunpcklqdq $T3,$T2,$T1 # transpose data + vpunpckhqdq $T3,$T2,$T3 + vpmadd52luq $H1,$S2,$D0lo + vpmadd52huq $H1,$S2,$D0hi + vpmadd52luq $H1,$R0,$D1lo + vpmadd52huq $H1,$R0,$D1hi + vpmadd52luq $H1,$R1,$D2lo + vpmadd52huq $H1,$R1,$D2hi + + ################################################################ + # partial reduction (interleaved with data splat) + vpsrlq \$44,$D0lo,$tmp + vpsllq \$8,$D0hi,$D0hi + vpandq $mask44,$D0lo,$H0 + vpaddq $tmp,$D0hi,$D0hi + + vpsrlq \$24,$T3,$T2 + vporq $PAD,$T2,$T2 + vpaddq $D0hi,$D1lo,$D1lo + + vpsrlq \$44,$D1lo,$tmp + vpsllq \$8,$D1hi,$D1hi + vpandq $mask44,$D1lo,$H1 + vpaddq $tmp,$D1hi,$D1hi + + vpandq $mask44,$T1,$T0 + vpsrlq \$44,$T1,$T1 + vpsllq \$20,$T3,$T3 + vpaddq $D1hi,$D2lo,$D2lo + + vpsrlq \$42,$D2lo,$tmp + vpsllq \$10,$D2hi,$D2hi + vpandq $mask42,$D2lo,$H2 + vpaddq $tmp,$D2hi,$D2hi + + vpaddq $T2,$H2,$H2 # accumulate input + vpaddq $D2hi,$H0,$H0 + vpsllq \$2,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + vporq $T3,$T1,$T1 + vpandq $mask44,$T1,$T1 + + vpsrlq \$44,$H0,$tmp # additional step + vpandq $mask44,$H0,$H0 + + vpaddq $tmp,$H1,$H1 + + sub \$8,$len # len-=128 + jnz .Loop_vpmadd52_8x + +.Ltail_vpmadd52_8x: + #vpaddq $T2,$H2,$H2 # accumulate input + vpaddq $T0,$H0,$H0 + vpaddq $T1,$H1,$H1 + + vpxorq $D0lo,$D0lo,$D0lo + vpmadd52luq $H2,$SS1,$D0lo + vpxorq $D0hi,$D0hi,$D0hi + vpmadd52huq $H2,$SS1,$D0hi + vpxorq $D1lo,$D1lo,$D1lo + vpmadd52luq $H2,$SS2,$D1lo + vpxorq $D1hi,$D1hi,$D1hi + vpmadd52huq $H2,$SS2,$D1hi + vpxorq $D2lo,$D2lo,$D2lo + vpmadd52luq $H2,$RR0,$D2lo + vpxorq $D2hi,$D2hi,$D2hi + vpmadd52huq $H2,$RR0,$D2hi + + vpmadd52luq $H0,$RR0,$D0lo + vpmadd52huq $H0,$RR0,$D0hi + vpmadd52luq $H0,$RR1,$D1lo + vpmadd52huq $H0,$RR1,$D1hi + vpmadd52luq $H0,$RR2,$D2lo + vpmadd52huq $H0,$RR2,$D2hi + + vpmadd52luq $H1,$SS2,$D0lo + vpmadd52huq $H1,$SS2,$D0hi + vpmadd52luq $H1,$RR0,$D1lo + vpmadd52huq $H1,$RR0,$D1hi + vpmadd52luq $H1,$RR1,$D2lo + vpmadd52huq $H1,$RR1,$D2hi + + ################################################################ + # horizontal addition + + mov \$1,%eax + kmovw %eax,%k1 + vpsrldq \$8,$D0lo,$T0 + vpsrldq \$8,$D0hi,$H0 + vpsrldq \$8,$D1lo,$T1 + vpsrldq \$8,$D1hi,$H1 + vpaddq $T0,$D0lo,$D0lo + vpaddq $H0,$D0hi,$D0hi + vpsrldq \$8,$D2lo,$T2 + vpsrldq \$8,$D2hi,$H2 + vpaddq $T1,$D1lo,$D1lo + vpaddq $H1,$D1hi,$D1hi + vpermq \$0x2,$D0lo,$T0 + vpermq \$0x2,$D0hi,$H0 + vpaddq $T2,$D2lo,$D2lo + vpaddq $H2,$D2hi,$D2hi + + vpermq \$0x2,$D1lo,$T1 + vpermq \$0x2,$D1hi,$H1 + vpaddq $T0,$D0lo,$D0lo + vpaddq $H0,$D0hi,$D0hi + vpermq \$0x2,$D2lo,$T2 + vpermq \$0x2,$D2hi,$H2 + vpaddq $T1,$D1lo,$D1lo + vpaddq $H1,$D1hi,$D1hi + vextracti64x4 \$1,$D0lo,%y#$T0 + vextracti64x4 \$1,$D0hi,%y#$H0 + vpaddq $T2,$D2lo,$D2lo + vpaddq $H2,$D2hi,$D2hi + + vextracti64x4 \$1,$D1lo,%y#$T1 + vextracti64x4 \$1,$D1hi,%y#$H1 + vextracti64x4 \$1,$D2lo,%y#$T2 + vextracti64x4 \$1,$D2hi,%y#$H2 +___ +######## switch back to %ymm +map(s/%z/%y/, $H0,$H1,$H2,$R0,$R1,$R2,$S1,$S2); +map(s/%z/%y/, $D0lo,$D0hi,$D1lo,$D1hi,$D2lo,$D2hi); +map(s/%z/%y/, $T0,$T1,$T2,$T3,$mask44,$mask42,$tmp,$PAD); + +$code.=<<___; + vpaddq $T0,$D0lo,${D0lo}{%k1}{z} + vpaddq $H0,$D0hi,${D0hi}{%k1}{z} + vpaddq $T1,$D1lo,${D1lo}{%k1}{z} + vpaddq $H1,$D1hi,${D1hi}{%k1}{z} + vpaddq $T2,$D2lo,${D2lo}{%k1}{z} + vpaddq $H2,$D2hi,${D2hi}{%k1}{z} + + ################################################################ + # partial reduction + vpsrlq \$44,$D0lo,$tmp + vpsllq \$8,$D0hi,$D0hi + vpandq $mask44,$D0lo,$H0 + vpaddq $tmp,$D0hi,$D0hi + + vpaddq $D0hi,$D1lo,$D1lo + + vpsrlq \$44,$D1lo,$tmp + vpsllq \$8,$D1hi,$D1hi + vpandq $mask44,$D1lo,$H1 + vpaddq $tmp,$D1hi,$D1hi + + vpaddq $D1hi,$D2lo,$D2lo + + vpsrlq \$42,$D2lo,$tmp + vpsllq \$10,$D2hi,$D2hi + vpandq $mask42,$D2lo,$H2 + vpaddq $tmp,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + vpsllq \$2,$D2hi,$D2hi + + vpaddq $D2hi,$H0,$H0 + + vpsrlq \$44,$H0,$tmp # additional step + vpandq $mask44,$H0,$H0 + + vpaddq $tmp,$H1,$H1 + + ################################################################ + + vmovq %x#$H0,0($ctx) + vmovq %x#$H1,8($ctx) + vmovq %x#$H2,16($ctx) + vzeroall + +.Lno_data_vpmadd52_8x: + ret +.size poly1305_blocks_vpmadd52_8x,.-poly1305_blocks_vpmadd52_8x +___ +} +$code.=<<___; +.type poly1305_emit_base2_44,\@function,3 +.align 32 +poly1305_emit_base2_44: + mov 0($ctx),%r8 # load hash value + mov 8($ctx),%r9 + mov 16($ctx),%r10 + + mov %r9,%rax # base 2^44 -> base 2^64 + shr \$20,%r9 + shl \$44,%rax + mov %r10,%rcx + shr \$40,%r10 + shl \$24,%rcx + + add %rax,%r8 + adc %rcx,%r9 + adc \$0,%r10 + + mov %r8,%rax + add \$5,%r8 # compare to modulus + mov %r9,%rcx + adc \$0,%r9 + adc \$0,%r10 + shr \$2,%r10 # did 130-bit value overflow? + cmovnz %r8,%rax + cmovnz %r9,%rcx + + add 0($nonce),%rax # accumulate nonce + adc 8($nonce),%rcx + mov %rax,0($mac) # write result + mov %rcx,8($mac) + + ret +.size poly1305_emit_base2_44,.-poly1305_emit_base2_44 +___ +} } +$code.=<<___; +.align 64 +.Lconst: +.Lmask24: +.long 0x0ffffff,0,0x0ffffff,0,0x0ffffff,0,0x0ffffff,0 +.L129: +.long `1<<24`,0,`1<<24`,0,`1<<24`,0,`1<<24`,0 +.Lmask26: +.long 0x3ffffff,0,0x3ffffff,0,0x3ffffff,0,0x3ffffff,0 +.Lpermd_avx2: +.long 2,2,2,3,2,0,2,1 +.Lpermd_avx512: +.long 0,0,0,1, 0,2,0,3, 0,4,0,5, 0,6,0,7 + +.L2_44_inp_permd: +.long 0,1,1,2,2,3,7,7 +.L2_44_inp_shift: +.quad 0,12,24,64 +.L2_44_mask: +.quad 0xfffffffffff,0xfffffffffff,0x3ffffffffff,0xffffffffffffffff +.L2_44_shift_rgt: +.quad 44,44,42,64 +.L2_44_shift_lft: +.quad 8,8,10,64 + +.align 64 +.Lx_mask44: +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.quad 0xfffffffffff,0xfffffffffff,0xfffffffffff,0xfffffffffff +.Lx_mask42: +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +.quad 0x3ffffffffff,0x3ffffffffff,0x3ffffffffff,0x3ffffffffff +___ +} +$code.=<<___; +.asciz "Poly1305 for x86_64, CRYPTOGAMS by \@dot-asm" +.align 16 +___ + +{ # chacha20-poly1305 helpers +my ($out,$inp,$otp,$len)=$win64 ? ("%rcx","%rdx","%r8", "%r9") : # Win64 order + ("%rdi","%rsi","%rdx","%rcx"); # Unix order +$code.=<<___; +.globl xor128_encrypt_n_pad +.type xor128_encrypt_n_pad,\@abi-omnipotent +.align 16 +xor128_encrypt_n_pad: + sub $otp,$inp + sub $otp,$out + mov $len,%r10 # put len aside + shr \$4,$len # len / 16 + jz .Ltail_enc + nop +.Loop_enc_xmm: + movdqu ($inp,$otp),%xmm0 + pxor ($otp),%xmm0 + movdqu %xmm0,($out,$otp) + movdqa %xmm0,($otp) + lea 16($otp),$otp + dec $len + jnz .Loop_enc_xmm + + and \$15,%r10 # len % 16 + jz .Ldone_enc + +.Ltail_enc: + mov \$16,$len + sub %r10,$len + xor %eax,%eax +.Loop_enc_byte: + mov ($inp,$otp),%al + xor ($otp),%al + mov %al,($out,$otp) + mov %al,($otp) + lea 1($otp),$otp + dec %r10 + jnz .Loop_enc_byte + + xor %eax,%eax +.Loop_enc_pad: + mov %al,($otp) + lea 1($otp),$otp + dec $len + jnz .Loop_enc_pad + +.Ldone_enc: + mov $otp,%rax + ret +.size xor128_encrypt_n_pad,.-xor128_encrypt_n_pad + +.globl xor128_decrypt_n_pad +.type xor128_decrypt_n_pad,\@abi-omnipotent +.align 16 +xor128_decrypt_n_pad: + sub $otp,$inp + sub $otp,$out + mov $len,%r10 # put len aside + shr \$4,$len # len / 16 + jz .Ltail_dec + nop +.Loop_dec_xmm: + movdqu ($inp,$otp),%xmm0 + movdqa ($otp),%xmm1 + pxor %xmm0,%xmm1 + movdqu %xmm1,($out,$otp) + movdqa %xmm0,($otp) + lea 16($otp),$otp + dec $len + jnz .Loop_dec_xmm + + pxor %xmm1,%xmm1 + and \$15,%r10 # len % 16 + jz .Ldone_dec + +.Ltail_dec: + mov \$16,$len + sub %r10,$len + xor %eax,%eax + xor %r11,%r11 +.Loop_dec_byte: + mov ($inp,$otp),%r11b + mov ($otp),%al + xor %r11b,%al + mov %al,($out,$otp) + mov %r11b,($otp) + lea 1($otp),$otp + dec %r10 + jnz .Loop_dec_byte + + xor %eax,%eax +.Loop_dec_pad: + mov %al,($otp) + lea 1($otp),$otp + dec $len + jnz .Loop_dec_pad + +.Ldone_dec: + mov $otp,%rax + ret +.size xor128_decrypt_n_pad,.-xor128_decrypt_n_pad +___ +} + +# EXCEPTION_DISPOSITION handler (EXCEPTION_RECORD *rec,ULONG64 frame, +# CONTEXT *context,DISPATCHER_CONTEXT *disp) +if ($win64) { +$rec="%rcx"; +$frame="%rdx"; +$context="%r8"; +$disp="%r9"; + +$code.=<<___; +.extern __imp_RtlVirtualUnwind +.type se_handler,\@abi-omnipotent +.align 16 +se_handler: + push %rsi + push %rdi + push %rbx + push %rbp + push %r12 + push %r13 + push %r14 + push %r15 + pushfq + sub \$64,%rsp + + mov 120($context),%rax # pull context->Rax + mov 248($context),%rbx # pull context->Rip + + mov 8($disp),%rsi # disp->ImageBase + mov 56($disp),%r11 # disp->HandlerData + + mov 0(%r11),%r10d # HandlerData[0] + lea (%rsi,%r10),%r10 # prologue label + cmp %r10,%rbx # context->Rip<.Lprologue + jb .Lcommon_seh_tail + + mov 152($context),%rax # pull context->Rsp + + mov 4(%r11),%r10d # HandlerData[1] + lea (%rsi,%r10),%r10 # epilogue label + cmp %r10,%rbx # context->Rip>=.Lepilogue + jae .Lcommon_seh_tail + + lea 56(%rax),%rax + + mov -8(%rax),%rbx + mov -16(%rax),%rbp + mov -24(%rax),%r12 + mov -32(%rax),%r13 + mov -40(%rax),%r14 + mov -48(%rax),%r15 + mov %rbx,144($context) # restore context->Rbx + mov %rbp,160($context) # restore context->Rbp + mov %r12,216($context) # restore context->R12 + mov %r13,224($context) # restore context->R13 + mov %r14,232($context) # restore context->R14 + mov %r15,240($context) # restore context->R14 + + jmp .Lcommon_seh_tail +.size se_handler,.-se_handler + +.type avx_handler,\@abi-omnipotent +.align 16 +avx_handler: + push %rsi + push %rdi + push %rbx + push %rbp + push %r12 + push %r13 + push %r14 + push %r15 + pushfq + sub \$64,%rsp + + mov 120($context),%rax # pull context->Rax + mov 248($context),%rbx # pull context->Rip + + mov 8($disp),%rsi # disp->ImageBase + mov 56($disp),%r11 # disp->HandlerData + + mov 0(%r11),%r10d # HandlerData[0] + lea (%rsi,%r10),%r10 # prologue label + cmp %r10,%rbx # context->RipRsp + + mov 4(%r11),%r10d # HandlerData[1] + lea (%rsi,%r10),%r10 # epilogue label + cmp %r10,%rbx # context->Rip>=epilogue label + jae .Lcommon_seh_tail + + mov 208($context),%rax # pull context->R11 + + lea 0x50(%rax),%rsi + lea 0xf8(%rax),%rax + lea 512($context),%rdi # &context.Xmm6 + mov \$20,%ecx + .long 0xa548f3fc # cld; rep movsq + +.Lcommon_seh_tail: + mov 8(%rax),%rdi + mov 16(%rax),%rsi + mov %rax,152($context) # restore context->Rsp + mov %rsi,168($context) # restore context->Rsi + mov %rdi,176($context) # restore context->Rdi + + mov 40($disp),%rdi # disp->ContextRecord + mov $context,%rsi # context + mov \$154,%ecx # sizeof(CONTEXT) + .long 0xa548f3fc # cld; rep movsq + + mov $disp,%rsi + xor %rcx,%rcx # arg1, UNW_FLAG_NHANDLER + mov 8(%rsi),%rdx # arg2, disp->ImageBase + mov 0(%rsi),%r8 # arg3, disp->ControlPc + mov 16(%rsi),%r9 # arg4, disp->FunctionEntry + mov 40(%rsi),%r10 # disp->ContextRecord + lea 56(%rsi),%r11 # &disp->HandlerData + lea 24(%rsi),%r12 # &disp->EstablisherFrame + mov %r10,32(%rsp) # arg5 + mov %r11,40(%rsp) # arg6 + mov %r12,48(%rsp) # arg7 + mov %rcx,56(%rsp) # arg8, (NULL) + call *__imp_RtlVirtualUnwind(%rip) + + mov \$1,%eax # ExceptionContinueSearch + add \$64,%rsp + popfq + pop %r15 + pop %r14 + pop %r13 + pop %r12 + pop %rbp + pop %rbx + pop %rdi + pop %rsi + ret +.size avx_handler,.-avx_handler + +.section .pdata +.align 4 + .rva .LSEH_begin_poly1305_init + .rva .LSEH_end_poly1305_init + .rva .LSEH_info_poly1305_init + + .rva .LSEH_begin_poly1305_blocks + .rva .LSEH_end_poly1305_blocks + .rva .LSEH_info_poly1305_blocks + + .rva .LSEH_begin_poly1305_emit + .rva .LSEH_end_poly1305_emit + .rva .LSEH_info_poly1305_emit +___ +$code.=<<___ if ($avx); + .rva .LSEH_begin_poly1305_blocks_avx + .rva .Lbase2_64_avx + .rva .LSEH_info_poly1305_blocks_avx_1 + + .rva .Lbase2_64_avx + .rva .Leven_avx + .rva .LSEH_info_poly1305_blocks_avx_2 + + .rva .Leven_avx + .rva .LSEH_end_poly1305_blocks_avx + .rva .LSEH_info_poly1305_blocks_avx_3 +___ +$code.=<<___ if ($avx>1); + .rva .LSEH_begin_poly1305_blocks_avx2 + .rva .Lbase2_64_avx2 + .rva .LSEH_info_poly1305_blocks_avx2_1 + + .rva .Lbase2_64_avx2 + .rva .Leven_avx2 + .rva .LSEH_info_poly1305_blocks_avx2_2 + + .rva .Leven_avx2 + .rva .LSEH_end_poly1305_blocks_avx2 + .rva .LSEH_info_poly1305_blocks_avx2_3 +___ +$code.=<<___ if ($avx>2); + .rva .LSEH_begin_poly1305_blocks_avx512 + .rva .LSEH_end_poly1305_blocks_avx512 + .rva .LSEH_info_poly1305_blocks_avx512 +___ +$code.=<<___ if ($avx>3); + .rva .LSEH_begin_poly1305_init_base2_44 + .rva .LSEH_end_poly1305_init_base2_44 + .rva .LSEH_info_poly1305_init_base2_44 + + .rva .LSEH_begin_poly1305_blocks_base2_44 + .rva .LSEH_end_poly1305_blocks_base2_44 + .rva .LSEH_info_poly1305_blocks_base2_44 + + .rva .LSEH_begin_poly1305_blocks_vpmadd52 + .rva .LSEH_end_poly1305_blocks_vpmadd52 + .rva .LSEH_info_poly1305_blocks_vpmadd52 + + .rva .LSEH_begin_poly1305_blocks_vpmadd52_4x + .rva .LSEH_end_poly1305_blocks_vpmadd52_4x + .rva .LSEH_info_poly1305_blocks_vpmadd52_4x + + .rva .LSEH_begin_poly1305_emit_base2_44 + .rva .LSEH_end_poly1305_emit_base2_44 + .rva .LSEH_info_poly1305_emit_base2_44 +___ +$code.=<<___; +.section .xdata +.align 8 +.LSEH_info_poly1305_init: + .byte 9,0,0,0 + .rva se_handler + .long 0,0 # 0,0 means "no stack frame allocated" + +.LSEH_info_poly1305_blocks: + .byte 9,0,0,0 + .rva se_handler + .rva .Lblocks_body,.Lblocks_epilogue + +.LSEH_info_poly1305_emit: + .byte 9,0,0,0 + .rva se_handler + .long 0,0 +___ +$code.=<<___ if ($avx); +.LSEH_info_poly1305_blocks_avx_1: + .byte 9,0,0,0 + .rva se_handler + .rva .Lblocks_avx_body,.Lblocks_avx_epilogue # HandlerData[] + +.LSEH_info_poly1305_blocks_avx_2: + .byte 9,0,0,0 + .rva se_handler + .rva .Lbase2_64_avx_body,.Lbase2_64_avx_epilogue # HandlerData[] + +.LSEH_info_poly1305_blocks_avx_3: + .byte 9,0,0,0 + .rva avx_handler + .rva .Ldo_avx_body,.Ldo_avx_epilogue # HandlerData[] +___ +$code.=<<___ if ($avx>1); +.LSEH_info_poly1305_blocks_avx2_1: + .byte 9,0,0,0 + .rva se_handler + .rva .Lblocks_avx2_body,.Lblocks_avx2_epilogue # HandlerData[] + +.LSEH_info_poly1305_blocks_avx2_2: + .byte 9,0,0,0 + .rva se_handler + .rva .Lbase2_64_avx2_body,.Lbase2_64_avx2_epilogue # HandlerData[] + +.LSEH_info_poly1305_blocks_avx2_3: + .byte 9,0,0,0 + .rva avx_handler + .rva .Ldo_avx2_body,.Ldo_avx2_epilogue # HandlerData[] +___ +$code.=<<___ if ($avx>2); +.LSEH_info_poly1305_blocks_avx512: + .byte 9,0,0,0 + .rva avx_handler + .rva .Ldo_avx512_body,.Ldo_avx512_epilogue # HandlerData[] +___ +$code.=<<___ if ($avx>3); +.LSEH_info_poly1305_init_base2_44: + .byte 9,0,0,0 + .rva se_handler + .long 0,0 + +.LSEH_info_poly1305_blocks_base2_44: + .byte 9,0,0,0 + .rva se_handler + .rva .Lblocks_base2_44_body,.Lblocks_base2_44_epilogue + +.LSEH_info_poly1305_blocks_vpmadd52: + .byte 9,0,0,0 + .rva se_handler + .long 0,0 + +.LSEH_info_poly1305_blocks_vpmadd52_4x: + .byte 9,0,0,0 + .rva se_handler + .long 0,0 + +.LSEH_info_poly1305_emit_base2_44: + .byte 9,0,0,0 + .rva se_handler + .long 0,0 +___ +} + +foreach (split('\n',$code)) { + s/\`([^\`]*)\`/eval($1)/ge; + s/%r([a-z]+)#d/%e$1/g; + s/%r([0-9]+)#d/%r$1d/g; + s/%x#%[yz]/%x/g or s/%y#%z/%y/g or s/%z#%[yz]/%z/g; + + print $_,"\n"; +} +close STDOUT; diff --git a/cbits/asm/sha1-armv8-ios64.S b/cbits/asm/sha1-armv8-ios64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha1-armv8-ios64.S @@ -0,0 +1,1216 @@ +#ifndef __KERNEL__ +# include "arm_arch.h" + +#else +.globl _crypton_sha1_asm_block_armv8 +#endif + +.text + +.globl _crypton_sha1_asm_block_data_order + +.align 6 +_crypton_sha1_asm_block_data_order: + adrp x16,_crypton_armcap_P@PAGE + ldr w16,[x16,_crypton_armcap_P@PAGEOFF] + tst w16,#ARMV8_SHA1 + b.ne Lv8_entry + + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + + ldp w20,w21,[x0] + ldp w22,w23,[x0,#8] + ldr w24,[x0,#16] + +Loop: + ldr x3,[x1],#64 + movz w28,#0x7999 + sub x2,x2,#1 + movk w28,#0x5a82,lsl#16 +#ifdef __AARCH64EB__ + ror x3,x3,#32 +#else + rev32 x3,x3 +#endif + add w24,w24,w28 // warm it up + add w24,w24,w3 + lsr x4,x3,#32 + ldur x5,[x1,#-56] + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + add w23,w23,w4 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x5,x5,#32 +#else + rev32 x5,x5 +#endif + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + add w22,w22,w5 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + lsr x6,x5,#32 + ldur x7,[x1,#-48] + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + add w21,w21,w6 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x7,x7,#32 +#else + rev32 x7,x7 +#endif + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w7 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + lsr x8,x7,#32 + ldur x9,[x1,#-40] + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + add w24,w24,w8 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x9,x9,#32 +#else + rev32 x9,x9 +#endif + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + add w23,w23,w9 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + lsr x10,x9,#32 + ldur x11,[x1,#-32] + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + add w22,w22,w10 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x11,x11,#32 +#else + rev32 x11,x11 +#endif + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + add w21,w21,w11 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + lsr x12,x11,#32 + ldur x13,[x1,#-24] + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w12 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x13,x13,#32 +#else + rev32 x13,x13 +#endif + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + add w24,w24,w13 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + lsr x14,x13,#32 + ldur x15,[x1,#-16] + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + add w23,w23,w14 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x15,x15,#32 +#else + rev32 x15,x15 +#endif + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + add w22,w22,w15 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + lsr x16,x15,#32 + ldur x17,[x1,#-8] + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + add w21,w21,w16 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x17,x17,#32 +#else + rev32 x17,x17 +#endif + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w17 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + lsr x19,x17,#32 + eor w3,w3,w5 + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + eor w3,w3,w11 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + eor w3,w3,w16 + ror w22,w22,#2 + add w24,w24,w19 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w3,w3,#31 + eor w4,w4,w6 + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + eor w4,w4,w12 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + eor w4,w4,w17 + ror w21,w21,#2 + add w23,w23,w3 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w4,w4,#31 + eor w5,w5,w7 + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + eor w5,w5,w13 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + eor w5,w5,w19 + ror w20,w20,#2 + add w22,w22,w4 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w5,w5,#31 + eor w6,w6,w8 + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + eor w6,w6,w14 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + eor w6,w6,w3 + ror w24,w24,#2 + add w21,w21,w5 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w6,w6,#31 + eor w7,w7,w9 + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + eor w7,w7,w15 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + eor w7,w7,w4 + ror w23,w23,#2 + add w20,w20,w6 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w7,w7,#31 + movz w28,#0xeba1 + movk w28,#0x6ed9,lsl#16 + eor w8,w8,w10 + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + eor w8,w8,w16 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + eor w8,w8,w5 + ror w22,w22,#2 + add w24,w24,w7 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w8,w8,#31 + eor w9,w9,w11 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w9,w9,w17 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w9,w9,w6 + add w23,w23,w8 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w9,w9,#31 + eor w10,w10,w12 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w10,w10,w19 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w10,w10,w7 + add w22,w22,w9 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w10,w10,#31 + eor w11,w11,w13 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w11,w11,w3 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w11,w11,w8 + add w21,w21,w10 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w11,w11,#31 + eor w12,w12,w14 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w12,w12,w4 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w12,w12,w9 + add w20,w20,w11 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w12,w12,#31 + eor w13,w13,w15 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w13,w13,w5 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w13,w13,w10 + add w24,w24,w12 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w13,w13,#31 + eor w14,w14,w16 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w14,w14,w6 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w14,w14,w11 + add w23,w23,w13 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w14,w14,#31 + eor w15,w15,w17 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w15,w15,w7 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w15,w15,w12 + add w22,w22,w14 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w15,w15,#31 + eor w16,w16,w19 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w16,w16,w8 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w16,w16,w13 + add w21,w21,w15 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w16,w16,#31 + eor w17,w17,w3 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w17,w17,w9 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w17,w17,w14 + add w20,w20,w16 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w17,w17,#31 + eor w19,w19,w4 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w19,w19,w10 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w19,w19,w15 + add w24,w24,w17 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w19,w19,#31 + eor w3,w3,w5 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w3,w3,w11 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w3,w3,w16 + add w23,w23,w19 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w3,w3,#31 + eor w4,w4,w6 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w4,w4,w12 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w4,w4,w17 + add w22,w22,w3 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w4,w4,#31 + eor w5,w5,w7 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w5,w5,w13 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w5,w5,w19 + add w21,w21,w4 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w5,w5,#31 + eor w6,w6,w8 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w6,w6,w14 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w6,w6,w3 + add w20,w20,w5 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w6,w6,#31 + eor w7,w7,w9 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w7,w7,w15 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w7,w7,w4 + add w24,w24,w6 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w7,w7,#31 + eor w8,w8,w10 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w8,w8,w16 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w8,w8,w5 + add w23,w23,w7 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w8,w8,#31 + eor w9,w9,w11 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w9,w9,w17 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w9,w9,w6 + add w22,w22,w8 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w9,w9,#31 + eor w10,w10,w12 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w10,w10,w19 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w10,w10,w7 + add w21,w21,w9 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w10,w10,#31 + eor w11,w11,w13 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w11,w11,w3 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w11,w11,w8 + add w20,w20,w10 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w11,w11,#31 + movz w28,#0xbcdc + movk w28,#0x8f1b,lsl#16 + eor w12,w12,w14 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w12,w12,w4 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w12,w12,w9 + add w24,w24,w11 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w12,w12,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w13,w13,w15 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w13,w13,w5 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w13,w13,w10 + add w23,w23,w12 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w13,w13,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w14,w14,w16 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w14,w14,w6 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w14,w14,w11 + add w22,w22,w13 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w14,w14,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w15,w15,w17 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w15,w15,w7 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w15,w15,w12 + add w21,w21,w14 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w15,w15,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w16,w16,w19 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w16,w16,w8 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w16,w16,w13 + add w20,w20,w15 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w16,w16,#31 + orr w25,w22,w23 + and w26,w22,w23 + eor w17,w17,w3 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w17,w17,w9 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w17,w17,w14 + add w24,w24,w16 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w17,w17,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w19,w19,w4 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w19,w19,w10 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w19,w19,w15 + add w23,w23,w17 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w19,w19,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w3,w3,w5 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w3,w3,w11 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w3,w3,w16 + add w22,w22,w19 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w3,w3,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w4,w4,w6 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w4,w4,w12 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w4,w4,w17 + add w21,w21,w3 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w4,w4,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w5,w5,w7 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w5,w5,w13 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w5,w5,w19 + add w20,w20,w4 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w5,w5,#31 + orr w25,w22,w23 + and w26,w22,w23 + eor w6,w6,w8 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w6,w6,w14 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w6,w6,w3 + add w24,w24,w5 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w6,w6,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w7,w7,w9 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w7,w7,w15 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w7,w7,w4 + add w23,w23,w6 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w7,w7,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w8,w8,w10 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w8,w8,w16 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w8,w8,w5 + add w22,w22,w7 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w8,w8,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w9,w9,w11 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w9,w9,w17 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w9,w9,w6 + add w21,w21,w8 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w9,w9,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w10,w10,w12 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w10,w10,w19 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w10,w10,w7 + add w20,w20,w9 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w10,w10,#31 + orr w25,w22,w23 + and w26,w22,w23 + eor w11,w11,w13 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w11,w11,w3 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w11,w11,w8 + add w24,w24,w10 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w11,w11,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w12,w12,w14 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w12,w12,w4 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w12,w12,w9 + add w23,w23,w11 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w12,w12,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w13,w13,w15 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w13,w13,w5 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w13,w13,w10 + add w22,w22,w12 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w13,w13,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w14,w14,w16 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w14,w14,w6 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w14,w14,w11 + add w21,w21,w13 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w14,w14,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w15,w15,w17 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w15,w15,w7 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w15,w15,w12 + add w20,w20,w14 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w15,w15,#31 + movz w28,#0xc1d6 + movk w28,#0xca62,lsl#16 + orr w25,w22,w23 + and w26,w22,w23 + eor w16,w16,w19 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w16,w16,w8 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w16,w16,w13 + add w24,w24,w15 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w16,w16,#31 + eor w17,w17,w3 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w17,w17,w9 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w17,w17,w14 + add w23,w23,w16 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w17,w17,#31 + eor w19,w19,w4 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w19,w19,w10 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w19,w19,w15 + add w22,w22,w17 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w19,w19,#31 + eor w3,w3,w5 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w3,w3,w11 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w3,w3,w16 + add w21,w21,w19 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w3,w3,#31 + eor w4,w4,w6 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w4,w4,w12 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w4,w4,w17 + add w20,w20,w3 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w4,w4,#31 + eor w5,w5,w7 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w5,w5,w13 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w5,w5,w19 + add w24,w24,w4 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w5,w5,#31 + eor w6,w6,w8 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w6,w6,w14 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w6,w6,w3 + add w23,w23,w5 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w6,w6,#31 + eor w7,w7,w9 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w7,w7,w15 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w7,w7,w4 + add w22,w22,w6 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w7,w7,#31 + eor w8,w8,w10 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w8,w8,w16 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w8,w8,w5 + add w21,w21,w7 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w8,w8,#31 + eor w9,w9,w11 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w9,w9,w17 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w9,w9,w6 + add w20,w20,w8 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w9,w9,#31 + eor w10,w10,w12 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w10,w10,w19 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w10,w10,w7 + add w24,w24,w9 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w10,w10,#31 + eor w11,w11,w13 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w11,w11,w3 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w11,w11,w8 + add w23,w23,w10 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w11,w11,#31 + eor w12,w12,w14 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w12,w12,w4 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w12,w12,w9 + add w22,w22,w11 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w12,w12,#31 + eor w13,w13,w15 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w13,w13,w5 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w13,w13,w10 + add w21,w21,w12 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w13,w13,#31 + eor w14,w14,w16 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w14,w14,w6 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w14,w14,w11 + add w20,w20,w13 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w14,w14,#31 + eor w15,w15,w17 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w15,w15,w7 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w15,w15,w12 + add w24,w24,w14 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w15,w15,#31 + eor w16,w16,w19 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w16,w16,w8 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w16,w16,w13 + add w23,w23,w15 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w16,w16,#31 + eor w17,w17,w3 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w17,w17,w9 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w17,w17,w14 + add w22,w22,w16 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w17,w17,#31 + eor w19,w19,w4 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w19,w19,w10 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w19,w19,w15 + add w21,w21,w17 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w19,w19,#31 + ldp w4,w5,[x0] + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w19 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ldp w6,w7,[x0,#8] + eor w25,w24,w22 + ror w27,w21,#27 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + ldr w8,[x0,#16] + add w20,w20,w25 // e+=F(b,c,d) + add w21,w21,w5 + add w22,w22,w6 + add w20,w20,w4 + add w23,w23,w7 + add w24,w24,w8 + stp w20,w21,[x0] + stp w22,w23,[x0,#8] + str w24,[x0,#16] + cbnz x2,Loop + + ldp x19,x20,[sp,#2*__SIZEOF_POINTER__] + ldp x21,x22,[sp,#4*__SIZEOF_POINTER__] + ldp x23,x24,[sp,#6*__SIZEOF_POINTER__] + ldp x25,x26,[sp,#8*__SIZEOF_POINTER__] + ldp x27,x28,[sp,#10*__SIZEOF_POINTER__] + ldr x29,[sp],#12*__SIZEOF_POINTER__ + ret + + +.align 6 +_crypton_sha1_asm_block_armv8: +Lv8_entry: + stp x29,x30,[sp,#-16]! + add x29,sp,#0 + + adr x4,Lconst + eor v1.16b,v1.16b,v1.16b + ld1 {v0.4s},[x0],#16 + ld1 {v1.s}[0],[x0] + sub x0,x0,#16 + ld1 {v16.4s,v17.4s,v18.4s,v19.4s},[x4] + +Loop_hw: + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + sub x2,x2,#1 + rev32 v4.16b,v4.16b + rev32 v5.16b,v5.16b + + add v20.4s,v16.4s,v4.4s + rev32 v6.16b,v6.16b + orr v22.16b,v0.16b,v0.16b // offload + + add v21.4s,v16.4s,v5.4s + rev32 v7.16b,v7.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b +.long 0x5e140020 //sha1c v0.16b,v1.16b,v20.4s // 0 + add v20.4s,v16.4s,v6.4s +.long 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 1 +.long 0x5e150060 //sha1c v0.16b,v3.16b,v21.4s + add v21.4s,v16.4s,v7.4s +.long 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.long 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 2 +.long 0x5e140040 //sha1c v0.16b,v2.16b,v20.4s + add v20.4s,v16.4s,v4.4s +.long 0x5e281885 //sha1su1 v5.16b,v4.16b +.long 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 3 +.long 0x5e150060 //sha1c v0.16b,v3.16b,v21.4s + add v21.4s,v17.4s,v5.4s +.long 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.long 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 4 +.long 0x5e140040 //sha1c v0.16b,v2.16b,v20.4s + add v20.4s,v17.4s,v6.4s +.long 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.long 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 5 +.long 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v17.4s,v7.4s +.long 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.long 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 6 +.long 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + add v20.4s,v17.4s,v4.4s +.long 0x5e281885 //sha1su1 v5.16b,v4.16b +.long 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 7 +.long 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v17.4s,v5.4s +.long 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.long 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 8 +.long 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + add v20.4s,v18.4s,v6.4s +.long 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.long 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 9 +.long 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v18.4s,v7.4s +.long 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.long 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 10 +.long 0x5e142040 //sha1m v0.16b,v2.16b,v20.4s + add v20.4s,v18.4s,v4.4s +.long 0x5e281885 //sha1su1 v5.16b,v4.16b +.long 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 11 +.long 0x5e152060 //sha1m v0.16b,v3.16b,v21.4s + add v21.4s,v18.4s,v5.4s +.long 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.long 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 12 +.long 0x5e142040 //sha1m v0.16b,v2.16b,v20.4s + add v20.4s,v18.4s,v6.4s +.long 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.long 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 13 +.long 0x5e152060 //sha1m v0.16b,v3.16b,v21.4s + add v21.4s,v19.4s,v7.4s +.long 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.long 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 14 +.long 0x5e142040 //sha1m v0.16b,v2.16b,v20.4s + add v20.4s,v19.4s,v4.4s +.long 0x5e281885 //sha1su1 v5.16b,v4.16b +.long 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 15 +.long 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v19.4s,v5.4s +.long 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.long 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.long 0x5e280803 //sha1h v3.16b,v0.16b // 16 +.long 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + add v20.4s,v19.4s,v6.4s +.long 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.long 0x5e280802 //sha1h v2.16b,v0.16b // 17 +.long 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v19.4s,v7.4s + +.long 0x5e280803 //sha1h v3.16b,v0.16b // 18 +.long 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + +.long 0x5e280802 //sha1h v2.16b,v0.16b // 19 +.long 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + + add v1.4s,v1.4s,v2.4s + add v0.4s,v0.4s,v22.4s + + cbnz x2,Loop_hw + + st1 {v0.4s},[x0],#16 + st1 {v1.s}[0],[x0] + + ldr x29,[sp],#16 + ret + +.align 6 +Lconst: +.long 0x5a827999,0x5a827999,0x5a827999,0x5a827999 //K_00_19 +.long 0x6ed9eba1,0x6ed9eba1,0x6ed9eba1,0x6ed9eba1 //K_20_39 +.long 0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc //K_40_59 +.long 0xca62c1d6,0xca62c1d6,0xca62c1d6,0xca62c1d6 //K_60_79 +.byte 83,72,65,49,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 +.align 2 +#if !defined(__KERNELL__) && !defined(_WIN64) +.comm __crypton_armcap_P,4 +.private_extern _crypton_armcap_P +#endif diff --git a/cbits/asm/sha1-armv8-linux64.S b/cbits/asm/sha1-armv8-linux64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha1-armv8-linux64.S @@ -0,0 +1,1218 @@ +#ifndef __KERNEL__ +# include "arm_arch.h" + +#else +.globl crypton_sha1_asm_block_armv8 +#endif + +.text + +.globl crypton_sha1_asm_block_data_order +.type crypton_sha1_asm_block_data_order,%function +.align 6 +crypton_sha1_asm_block_data_order: + adrp x16,crypton_armcap_P + ldr w16,[x16,#:lo12:crypton_armcap_P] + tst w16,#ARMV8_SHA1 + b.ne .Lv8_entry + + stp x29,x30,[sp,#-12*__SIZEOF_POINTER__]! + add x29,sp,#0 + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + + ldp w20,w21,[x0] + ldp w22,w23,[x0,#8] + ldr w24,[x0,#16] + +.Loop: + ldr x3,[x1],#64 + movz w28,#0x7999 + sub x2,x2,#1 + movk w28,#0x5a82,lsl#16 +#ifdef __AARCH64EB__ + ror x3,x3,#32 +#else + rev32 x3,x3 +#endif + add w24,w24,w28 // warm it up + add w24,w24,w3 + lsr x4,x3,#32 + ldur x5,[x1,#-56] + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + add w23,w23,w4 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x5,x5,#32 +#else + rev32 x5,x5 +#endif + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + add w22,w22,w5 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + lsr x6,x5,#32 + ldur x7,[x1,#-48] + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + add w21,w21,w6 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x7,x7,#32 +#else + rev32 x7,x7 +#endif + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w7 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + lsr x8,x7,#32 + ldur x9,[x1,#-40] + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + add w24,w24,w8 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x9,x9,#32 +#else + rev32 x9,x9 +#endif + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + add w23,w23,w9 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + lsr x10,x9,#32 + ldur x11,[x1,#-32] + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + add w22,w22,w10 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x11,x11,#32 +#else + rev32 x11,x11 +#endif + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + add w21,w21,w11 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + lsr x12,x11,#32 + ldur x13,[x1,#-24] + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w12 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x13,x13,#32 +#else + rev32 x13,x13 +#endif + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + add w24,w24,w13 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + lsr x14,x13,#32 + ldur x15,[x1,#-16] + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + add w23,w23,w14 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x15,x15,#32 +#else + rev32 x15,x15 +#endif + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + add w22,w22,w15 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + lsr x16,x15,#32 + ldur x17,[x1,#-8] + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + add w21,w21,w16 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) +#ifdef __AARCH64EB__ + ror x17,x17,#32 +#else + rev32 x17,x17 +#endif + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w17 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + lsr x19,x17,#32 + eor w3,w3,w5 + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + eor w3,w3,w11 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + eor w3,w3,w16 + ror w22,w22,#2 + add w24,w24,w19 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w3,w3,#31 + eor w4,w4,w6 + bic w25,w23,w21 + and w26,w22,w21 + ror w27,w20,#27 + eor w4,w4,w12 + add w23,w23,w28 // future e+=K + orr w25,w25,w26 + add w24,w24,w27 // e+=rot(a,5) + eor w4,w4,w17 + ror w21,w21,#2 + add w23,w23,w3 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w4,w4,#31 + eor w5,w5,w7 + bic w25,w22,w20 + and w26,w21,w20 + ror w27,w24,#27 + eor w5,w5,w13 + add w22,w22,w28 // future e+=K + orr w25,w25,w26 + add w23,w23,w27 // e+=rot(a,5) + eor w5,w5,w19 + ror w20,w20,#2 + add w22,w22,w4 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w5,w5,#31 + eor w6,w6,w8 + bic w25,w21,w24 + and w26,w20,w24 + ror w27,w23,#27 + eor w6,w6,w14 + add w21,w21,w28 // future e+=K + orr w25,w25,w26 + add w22,w22,w27 // e+=rot(a,5) + eor w6,w6,w3 + ror w24,w24,#2 + add w21,w21,w5 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w6,w6,#31 + eor w7,w7,w9 + bic w25,w20,w23 + and w26,w24,w23 + ror w27,w22,#27 + eor w7,w7,w15 + add w20,w20,w28 // future e+=K + orr w25,w25,w26 + add w21,w21,w27 // e+=rot(a,5) + eor w7,w7,w4 + ror w23,w23,#2 + add w20,w20,w6 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w7,w7,#31 + movz w28,#0xeba1 + movk w28,#0x6ed9,lsl#16 + eor w8,w8,w10 + bic w25,w24,w22 + and w26,w23,w22 + ror w27,w21,#27 + eor w8,w8,w16 + add w24,w24,w28 // future e+=K + orr w25,w25,w26 + add w20,w20,w27 // e+=rot(a,5) + eor w8,w8,w5 + ror w22,w22,#2 + add w24,w24,w7 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w8,w8,#31 + eor w9,w9,w11 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w9,w9,w17 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w9,w9,w6 + add w23,w23,w8 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w9,w9,#31 + eor w10,w10,w12 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w10,w10,w19 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w10,w10,w7 + add w22,w22,w9 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w10,w10,#31 + eor w11,w11,w13 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w11,w11,w3 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w11,w11,w8 + add w21,w21,w10 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w11,w11,#31 + eor w12,w12,w14 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w12,w12,w4 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w12,w12,w9 + add w20,w20,w11 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w12,w12,#31 + eor w13,w13,w15 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w13,w13,w5 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w13,w13,w10 + add w24,w24,w12 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w13,w13,#31 + eor w14,w14,w16 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w14,w14,w6 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w14,w14,w11 + add w23,w23,w13 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w14,w14,#31 + eor w15,w15,w17 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w15,w15,w7 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w15,w15,w12 + add w22,w22,w14 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w15,w15,#31 + eor w16,w16,w19 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w16,w16,w8 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w16,w16,w13 + add w21,w21,w15 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w16,w16,#31 + eor w17,w17,w3 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w17,w17,w9 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w17,w17,w14 + add w20,w20,w16 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w17,w17,#31 + eor w19,w19,w4 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w19,w19,w10 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w19,w19,w15 + add w24,w24,w17 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w19,w19,#31 + eor w3,w3,w5 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w3,w3,w11 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w3,w3,w16 + add w23,w23,w19 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w3,w3,#31 + eor w4,w4,w6 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w4,w4,w12 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w4,w4,w17 + add w22,w22,w3 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w4,w4,#31 + eor w5,w5,w7 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w5,w5,w13 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w5,w5,w19 + add w21,w21,w4 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w5,w5,#31 + eor w6,w6,w8 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w6,w6,w14 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w6,w6,w3 + add w20,w20,w5 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w6,w6,#31 + eor w7,w7,w9 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w7,w7,w15 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w7,w7,w4 + add w24,w24,w6 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w7,w7,#31 + eor w8,w8,w10 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w8,w8,w16 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w8,w8,w5 + add w23,w23,w7 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w8,w8,#31 + eor w9,w9,w11 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w9,w9,w17 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w9,w9,w6 + add w22,w22,w8 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w9,w9,#31 + eor w10,w10,w12 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w10,w10,w19 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w10,w10,w7 + add w21,w21,w9 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w10,w10,#31 + eor w11,w11,w13 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w11,w11,w3 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w11,w11,w8 + add w20,w20,w10 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w11,w11,#31 + movz w28,#0xbcdc + movk w28,#0x8f1b,lsl#16 + eor w12,w12,w14 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w12,w12,w4 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w12,w12,w9 + add w24,w24,w11 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w12,w12,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w13,w13,w15 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w13,w13,w5 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w13,w13,w10 + add w23,w23,w12 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w13,w13,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w14,w14,w16 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w14,w14,w6 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w14,w14,w11 + add w22,w22,w13 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w14,w14,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w15,w15,w17 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w15,w15,w7 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w15,w15,w12 + add w21,w21,w14 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w15,w15,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w16,w16,w19 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w16,w16,w8 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w16,w16,w13 + add w20,w20,w15 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w16,w16,#31 + orr w25,w22,w23 + and w26,w22,w23 + eor w17,w17,w3 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w17,w17,w9 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w17,w17,w14 + add w24,w24,w16 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w17,w17,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w19,w19,w4 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w19,w19,w10 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w19,w19,w15 + add w23,w23,w17 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w19,w19,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w3,w3,w5 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w3,w3,w11 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w3,w3,w16 + add w22,w22,w19 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w3,w3,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w4,w4,w6 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w4,w4,w12 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w4,w4,w17 + add w21,w21,w3 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w4,w4,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w5,w5,w7 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w5,w5,w13 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w5,w5,w19 + add w20,w20,w4 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w5,w5,#31 + orr w25,w22,w23 + and w26,w22,w23 + eor w6,w6,w8 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w6,w6,w14 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w6,w6,w3 + add w24,w24,w5 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w6,w6,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w7,w7,w9 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w7,w7,w15 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w7,w7,w4 + add w23,w23,w6 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w7,w7,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w8,w8,w10 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w8,w8,w16 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w8,w8,w5 + add w22,w22,w7 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w8,w8,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w9,w9,w11 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w9,w9,w17 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w9,w9,w6 + add w21,w21,w8 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w9,w9,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w10,w10,w12 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w10,w10,w19 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w10,w10,w7 + add w20,w20,w9 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w10,w10,#31 + orr w25,w22,w23 + and w26,w22,w23 + eor w11,w11,w13 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w11,w11,w3 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w11,w11,w8 + add w24,w24,w10 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w11,w11,#31 + orr w25,w21,w22 + and w26,w21,w22 + eor w12,w12,w14 + ror w27,w20,#27 + and w25,w25,w23 + add w23,w23,w28 // future e+=K + eor w12,w12,w4 + add w24,w24,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w21,w21,#2 + eor w12,w12,w9 + add w23,w23,w11 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w12,w12,#31 + orr w25,w20,w21 + and w26,w20,w21 + eor w13,w13,w15 + ror w27,w24,#27 + and w25,w25,w22 + add w22,w22,w28 // future e+=K + eor w13,w13,w5 + add w23,w23,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w20,w20,#2 + eor w13,w13,w10 + add w22,w22,w12 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w13,w13,#31 + orr w25,w24,w20 + and w26,w24,w20 + eor w14,w14,w16 + ror w27,w23,#27 + and w25,w25,w21 + add w21,w21,w28 // future e+=K + eor w14,w14,w6 + add w22,w22,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w24,w24,#2 + eor w14,w14,w11 + add w21,w21,w13 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w14,w14,#31 + orr w25,w23,w24 + and w26,w23,w24 + eor w15,w15,w17 + ror w27,w22,#27 + and w25,w25,w20 + add w20,w20,w28 // future e+=K + eor w15,w15,w7 + add w21,w21,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w23,w23,#2 + eor w15,w15,w12 + add w20,w20,w14 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w15,w15,#31 + movz w28,#0xc1d6 + movk w28,#0xca62,lsl#16 + orr w25,w22,w23 + and w26,w22,w23 + eor w16,w16,w19 + ror w27,w21,#27 + and w25,w25,w24 + add w24,w24,w28 // future e+=K + eor w16,w16,w8 + add w20,w20,w27 // e+=rot(a,5) + orr w25,w25,w26 + ror w22,w22,#2 + eor w16,w16,w13 + add w24,w24,w15 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w16,w16,#31 + eor w17,w17,w3 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w17,w17,w9 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w17,w17,w14 + add w23,w23,w16 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w17,w17,#31 + eor w19,w19,w4 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w19,w19,w10 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w19,w19,w15 + add w22,w22,w17 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w19,w19,#31 + eor w3,w3,w5 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w3,w3,w11 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w3,w3,w16 + add w21,w21,w19 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w3,w3,#31 + eor w4,w4,w6 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w4,w4,w12 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w4,w4,w17 + add w20,w20,w3 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w4,w4,#31 + eor w5,w5,w7 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w5,w5,w13 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w5,w5,w19 + add w24,w24,w4 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w5,w5,#31 + eor w6,w6,w8 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w6,w6,w14 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w6,w6,w3 + add w23,w23,w5 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w6,w6,#31 + eor w7,w7,w9 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w7,w7,w15 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w7,w7,w4 + add w22,w22,w6 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w7,w7,#31 + eor w8,w8,w10 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w8,w8,w16 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w8,w8,w5 + add w21,w21,w7 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w8,w8,#31 + eor w9,w9,w11 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w9,w9,w17 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w9,w9,w6 + add w20,w20,w8 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w9,w9,#31 + eor w10,w10,w12 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w10,w10,w19 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w10,w10,w7 + add w24,w24,w9 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w10,w10,#31 + eor w11,w11,w13 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w11,w11,w3 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w11,w11,w8 + add w23,w23,w10 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w11,w11,#31 + eor w12,w12,w14 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w12,w12,w4 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w12,w12,w9 + add w22,w22,w11 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w12,w12,#31 + eor w13,w13,w15 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w13,w13,w5 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w13,w13,w10 + add w21,w21,w12 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w13,w13,#31 + eor w14,w14,w16 + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w14,w14,w6 + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + eor w14,w14,w11 + add w20,w20,w13 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ror w14,w14,#31 + eor w15,w15,w17 + eor w25,w24,w22 + ror w27,w21,#27 + add w24,w24,w28 // future e+=K + eor w15,w15,w7 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + eor w15,w15,w12 + add w24,w24,w14 // future e+=X[i] + add w20,w20,w25 // e+=F(b,c,d) + ror w15,w15,#31 + eor w16,w16,w19 + eor w25,w23,w21 + ror w27,w20,#27 + add w23,w23,w28 // future e+=K + eor w16,w16,w8 + eor w25,w25,w22 + add w24,w24,w27 // e+=rot(a,5) + ror w21,w21,#2 + eor w16,w16,w13 + add w23,w23,w15 // future e+=X[i] + add w24,w24,w25 // e+=F(b,c,d) + ror w16,w16,#31 + eor w17,w17,w3 + eor w25,w22,w20 + ror w27,w24,#27 + add w22,w22,w28 // future e+=K + eor w17,w17,w9 + eor w25,w25,w21 + add w23,w23,w27 // e+=rot(a,5) + ror w20,w20,#2 + eor w17,w17,w14 + add w22,w22,w16 // future e+=X[i] + add w23,w23,w25 // e+=F(b,c,d) + ror w17,w17,#31 + eor w19,w19,w4 + eor w25,w21,w24 + ror w27,w23,#27 + add w21,w21,w28 // future e+=K + eor w19,w19,w10 + eor w25,w25,w20 + add w22,w22,w27 // e+=rot(a,5) + ror w24,w24,#2 + eor w19,w19,w15 + add w21,w21,w17 // future e+=X[i] + add w22,w22,w25 // e+=F(b,c,d) + ror w19,w19,#31 + ldp w4,w5,[x0] + eor w25,w20,w23 + ror w27,w22,#27 + add w20,w20,w28 // future e+=K + eor w25,w25,w24 + add w21,w21,w27 // e+=rot(a,5) + ror w23,w23,#2 + add w20,w20,w19 // future e+=X[i] + add w21,w21,w25 // e+=F(b,c,d) + ldp w6,w7,[x0,#8] + eor w25,w24,w22 + ror w27,w21,#27 + eor w25,w25,w23 + add w20,w20,w27 // e+=rot(a,5) + ror w22,w22,#2 + ldr w8,[x0,#16] + add w20,w20,w25 // e+=F(b,c,d) + add w21,w21,w5 + add w22,w22,w6 + add w20,w20,w4 + add w23,w23,w7 + add w24,w24,w8 + stp w20,w21,[x0] + stp w22,w23,[x0,#8] + str w24,[x0,#16] + cbnz x2,.Loop + + ldp x19,x20,[sp,#2*__SIZEOF_POINTER__] + ldp x21,x22,[sp,#4*__SIZEOF_POINTER__] + ldp x23,x24,[sp,#6*__SIZEOF_POINTER__] + ldp x25,x26,[sp,#8*__SIZEOF_POINTER__] + ldp x27,x28,[sp,#10*__SIZEOF_POINTER__] + ldr x29,[sp],#12*__SIZEOF_POINTER__ + ret +.size crypton_sha1_asm_block_data_order,.-crypton_sha1_asm_block_data_order +.type crypton_sha1_asm_block_armv8,%function +.align 6 +crypton_sha1_asm_block_armv8: +.Lv8_entry: + stp x29,x30,[sp,#-16]! + add x29,sp,#0 + + adr x4,.Lconst + eor v1.16b,v1.16b,v1.16b + ld1 {v0.4s},[x0],#16 + ld1 {v1.s}[0],[x0] + sub x0,x0,#16 + ld1 {v16.4s,v17.4s,v18.4s,v19.4s},[x4] + +.Loop_hw: + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + sub x2,x2,#1 + rev32 v4.16b,v4.16b + rev32 v5.16b,v5.16b + + add v20.4s,v16.4s,v4.4s + rev32 v6.16b,v6.16b + orr v22.16b,v0.16b,v0.16b // offload + + add v21.4s,v16.4s,v5.4s + rev32 v7.16b,v7.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b +.inst 0x5e140020 //sha1c v0.16b,v1.16b,v20.4s // 0 + add v20.4s,v16.4s,v6.4s +.inst 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 1 +.inst 0x5e150060 //sha1c v0.16b,v3.16b,v21.4s + add v21.4s,v16.4s,v7.4s +.inst 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.inst 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 2 +.inst 0x5e140040 //sha1c v0.16b,v2.16b,v20.4s + add v20.4s,v16.4s,v4.4s +.inst 0x5e281885 //sha1su1 v5.16b,v4.16b +.inst 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 3 +.inst 0x5e150060 //sha1c v0.16b,v3.16b,v21.4s + add v21.4s,v17.4s,v5.4s +.inst 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.inst 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 4 +.inst 0x5e140040 //sha1c v0.16b,v2.16b,v20.4s + add v20.4s,v17.4s,v6.4s +.inst 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.inst 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 5 +.inst 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v17.4s,v7.4s +.inst 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.inst 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 6 +.inst 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + add v20.4s,v17.4s,v4.4s +.inst 0x5e281885 //sha1su1 v5.16b,v4.16b +.inst 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 7 +.inst 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v17.4s,v5.4s +.inst 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.inst 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 8 +.inst 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + add v20.4s,v18.4s,v6.4s +.inst 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.inst 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 9 +.inst 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v18.4s,v7.4s +.inst 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.inst 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 10 +.inst 0x5e142040 //sha1m v0.16b,v2.16b,v20.4s + add v20.4s,v18.4s,v4.4s +.inst 0x5e281885 //sha1su1 v5.16b,v4.16b +.inst 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 11 +.inst 0x5e152060 //sha1m v0.16b,v3.16b,v21.4s + add v21.4s,v18.4s,v5.4s +.inst 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.inst 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 12 +.inst 0x5e142040 //sha1m v0.16b,v2.16b,v20.4s + add v20.4s,v18.4s,v6.4s +.inst 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.inst 0x5e0630a4 //sha1su0 v4.16b,v5.16b,v6.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 13 +.inst 0x5e152060 //sha1m v0.16b,v3.16b,v21.4s + add v21.4s,v19.4s,v7.4s +.inst 0x5e2818e4 //sha1su1 v4.16b,v7.16b +.inst 0x5e0730c5 //sha1su0 v5.16b,v6.16b,v7.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 14 +.inst 0x5e142040 //sha1m v0.16b,v2.16b,v20.4s + add v20.4s,v19.4s,v4.4s +.inst 0x5e281885 //sha1su1 v5.16b,v4.16b +.inst 0x5e0430e6 //sha1su0 v6.16b,v7.16b,v4.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 15 +.inst 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v19.4s,v5.4s +.inst 0x5e2818a6 //sha1su1 v6.16b,v5.16b +.inst 0x5e053087 //sha1su0 v7.16b,v4.16b,v5.16b +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 16 +.inst 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + add v20.4s,v19.4s,v6.4s +.inst 0x5e2818c7 //sha1su1 v7.16b,v6.16b +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 17 +.inst 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + add v21.4s,v19.4s,v7.4s + +.inst 0x5e280803 //sha1h v3.16b,v0.16b // 18 +.inst 0x5e141040 //sha1p v0.16b,v2.16b,v20.4s + +.inst 0x5e280802 //sha1h v2.16b,v0.16b // 19 +.inst 0x5e151060 //sha1p v0.16b,v3.16b,v21.4s + + add v1.4s,v1.4s,v2.4s + add v0.4s,v0.4s,v22.4s + + cbnz x2,.Loop_hw + + st1 {v0.4s},[x0],#16 + st1 {v1.s}[0],[x0] + + ldr x29,[sp],#16 + ret +.size crypton_sha1_asm_block_armv8,.-crypton_sha1_asm_block_armv8 +.align 6 +.Lconst: +.long 0x5a827999,0x5a827999,0x5a827999,0x5a827999 //K_00_19 +.long 0x6ed9eba1,0x6ed9eba1,0x6ed9eba1,0x6ed9eba1 //K_20_39 +.long 0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc //K_40_59 +.long 0xca62c1d6,0xca62c1d6,0xca62c1d6,0xca62c1d6 //K_60_79 +.byte 83,72,65,49,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 +.align 2 +#if !defined(__KERNELL__) && !defined(_WIN64) +.comm crypton_armcap_P,4,4 +.hidden crypton_armcap_P +#endif + +.section .note.GNU-stack,"",%progbits diff --git a/cbits/asm/sha1-armv8.pl b/cbits/asm/sha1-armv8.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/sha1-armv8.pl @@ -0,0 +1,362 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# SHA1 for ARMv8. +# +# Performance in cycles per processed byte and improvement coefficient +# over code generated with "default" compiler: +# +# hardware-assisted software(*) +# Apple A7 2.31 4.13 (+14%) +# Apple A10 1.61 +# Apple A14/M1 1.32 3.82 (-13%)(***) +# Cortex-A53 2.24 8.03 (+97%) +# Cortex-A57 2.35 7.88 (+74%) +# Cortex-A76 1.64 5.20 +# Cortex-X2 1.63 4.07 +# Cortex-X925 1.64 3.81 +# Denver 2.13 3.97 (+0%)(**) +# X-Gene 8.80 (+200%) +# Mongoose 2.05 6.50 (+160%) +# Kryo 1.88 8.00 (+90%) +# ThunderX2 2.64 6.36 (+150%) +# Snapdraon X 1.48 3.82 +# +# (*) Software results are presented mostly for reference purposes. +# (**) Keep in mind that Denver relies on binary translation, which +# optimizes compiler output at run-time. +# (***) There is some room for improvement on "extra-wide" processor +# such as A14/M1. Nothing is done, because it's not used anyway. + +$flavour = shift; +$output = shift; + +if ($flavour && $flavour ne "void") { + $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; + ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or + ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or + die "can't locate arm-xlate.pl"; + + open STDOUT,"| \"$^X\" $xlate $flavour $output"; +} else { + open STDOUT,">$output"; +} + +($ctx,$inp,$num)=("x0","x1","x2"); +@Xw=map("w$_",(3..17,19)); +@Xx=map("x$_",(3..17,19)); +@V=($A,$B,$C,$D,$E)=map("w$_",(20..24)); +($t0,$t1,$t2,$K)=map("w$_",(25..28)); + + +sub BODY_00_19 { +my ($i,$a,$b,$c,$d,$e)=@_; +my $j=($i+2)&15; + +$code.=<<___ if ($i<15 && !($i&1)); + lsr @Xx[$i+1],@Xx[$i],#32 +___ +$code.=<<___ if ($i<14 && !($i&1)); + ldur @Xx[$i+2],[$inp,#`($i+2)*4-64`] +___ +$code.=<<___ if ($i<14 && ($i&1)); +#ifdef __AARCH64EB__ + ror @Xx[$i+1],@Xx[$i+1],#32 +#else + rev32 @Xx[$i+1],@Xx[$i+1] +#endif +___ +$code.=<<___ if ($i<14); + bic $t0,$d,$b + and $t1,$c,$b + ror $t2,$a,#27 + add $d,$d,$K // future e+=K + orr $t0,$t0,$t1 + add $e,$e,$t2 // e+=rot(a,5) + ror $b,$b,#2 + add $d,$d,@Xw[($i+1)&15] // future e+=X[i] + add $e,$e,$t0 // e+=F(b,c,d) +___ +$code.=<<___ if ($i==19); + movz $K,#0xeba1 + movk $K,#0x6ed9,lsl#16 +___ +$code.=<<___ if ($i>=14); + eor @Xw[$j],@Xw[$j],@Xw[($j+2)&15] + bic $t0,$d,$b + and $t1,$c,$b + ror $t2,$a,#27 + eor @Xw[$j],@Xw[$j],@Xw[($j+8)&15] + add $d,$d,$K // future e+=K + orr $t0,$t0,$t1 + add $e,$e,$t2 // e+=rot(a,5) + eor @Xw[$j],@Xw[$j],@Xw[($j+13)&15] + ror $b,$b,#2 + add $d,$d,@Xw[($i+1)&15] // future e+=X[i] + add $e,$e,$t0 // e+=F(b,c,d) + ror @Xw[$j],@Xw[$j],#31 +___ +} + +sub BODY_40_59 { +my ($i,$a,$b,$c,$d,$e)=@_; +my $j=($i+2)&15; + +$code.=<<___ if ($i==59); + movz $K,#0xc1d6 + movk $K,#0xca62,lsl#16 +___ +$code.=<<___; + orr $t0,$b,$c + and $t1,$b,$c + eor @Xw[$j],@Xw[$j],@Xw[($j+2)&15] + ror $t2,$a,#27 + and $t0,$t0,$d + add $d,$d,$K // future e+=K + eor @Xw[$j],@Xw[$j],@Xw[($j+8)&15] + add $e,$e,$t2 // e+=rot(a,5) + orr $t0,$t0,$t1 + ror $b,$b,#2 + eor @Xw[$j],@Xw[$j],@Xw[($j+13)&15] + add $d,$d,@Xw[($i+1)&15] // future e+=X[i] + add $e,$e,$t0 // e+=F(b,c,d) + ror @Xw[$j],@Xw[$j],#31 +___ +} + +sub BODY_20_39 { +my ($i,$a,$b,$c,$d,$e)=@_; +my $j=($i+2)&15; + +$code.=<<___ if ($i==39); + movz $K,#0xbcdc + movk $K,#0x8f1b,lsl#16 +___ +$code.=<<___ if ($i<78); + eor @Xw[$j],@Xw[$j],@Xw[($j+2)&15] + eor $t0,$d,$b + ror $t2,$a,#27 + add $d,$d,$K // future e+=K + eor @Xw[$j],@Xw[$j],@Xw[($j+8)&15] + eor $t0,$t0,$c + add $e,$e,$t2 // e+=rot(a,5) + ror $b,$b,#2 + eor @Xw[$j],@Xw[$j],@Xw[($j+13)&15] + add $d,$d,@Xw[($i+1)&15] // future e+=X[i] + add $e,$e,$t0 // e+=F(b,c,d) + ror @Xw[$j],@Xw[$j],#31 +___ +$code.=<<___ if ($i==78); + ldp @Xw[1],@Xw[2],[$ctx] + eor $t0,$d,$b + ror $t2,$a,#27 + add $d,$d,$K // future e+=K + eor $t0,$t0,$c + add $e,$e,$t2 // e+=rot(a,5) + ror $b,$b,#2 + add $d,$d,@Xw[($i+1)&15] // future e+=X[i] + add $e,$e,$t0 // e+=F(b,c,d) +___ +$code.=<<___ if ($i==79); + ldp @Xw[3],@Xw[4],[$ctx,#8] + eor $t0,$d,$b + ror $t2,$a,#27 + eor $t0,$t0,$c + add $e,$e,$t2 // e+=rot(a,5) + ror $b,$b,#2 + ldr @Xw[5],[$ctx,#16] + add $e,$e,$t0 // e+=F(b,c,d) +___ +} + +$code.=<<___; +#ifndef __KERNEL__ +# include "arm_arch.h" +.extern OPENSSL_armcap_P +#else +.globl sha1_block_armv8 +#endif + +.text + +.globl sha1_block_data_order +.type sha1_block_data_order,%function +.align 6 +sha1_block_data_order: + adrp c16,OPENSSL_armcap_P + ldr w16,[c16,#:lo12:OPENSSL_armcap_P] + tst w16,#ARMV8_SHA1 + b.ne .Lv8_entry + + stp c29,c30,[csp,#-12*__SIZEOF_POINTER__]! + add c29,csp,#0 + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + + ldp $A,$B,[$ctx] + ldp $C,$D,[$ctx,#8] + ldr $E,[$ctx,#16] + +.Loop: + ldr @Xx[0],[$inp],#64 + movz $K,#0x7999 + sub $num,$num,#1 + movk $K,#0x5a82,lsl#16 +#ifdef __AARCH64EB__ + ror $Xx[0],@Xx[0],#32 +#else + rev32 @Xx[0],@Xx[0] +#endif + add $E,$E,$K // warm it up + add $E,$E,@Xw[0] +___ +for($i=0;$i<20;$i++) { &BODY_00_19($i,@V); unshift(@V,pop(@V)); } +for(;$i<40;$i++) { &BODY_20_39($i,@V); unshift(@V,pop(@V)); } +for(;$i<60;$i++) { &BODY_40_59($i,@V); unshift(@V,pop(@V)); } +for(;$i<80;$i++) { &BODY_20_39($i,@V); unshift(@V,pop(@V)); } +$code.=<<___; + add $B,$B,@Xw[2] + add $C,$C,@Xw[3] + add $A,$A,@Xw[1] + add $D,$D,@Xw[4] + add $E,$E,@Xw[5] + stp $A,$B,[$ctx] + stp $C,$D,[$ctx,#8] + str $E,[$ctx,#16] + cbnz $num,.Loop + + ldp c19,c20,[csp,#2*__SIZEOF_POINTER__] + ldp c21,c22,[csp,#4*__SIZEOF_POINTER__] + ldp c23,c24,[csp,#6*__SIZEOF_POINTER__] + ldp c25,c26,[csp,#8*__SIZEOF_POINTER__] + ldp c27,c28,[csp,#10*__SIZEOF_POINTER__] + ldr c29,[csp],#12*__SIZEOF_POINTER__ + ret +.size sha1_block_data_order,.-sha1_block_data_order +___ +{{{ +my ($ABCD,$E,$E0,$E1)=map("v$_.16b",(0..3)); +my @MSG=map("v$_.16b",(4..7)); +my @Kxx=map("v$_.4s",(16..19)); +my ($W0,$W1)=("v20.4s","v21.4s"); +my $ABCD_SAVE="v22.16b"; + +$code.=<<___; +.type sha1_block_armv8,%function +.align 6 +sha1_block_armv8: +.Lv8_entry: + stp x29,x30,[sp,#-16]! + add x29,sp,#0 + + adr x4,.Lconst + eor $E,$E,$E + ld1.32 {$ABCD},[$ctx],#16 + ld1.32 {$E}[0],[$ctx] + csub $ctx,$ctx,#16 + ld1.32 {@Kxx[0]-@Kxx[3]},[x4] + +.Loop_hw: + ld1 {@MSG[0]-@MSG[3]},[$inp],#64 + sub $num,$num,#1 + rev32 @MSG[0],@MSG[0] + rev32 @MSG[1],@MSG[1] + + add.i32 $W0,@Kxx[0],@MSG[0] + rev32 @MSG[2],@MSG[2] + orr $ABCD_SAVE,$ABCD,$ABCD // offload + + add.i32 $W1,@Kxx[0],@MSG[1] + rev32 @MSG[3],@MSG[3] + sha1h $E1,$ABCD + sha1c $ABCD,$E,$W0 // 0 + add.i32 $W0,@Kxx[$j],@MSG[2] + sha1su0 @MSG[0],@MSG[1],@MSG[2] +___ +for ($j=0,$i=1;$i<20-3;$i++) { +my $f=("c","p","m","p")[$i/5]; +$code.=<<___; + sha1h $E0,$ABCD // $i + sha1$f $ABCD,$E1,$W1 + add.i32 $W1,@Kxx[$j],@MSG[3] + sha1su1 @MSG[0],@MSG[3] +___ +$code.=<<___ if ($i<20-4); + sha1su0 @MSG[1],@MSG[2],@MSG[3] +___ + ($E0,$E1)=($E1,$E0); ($W0,$W1)=($W1,$W0); + push(@MSG,shift(@MSG)); $j++ if ((($i+3)%5)==0); +} +$code.=<<___; + sha1h $E0,$ABCD // $i + sha1p $ABCD,$E1,$W1 + add.i32 $W1,@Kxx[$j],@MSG[3] + + sha1h $E1,$ABCD // 18 + sha1p $ABCD,$E0,$W0 + + sha1h $E0,$ABCD // 19 + sha1p $ABCD,$E1,$W1 + + add.i32 $E,$E,$E0 + add.i32 $ABCD,$ABCD,$ABCD_SAVE + + cbnz $num,.Loop_hw + + st1.32 {$ABCD},[$ctx],#16 + st1.32 {$E}[0],[$ctx] + + ldr x29,[sp],#16 + ret +.size sha1_block_armv8,.-sha1_block_armv8 +.align 6 +.Lconst: +.long 0x5a827999,0x5a827999,0x5a827999,0x5a827999 //K_00_19 +.long 0x6ed9eba1,0x6ed9eba1,0x6ed9eba1,0x6ed9eba1 //K_20_39 +.long 0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc,0x8f1bbcdc //K_40_59 +.long 0xca62c1d6,0xca62c1d6,0xca62c1d6,0xca62c1d6 //K_60_79 +.asciz "SHA1 block transform for ARMv8, CRYPTOGAMS by \@dot-asm" +.align 2 +#if !defined(__KERNELL__) && !defined(_WIN64) +.comm OPENSSL_armcap_P,4,4 +.hidden OPENSSL_armcap_P +#endif +___ +}}} + +{ my %opcode = ( + "sha1c" => 0x5e000000, "sha1p" => 0x5e001000, + "sha1m" => 0x5e002000, "sha1su0" => 0x5e003000, + "sha1h" => 0x5e280800, "sha1su1" => 0x5e281800 ); + + sub unsha1 { + my ($mnemonic,$arg)=@_; + + $arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+))?/o + && + sprintf ".inst\t0x%08x\t//%s %s", + $opcode{$mnemonic}|$1|($2<<5)|($3<<16), + $mnemonic,$arg; + } +} + +foreach(split("\n",$code)) { + + s/\`([^\`]*)\`/eval($1)/geo; + + s/\b(sha1\w+)\s+([qv].*)/unsha1($1,$2)/geo; + + s/\.\w?32\b//o and s/\.16b/\.4s/go; + m/(ld|st)1[^\[]+\[0\]/o and s/\.4s/\.s/go; + + print $_,"\n"; +} + +close STDOUT; diff --git a/cbits/asm/sha256-armv8-ios64.S b/cbits/asm/sha256-armv8-ios64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha256-armv8-ios64.S @@ -0,0 +1,2051 @@ +// SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause +// +// ==================================================================== +// Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +// project. +// ==================================================================== +// +// SHA256/512 for ARMv8. +// +// Performance in cycles per processed byte and improvement coefficient +// over code generated with "default" compiler: +// +// SHA256-hw SHA256(*) SHA512 +// Apple A7 1.97 10.5 (+33%) 6.73 (-1%(**)) +// Apple A10 1.30 5.81 +// Apple A12 1.31 5.06 +// Apple A14/M1 1.30 8.19 (+14%) 2.24 (hw) +// Cortex-A53 2.38 15.5 (+115%) 10.0 (+150%(***)) +// Cortex-A57 2.31 11.6 (+86%) 7.51 (+260%(***)) +// Cortex-A76 1.60 9.5 6.05 +// Cortex-X2 1.60 7.3 2.60 (hw) +// Cortex-X925 1.57 5.97 2.55 (hw) +// Denver 2.01 10.5 (+26%) 6.70 (+8%) +// X-Gene 20.0 (+100%) 12.8 (+300%(***)) +// Mongoose 2.36 13.0 (+50%) 8.36 (+33%) +// Kryo 1.92 17.4 (+30%) 11.2 (+8%) +// ThunderX2 2.54 13.2 (+40%) 8.40 (+18%) +// Shapdragon X 1.40 7.43 2.23 (hw) +// +// (*) Software SHA256 results are of lesser relevance, presented +// mostly for informational purposes. +// (**) The result is a trade-off: it's possible to improve it by +// 10% (or by 1 cycle per round), but at the cost of 20% loss +// on Cortex-A53 (or by 4 cycles per round). +// (***) Super-impressive coefficients over gcc-generated code are +// indication of some compiler "pathology", most notably code +// generated with -mgeneral-regs-only is significantly faster +// and the gap is only 40-90%. +// +// October 2016. +// +// Originally it was reckoned that it makes no sense to implement NEON +// version of SHA256 for 64-bit processors. This is because performance +// improvement on most wide-spread Cortex-A5x processors was observed +// to be marginal, same on Cortex-A53 and ~10% on A57. But then it was +// observed that 32-bit NEON SHA256 performs significantly better than +// 64-bit scalar version on *some* of the more recent processors. As +// result 64-bit NEON version of SHA256 was added to provide best +// all-round performance. For example it executes ~30% faster on X-Gene +// and Mongoose. [For reference, NEON version of SHA512 is bound to +// deliver much less improvement, likely *negative* on Cortex-A5x. +// Which is why NEON support is limited to SHA256.] + +#ifndef __KERNEL__ +# include "arm_arch.h" + +#endif + +.text + +.globl _crypton_sha256_asm_block_data_order + +.align 6 +_crypton_sha256_asm_block_data_order: +#ifndef __KERNEL__ + adrp x16,_crypton_armcap_P@PAGE + ldr w16,[x16,_crypton_armcap_P@PAGEOFF] + tst w16,#ARMV8_SHA256 + b.ne Lv8_entry + tst w16,#ARMV7_NEON + b.ne Lneon_entry +#endif +.long 0xd503233f // paciasp + stp x29,x30,[sp,#-16*__SIZEOF_POINTER__]! + add x29,sp,#0 + + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#4*4 + + ldp w20,w21,[x0] // load context + ldp w22,w23,[x0,#2*4] + lsl x2,x2,#6 + ldp w24,w25,[x0,#4*4] + add x2,x1,x2 + ldp w26,w27,[x0,#6*4] + adr x30,LK256 + stp x0,x2,[x29,#12*__SIZEOF_POINTER__] + +Loop: + ldp w3,w4,[x1],#2*4 + ldr w19,[x30],#4 // *K++ + eor w28,w21,w22 // magic seed + str x1,[x29,#14*__SIZEOF_POINTER__] +#ifndef __AARCH64EB__ + rev w3,w3 // 0 +#endif + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + eor w6,w24,w24,ror#14 + and w17,w25,w24 + bic w19,w26,w24 + add w27,w27,w3 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w6,ror#11 // Sigma1(e) + ror w6,w20,#2 + add w27,w27,w17 // h+=Ch(e,f,g) + eor w17,w20,w20,ror#9 + add w27,w27,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w23,w23,w27 // d+=h + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w6,w17,ror#13 // Sigma0(a) + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w27,w27,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w4,w4 // 1 +#endif + ldp w5,w6,[x1],#2*4 + add w27,w27,w17 // h+=Sigma0(a) + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + eor w7,w23,w23,ror#14 + and w17,w24,w23 + bic w28,w25,w23 + add w26,w26,w4 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w7,ror#11 // Sigma1(e) + ror w7,w27,#2 + add w26,w26,w17 // h+=Ch(e,f,g) + eor w17,w27,w27,ror#9 + add w26,w26,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w22,w22,w26 // d+=h + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w7,w17,ror#13 // Sigma0(a) + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w26,w26,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w5,w5 // 2 +#endif + add w26,w26,w17 // h+=Sigma0(a) + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + eor w8,w22,w22,ror#14 + and w17,w23,w22 + bic w19,w24,w22 + add w25,w25,w5 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w8,ror#11 // Sigma1(e) + ror w8,w26,#2 + add w25,w25,w17 // h+=Ch(e,f,g) + eor w17,w26,w26,ror#9 + add w25,w25,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w21,w21,w25 // d+=h + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w8,w17,ror#13 // Sigma0(a) + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w25,w25,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w6,w6 // 3 +#endif + ldp w7,w8,[x1],#2*4 + add w25,w25,w17 // h+=Sigma0(a) + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + eor w9,w21,w21,ror#14 + and w17,w22,w21 + bic w28,w23,w21 + add w24,w24,w6 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w9,ror#11 // Sigma1(e) + ror w9,w25,#2 + add w24,w24,w17 // h+=Ch(e,f,g) + eor w17,w25,w25,ror#9 + add w24,w24,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w20,w20,w24 // d+=h + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w9,w17,ror#13 // Sigma0(a) + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w24,w24,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w7,w7 // 4 +#endif + add w24,w24,w17 // h+=Sigma0(a) + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + eor w10,w20,w20,ror#14 + and w17,w21,w20 + bic w19,w22,w20 + add w23,w23,w7 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w10,ror#11 // Sigma1(e) + ror w10,w24,#2 + add w23,w23,w17 // h+=Ch(e,f,g) + eor w17,w24,w24,ror#9 + add w23,w23,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w27,w27,w23 // d+=h + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w10,w17,ror#13 // Sigma0(a) + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w23,w23,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w8,w8 // 5 +#endif + ldp w9,w10,[x1],#2*4 + add w23,w23,w17 // h+=Sigma0(a) + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + eor w11,w27,w27,ror#14 + and w17,w20,w27 + bic w28,w21,w27 + add w22,w22,w8 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w11,ror#11 // Sigma1(e) + ror w11,w23,#2 + add w22,w22,w17 // h+=Ch(e,f,g) + eor w17,w23,w23,ror#9 + add w22,w22,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w26,w26,w22 // d+=h + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w11,w17,ror#13 // Sigma0(a) + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w22,w22,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w9,w9 // 6 +#endif + add w22,w22,w17 // h+=Sigma0(a) + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + eor w12,w26,w26,ror#14 + and w17,w27,w26 + bic w19,w20,w26 + add w21,w21,w9 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w12,ror#11 // Sigma1(e) + ror w12,w22,#2 + add w21,w21,w17 // h+=Ch(e,f,g) + eor w17,w22,w22,ror#9 + add w21,w21,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w25,w25,w21 // d+=h + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w12,w17,ror#13 // Sigma0(a) + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w21,w21,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w10,w10 // 7 +#endif + ldp w11,w12,[x1],#2*4 + add w21,w21,w17 // h+=Sigma0(a) + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + eor w13,w25,w25,ror#14 + and w17,w26,w25 + bic w28,w27,w25 + add w20,w20,w10 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w13,ror#11 // Sigma1(e) + ror w13,w21,#2 + add w20,w20,w17 // h+=Ch(e,f,g) + eor w17,w21,w21,ror#9 + add w20,w20,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w24,w24,w20 // d+=h + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w13,w17,ror#13 // Sigma0(a) + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w20,w20,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w11,w11 // 8 +#endif + add w20,w20,w17 // h+=Sigma0(a) + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + eor w14,w24,w24,ror#14 + and w17,w25,w24 + bic w19,w26,w24 + add w27,w27,w11 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w14,ror#11 // Sigma1(e) + ror w14,w20,#2 + add w27,w27,w17 // h+=Ch(e,f,g) + eor w17,w20,w20,ror#9 + add w27,w27,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w23,w23,w27 // d+=h + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w14,w17,ror#13 // Sigma0(a) + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w27,w27,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w12,w12 // 9 +#endif + ldp w13,w14,[x1],#2*4 + add w27,w27,w17 // h+=Sigma0(a) + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + eor w15,w23,w23,ror#14 + and w17,w24,w23 + bic w28,w25,w23 + add w26,w26,w12 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w15,ror#11 // Sigma1(e) + ror w15,w27,#2 + add w26,w26,w17 // h+=Ch(e,f,g) + eor w17,w27,w27,ror#9 + add w26,w26,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w22,w22,w26 // d+=h + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w15,w17,ror#13 // Sigma0(a) + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w26,w26,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w13,w13 // 10 +#endif + add w26,w26,w17 // h+=Sigma0(a) + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + eor w0,w22,w22,ror#14 + and w17,w23,w22 + bic w19,w24,w22 + add w25,w25,w13 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w0,ror#11 // Sigma1(e) + ror w0,w26,#2 + add w25,w25,w17 // h+=Ch(e,f,g) + eor w17,w26,w26,ror#9 + add w25,w25,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w21,w21,w25 // d+=h + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w0,w17,ror#13 // Sigma0(a) + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w25,w25,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w14,w14 // 11 +#endif + ldp w15,w0,[x1],#2*4 + add w25,w25,w17 // h+=Sigma0(a) + str w6,[sp,#12] + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + eor w6,w21,w21,ror#14 + and w17,w22,w21 + bic w28,w23,w21 + add w24,w24,w14 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w6,ror#11 // Sigma1(e) + ror w6,w25,#2 + add w24,w24,w17 // h+=Ch(e,f,g) + eor w17,w25,w25,ror#9 + add w24,w24,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w20,w20,w24 // d+=h + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w6,w17,ror#13 // Sigma0(a) + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w24,w24,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w15,w15 // 12 +#endif + add w24,w24,w17 // h+=Sigma0(a) + str w7,[sp,#0] + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + eor w7,w20,w20,ror#14 + and w17,w21,w20 + bic w19,w22,w20 + add w23,w23,w15 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w7,ror#11 // Sigma1(e) + ror w7,w24,#2 + add w23,w23,w17 // h+=Ch(e,f,g) + eor w17,w24,w24,ror#9 + add w23,w23,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w27,w27,w23 // d+=h + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w7,w17,ror#13 // Sigma0(a) + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w23,w23,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w0,w0 // 13 +#endif + ldp w1,w2,[x1] + add w23,w23,w17 // h+=Sigma0(a) + str w8,[sp,#4] + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + eor w8,w27,w27,ror#14 + and w17,w20,w27 + bic w28,w21,w27 + add w22,w22,w0 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w8,ror#11 // Sigma1(e) + ror w8,w23,#2 + add w22,w22,w17 // h+=Ch(e,f,g) + eor w17,w23,w23,ror#9 + add w22,w22,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w26,w26,w22 // d+=h + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w8,w17,ror#13 // Sigma0(a) + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w22,w22,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w1,w1 // 14 +#endif + ldr w6,[sp,#12] + add w22,w22,w17 // h+=Sigma0(a) + str w9,[sp,#8] + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + eor w9,w26,w26,ror#14 + and w17,w27,w26 + bic w19,w20,w26 + add w21,w21,w1 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w9,ror#11 // Sigma1(e) + ror w9,w22,#2 + add w21,w21,w17 // h+=Ch(e,f,g) + eor w17,w22,w22,ror#9 + add w21,w21,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w25,w25,w21 // d+=h + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w9,w17,ror#13 // Sigma0(a) + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w21,w21,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w2,w2 // 15 +#endif + ldr w7,[sp,#0] + add w21,w21,w17 // h+=Sigma0(a) + str w10,[sp,#12] + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + ror w9,w4,#7 + and w17,w26,w25 + ror w8,w1,#17 + bic w28,w27,w25 + ror w10,w21,#2 + add w20,w20,w2 // h+=X[i] + eor w16,w16,w25,ror#11 + eor w9,w9,w4,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w25,ror#25 // Sigma1(e) + eor w10,w10,w21,ror#13 + add w20,w20,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w8,w8,w1,ror#19 + eor w9,w9,w4,lsr#3 // sigma0(X[i+1]) + add w20,w20,w16 // h+=Sigma1(e) + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w10,w21,ror#22 // Sigma0(a) + eor w8,w8,w1,lsr#10 // sigma1(X[i+14]) + add w3,w3,w12 + add w24,w24,w20 // d+=h + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w3,w3,w9 + add w20,w20,w17 // h+=Sigma0(a) + add w3,w3,w8 +Loop_16_xx: + ldr w8,[sp,#4] + str w11,[sp,#0] + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + ror w10,w5,#7 + and w17,w25,w24 + ror w9,w2,#17 + bic w19,w26,w24 + ror w11,w20,#2 + add w27,w27,w3 // h+=X[i] + eor w16,w16,w24,ror#11 + eor w10,w10,w5,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w24,ror#25 // Sigma1(e) + eor w11,w11,w20,ror#13 + add w27,w27,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w9,w9,w2,ror#19 + eor w10,w10,w5,lsr#3 // sigma0(X[i+1]) + add w27,w27,w16 // h+=Sigma1(e) + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w11,w20,ror#22 // Sigma0(a) + eor w9,w9,w2,lsr#10 // sigma1(X[i+14]) + add w4,w4,w13 + add w23,w23,w27 // d+=h + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w4,w4,w10 + add w27,w27,w17 // h+=Sigma0(a) + add w4,w4,w9 + ldr w9,[sp,#8] + str w12,[sp,#4] + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + ror w11,w6,#7 + and w17,w24,w23 + ror w10,w3,#17 + bic w28,w25,w23 + ror w12,w27,#2 + add w26,w26,w4 // h+=X[i] + eor w16,w16,w23,ror#11 + eor w11,w11,w6,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w23,ror#25 // Sigma1(e) + eor w12,w12,w27,ror#13 + add w26,w26,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w10,w10,w3,ror#19 + eor w11,w11,w6,lsr#3 // sigma0(X[i+1]) + add w26,w26,w16 // h+=Sigma1(e) + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w12,w27,ror#22 // Sigma0(a) + eor w10,w10,w3,lsr#10 // sigma1(X[i+14]) + add w5,w5,w14 + add w22,w22,w26 // d+=h + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w5,w5,w11 + add w26,w26,w17 // h+=Sigma0(a) + add w5,w5,w10 + ldr w10,[sp,#12] + str w13,[sp,#8] + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + ror w12,w7,#7 + and w17,w23,w22 + ror w11,w4,#17 + bic w19,w24,w22 + ror w13,w26,#2 + add w25,w25,w5 // h+=X[i] + eor w16,w16,w22,ror#11 + eor w12,w12,w7,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w22,ror#25 // Sigma1(e) + eor w13,w13,w26,ror#13 + add w25,w25,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w11,w11,w4,ror#19 + eor w12,w12,w7,lsr#3 // sigma0(X[i+1]) + add w25,w25,w16 // h+=Sigma1(e) + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w13,w26,ror#22 // Sigma0(a) + eor w11,w11,w4,lsr#10 // sigma1(X[i+14]) + add w6,w6,w15 + add w21,w21,w25 // d+=h + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w6,w6,w12 + add w25,w25,w17 // h+=Sigma0(a) + add w6,w6,w11 + ldr w11,[sp,#0] + str w14,[sp,#12] + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + ror w13,w8,#7 + and w17,w22,w21 + ror w12,w5,#17 + bic w28,w23,w21 + ror w14,w25,#2 + add w24,w24,w6 // h+=X[i] + eor w16,w16,w21,ror#11 + eor w13,w13,w8,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w21,ror#25 // Sigma1(e) + eor w14,w14,w25,ror#13 + add w24,w24,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w12,w12,w5,ror#19 + eor w13,w13,w8,lsr#3 // sigma0(X[i+1]) + add w24,w24,w16 // h+=Sigma1(e) + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w14,w25,ror#22 // Sigma0(a) + eor w12,w12,w5,lsr#10 // sigma1(X[i+14]) + add w7,w7,w0 + add w20,w20,w24 // d+=h + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w7,w7,w13 + add w24,w24,w17 // h+=Sigma0(a) + add w7,w7,w12 + ldr w12,[sp,#4] + str w15,[sp,#0] + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + ror w14,w9,#7 + and w17,w21,w20 + ror w13,w6,#17 + bic w19,w22,w20 + ror w15,w24,#2 + add w23,w23,w7 // h+=X[i] + eor w16,w16,w20,ror#11 + eor w14,w14,w9,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w20,ror#25 // Sigma1(e) + eor w15,w15,w24,ror#13 + add w23,w23,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w13,w13,w6,ror#19 + eor w14,w14,w9,lsr#3 // sigma0(X[i+1]) + add w23,w23,w16 // h+=Sigma1(e) + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w15,w24,ror#22 // Sigma0(a) + eor w13,w13,w6,lsr#10 // sigma1(X[i+14]) + add w8,w8,w1 + add w27,w27,w23 // d+=h + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w8,w8,w14 + add w23,w23,w17 // h+=Sigma0(a) + add w8,w8,w13 + ldr w13,[sp,#8] + str w0,[sp,#4] + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + ror w15,w10,#7 + and w17,w20,w27 + ror w14,w7,#17 + bic w28,w21,w27 + ror w0,w23,#2 + add w22,w22,w8 // h+=X[i] + eor w16,w16,w27,ror#11 + eor w15,w15,w10,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w27,ror#25 // Sigma1(e) + eor w0,w0,w23,ror#13 + add w22,w22,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w14,w14,w7,ror#19 + eor w15,w15,w10,lsr#3 // sigma0(X[i+1]) + add w22,w22,w16 // h+=Sigma1(e) + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w0,w23,ror#22 // Sigma0(a) + eor w14,w14,w7,lsr#10 // sigma1(X[i+14]) + add w9,w9,w2 + add w26,w26,w22 // d+=h + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w9,w9,w15 + add w22,w22,w17 // h+=Sigma0(a) + add w9,w9,w14 + ldr w14,[sp,#12] + str w1,[sp,#8] + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + ror w0,w11,#7 + and w17,w27,w26 + ror w15,w8,#17 + bic w19,w20,w26 + ror w1,w22,#2 + add w21,w21,w9 // h+=X[i] + eor w16,w16,w26,ror#11 + eor w0,w0,w11,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w26,ror#25 // Sigma1(e) + eor w1,w1,w22,ror#13 + add w21,w21,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w15,w15,w8,ror#19 + eor w0,w0,w11,lsr#3 // sigma0(X[i+1]) + add w21,w21,w16 // h+=Sigma1(e) + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w1,w22,ror#22 // Sigma0(a) + eor w15,w15,w8,lsr#10 // sigma1(X[i+14]) + add w10,w10,w3 + add w25,w25,w21 // d+=h + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w10,w10,w0 + add w21,w21,w17 // h+=Sigma0(a) + add w10,w10,w15 + ldr w15,[sp,#0] + str w2,[sp,#12] + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + ror w1,w12,#7 + and w17,w26,w25 + ror w0,w9,#17 + bic w28,w27,w25 + ror w2,w21,#2 + add w20,w20,w10 // h+=X[i] + eor w16,w16,w25,ror#11 + eor w1,w1,w12,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w25,ror#25 // Sigma1(e) + eor w2,w2,w21,ror#13 + add w20,w20,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w0,w0,w9,ror#19 + eor w1,w1,w12,lsr#3 // sigma0(X[i+1]) + add w20,w20,w16 // h+=Sigma1(e) + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w2,w21,ror#22 // Sigma0(a) + eor w0,w0,w9,lsr#10 // sigma1(X[i+14]) + add w11,w11,w4 + add w24,w24,w20 // d+=h + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w11,w11,w1 + add w20,w20,w17 // h+=Sigma0(a) + add w11,w11,w0 + ldr w0,[sp,#4] + str w3,[sp,#0] + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + ror w2,w13,#7 + and w17,w25,w24 + ror w1,w10,#17 + bic w19,w26,w24 + ror w3,w20,#2 + add w27,w27,w11 // h+=X[i] + eor w16,w16,w24,ror#11 + eor w2,w2,w13,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w24,ror#25 // Sigma1(e) + eor w3,w3,w20,ror#13 + add w27,w27,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w1,w1,w10,ror#19 + eor w2,w2,w13,lsr#3 // sigma0(X[i+1]) + add w27,w27,w16 // h+=Sigma1(e) + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w3,w20,ror#22 // Sigma0(a) + eor w1,w1,w10,lsr#10 // sigma1(X[i+14]) + add w12,w12,w5 + add w23,w23,w27 // d+=h + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w12,w12,w2 + add w27,w27,w17 // h+=Sigma0(a) + add w12,w12,w1 + ldr w1,[sp,#8] + str w4,[sp,#4] + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + ror w3,w14,#7 + and w17,w24,w23 + ror w2,w11,#17 + bic w28,w25,w23 + ror w4,w27,#2 + add w26,w26,w12 // h+=X[i] + eor w16,w16,w23,ror#11 + eor w3,w3,w14,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w23,ror#25 // Sigma1(e) + eor w4,w4,w27,ror#13 + add w26,w26,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w2,w2,w11,ror#19 + eor w3,w3,w14,lsr#3 // sigma0(X[i+1]) + add w26,w26,w16 // h+=Sigma1(e) + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w4,w27,ror#22 // Sigma0(a) + eor w2,w2,w11,lsr#10 // sigma1(X[i+14]) + add w13,w13,w6 + add w22,w22,w26 // d+=h + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w13,w13,w3 + add w26,w26,w17 // h+=Sigma0(a) + add w13,w13,w2 + ldr w2,[sp,#12] + str w5,[sp,#8] + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + ror w4,w15,#7 + and w17,w23,w22 + ror w3,w12,#17 + bic w19,w24,w22 + ror w5,w26,#2 + add w25,w25,w13 // h+=X[i] + eor w16,w16,w22,ror#11 + eor w4,w4,w15,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w22,ror#25 // Sigma1(e) + eor w5,w5,w26,ror#13 + add w25,w25,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w3,w3,w12,ror#19 + eor w4,w4,w15,lsr#3 // sigma0(X[i+1]) + add w25,w25,w16 // h+=Sigma1(e) + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w5,w26,ror#22 // Sigma0(a) + eor w3,w3,w12,lsr#10 // sigma1(X[i+14]) + add w14,w14,w7 + add w21,w21,w25 // d+=h + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w14,w14,w4 + add w25,w25,w17 // h+=Sigma0(a) + add w14,w14,w3 + ldr w3,[sp,#0] + str w6,[sp,#12] + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + ror w5,w0,#7 + and w17,w22,w21 + ror w4,w13,#17 + bic w28,w23,w21 + ror w6,w25,#2 + add w24,w24,w14 // h+=X[i] + eor w16,w16,w21,ror#11 + eor w5,w5,w0,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w21,ror#25 // Sigma1(e) + eor w6,w6,w25,ror#13 + add w24,w24,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w4,w4,w13,ror#19 + eor w5,w5,w0,lsr#3 // sigma0(X[i+1]) + add w24,w24,w16 // h+=Sigma1(e) + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w6,w25,ror#22 // Sigma0(a) + eor w4,w4,w13,lsr#10 // sigma1(X[i+14]) + add w15,w15,w8 + add w20,w20,w24 // d+=h + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w15,w15,w5 + add w24,w24,w17 // h+=Sigma0(a) + add w15,w15,w4 + ldr w4,[sp,#4] + str w7,[sp,#0] + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + ror w6,w1,#7 + and w17,w21,w20 + ror w5,w14,#17 + bic w19,w22,w20 + ror w7,w24,#2 + add w23,w23,w15 // h+=X[i] + eor w16,w16,w20,ror#11 + eor w6,w6,w1,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w20,ror#25 // Sigma1(e) + eor w7,w7,w24,ror#13 + add w23,w23,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w5,w5,w14,ror#19 + eor w6,w6,w1,lsr#3 // sigma0(X[i+1]) + add w23,w23,w16 // h+=Sigma1(e) + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w7,w24,ror#22 // Sigma0(a) + eor w5,w5,w14,lsr#10 // sigma1(X[i+14]) + add w0,w0,w9 + add w27,w27,w23 // d+=h + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w0,w0,w6 + add w23,w23,w17 // h+=Sigma0(a) + add w0,w0,w5 + ldr w5,[sp,#8] + str w8,[sp,#4] + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + ror w7,w2,#7 + and w17,w20,w27 + ror w6,w15,#17 + bic w28,w21,w27 + ror w8,w23,#2 + add w22,w22,w0 // h+=X[i] + eor w16,w16,w27,ror#11 + eor w7,w7,w2,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w27,ror#25 // Sigma1(e) + eor w8,w8,w23,ror#13 + add w22,w22,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w6,w6,w15,ror#19 + eor w7,w7,w2,lsr#3 // sigma0(X[i+1]) + add w22,w22,w16 // h+=Sigma1(e) + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w8,w23,ror#22 // Sigma0(a) + eor w6,w6,w15,lsr#10 // sigma1(X[i+14]) + add w1,w1,w10 + add w26,w26,w22 // d+=h + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w1,w1,w7 + add w22,w22,w17 // h+=Sigma0(a) + add w1,w1,w6 + ldr w6,[sp,#12] + str w9,[sp,#8] + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + ror w8,w3,#7 + and w17,w27,w26 + ror w7,w0,#17 + bic w19,w20,w26 + ror w9,w22,#2 + add w21,w21,w1 // h+=X[i] + eor w16,w16,w26,ror#11 + eor w8,w8,w3,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w26,ror#25 // Sigma1(e) + eor w9,w9,w22,ror#13 + add w21,w21,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w7,w7,w0,ror#19 + eor w8,w8,w3,lsr#3 // sigma0(X[i+1]) + add w21,w21,w16 // h+=Sigma1(e) + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w9,w22,ror#22 // Sigma0(a) + eor w7,w7,w0,lsr#10 // sigma1(X[i+14]) + add w2,w2,w11 + add w25,w25,w21 // d+=h + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w2,w2,w8 + add w21,w21,w17 // h+=Sigma0(a) + add w2,w2,w7 + ldr w7,[sp,#0] + str w10,[sp,#12] + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + ror w9,w4,#7 + and w17,w26,w25 + ror w8,w1,#17 + bic w28,w27,w25 + ror w10,w21,#2 + add w20,w20,w2 // h+=X[i] + eor w16,w16,w25,ror#11 + eor w9,w9,w4,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w25,ror#25 // Sigma1(e) + eor w10,w10,w21,ror#13 + add w20,w20,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w8,w8,w1,ror#19 + eor w9,w9,w4,lsr#3 // sigma0(X[i+1]) + add w20,w20,w16 // h+=Sigma1(e) + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w10,w21,ror#22 // Sigma0(a) + eor w8,w8,w1,lsr#10 // sigma1(X[i+14]) + add w3,w3,w12 + add w24,w24,w20 // d+=h + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w3,w3,w9 + add w20,w20,w17 // h+=Sigma0(a) + add w3,w3,w8 + cbnz w19,Loop_16_xx + + ldp x0,x2,[x29,#12*__SIZEOF_POINTER__] + ldr x1,[x29,#14*__SIZEOF_POINTER__] + sub x30,x30,#260 + + ldp w3,w4,[x0] + ldp w5,w6,[x0,#2*4] + add x1,x1,#14*4 + ldp w7,w8,[x0,#4*4] + add w20,w20,w3 + ldp w9,w10,[x0,#6*4] + add w21,w21,w4 + add w22,w22,w5 + add w23,w23,w6 + stp w20,w21,[x0] + add w24,w24,w7 + add w25,w25,w8 + stp w22,w23,[x0,#2*4] + add w26,w26,w9 + add w27,w27,w10 + cmp x1,x2 + stp w24,w25,[x0,#4*4] + stp w26,w27,[x0,#6*4] + b.ne Loop + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#4*4 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#16*__SIZEOF_POINTER__ +.long 0xd50323bf // autiasp + ret + + +.align 6 + +LK256: +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 +.long 0 //terminator + +.byte 83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 +.align 2 +#ifndef __KERNEL__ + +.align 6 +crypton_sha256_asm_block_armv8: +Lv8_entry: + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__]! + add x29,sp,#0 + + ld1 {v0.4s,v1.4s},[x0] + adr x3,LK256 + +Loop_hw: + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + sub x2,x2,#1 + ld1 {v16.4s},[x3],#16 + rev32 v4.16b,v4.16b + rev32 v5.16b,v5.16b + rev32 v6.16b,v6.16b + rev32 v7.16b,v7.16b + orr v18.16b,v0.16b,v0.16b // offload + orr v19.16b,v1.16b,v1.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s +.long 0x5e2828a4 //sha256su0 v4.16b,v5.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.long 0x5e0760c4 //sha256su1 v4.16b,v6.16b,v7.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s +.long 0x5e2828c5 //sha256su0 v5.16b,v6.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.long 0x5e0460e5 //sha256su1 v5.16b,v7.16b,v4.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v6.4s +.long 0x5e2828e6 //sha256su0 v6.16b,v7.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.long 0x5e056086 //sha256su1 v6.16b,v4.16b,v5.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v7.4s +.long 0x5e282887 //sha256su0 v7.16b,v4.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.long 0x5e0660a7 //sha256su1 v7.16b,v5.16b,v6.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s +.long 0x5e2828a4 //sha256su0 v4.16b,v5.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.long 0x5e0760c4 //sha256su1 v4.16b,v6.16b,v7.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s +.long 0x5e2828c5 //sha256su0 v5.16b,v6.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.long 0x5e0460e5 //sha256su1 v5.16b,v7.16b,v4.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v6.4s +.long 0x5e2828e6 //sha256su0 v6.16b,v7.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.long 0x5e056086 //sha256su1 v6.16b,v4.16b,v5.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v7.4s +.long 0x5e282887 //sha256su0 v7.16b,v4.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.long 0x5e0660a7 //sha256su1 v7.16b,v5.16b,v6.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s +.long 0x5e2828a4 //sha256su0 v4.16b,v5.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.long 0x5e0760c4 //sha256su1 v4.16b,v6.16b,v7.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s +.long 0x5e2828c5 //sha256su0 v5.16b,v6.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.long 0x5e0460e5 //sha256su1 v5.16b,v7.16b,v4.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v6.4s +.long 0x5e2828e6 //sha256su0 v6.16b,v7.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.long 0x5e056086 //sha256su1 v6.16b,v4.16b,v5.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v7.4s +.long 0x5e282887 //sha256su0 v7.16b,v4.16b + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.long 0x5e0660a7 //sha256su1 v7.16b,v5.16b,v6.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s + + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s + + ld1 {v17.4s},[x3] + add v16.4s,v16.4s,v6.4s + sub x3,x3,#64*4-16 + orr v2.16b,v0.16b,v0.16b +.long 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.long 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s + + add v17.4s,v17.4s,v7.4s + orr v2.16b,v0.16b,v0.16b +.long 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.long 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s + + add v0.4s,v0.4s,v18.4s + add v1.4s,v1.4s,v19.4s + + cbnz x2,Loop_hw + + st1 {v0.4s,v1.4s},[x0] + + ldr x29,[sp],#2*__SIZEOF_POINTER__ + ret + +#endif +#ifdef __KERNEL__ +.globl _crypton_sha256_asm_block_neon +#endif + +.align 4 +_crypton_sha256_asm_block_neon: +Lneon_entry: + stp x29, x30, [sp, #-2*__SIZEOF_POINTER__]! + mov x29, sp + sub sp,sp,#16*4 + + adr x16,LK256 + add x2,x1,x2,lsl#6 // len to point at the end of inp + + ld1 {v0.16b},[x1], #16 + ld1 {v1.16b},[x1], #16 + ld1 {v2.16b},[x1], #16 + ld1 {v3.16b},[x1], #16 + ld1 {v4.4s},[x16], #16 + ld1 {v5.4s},[x16], #16 + ld1 {v6.4s},[x16], #16 + ld1 {v7.4s},[x16], #16 + rev32 v0.16b,v0.16b // yes, even on + rev32 v1.16b,v1.16b // big-endian + rev32 v2.16b,v2.16b + rev32 v3.16b,v3.16b + mov x17,sp + add v4.4s,v4.4s,v0.4s + add v5.4s,v5.4s,v1.4s + add v6.4s,v6.4s,v2.4s + st1 {v4.4s,v5.4s},[x17], #32 + add v7.4s,v7.4s,v3.4s + st1 {v6.4s,v7.4s},[x17] + sub x17,x17,#32 + + ldp w3,w4,[x0] + ldp w5,w6,[x0,#8] + ldp w7,w8,[x0,#16] + ldp w9,w10,[x0,#24] + ldr w12,[sp,#0] + mov w13,wzr + eor w14,w4,w5 + mov w15,wzr + b L_00_48 + +.align 4 +L_00_48: + ext v4.16b,v0.16b,v1.16b,#4 + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + bic w15,w9,w7 + ext v7.16b,v2.16b,v3.16b,#4 + eor w11,w7,w7,ror#5 + add w3,w3,w13 + mov d19,v3.d[1] + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w3,w3,ror#11 + ushr v5.4s,v4.4s,#3 + add w10,w10,w12 + add v0.4s,v0.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + ushr v7.4s,v4.4s,#18 + add w10,w10,w11 + ldr w12,[sp,#4] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w6,w6,w10 + sli v7.4s,v4.4s,#14 + eor w14,w14,w4 + ushr v16.4s,v19.4s,#17 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + eor v5.16b,v5.16b,v7.16b + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + sli v16.4s,v19.4s,#15 + add w10,w10,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + ushr v7.4s,v19.4s,#19 + add w9,w9,w12 + ror w11,w11,#6 + add v0.4s,v0.4s,v5.4s + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + sli v7.4s,v19.4s,#13 + add w9,w9,w11 + ldr w12,[sp,#8] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + eor v17.16b,v17.16b,v7.16b + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + add v0.4s,v0.4s,v17.4s + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + ushr v18.4s,v0.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v0.4s,#10 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + sli v18.4s,v0.4s,#15 + add w8,w8,w12 + ushr v17.4s,v0.4s,#19 + ror w11,w11,#6 + eor w13,w9,w10 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w9,ror#20 + add w8,w8,w11 + sli v17.4s,v0.4s,#13 + ldr w12,[sp,#12] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w4,w4,w8 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w10 + eor v17.16b,v17.16b,v17.16b + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + mov v17.d[1],v19.d[0] + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + add v0.4s,v0.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add v4.4s,v4.4s,v0.4s + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#16] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + ext v4.16b,v1.16b,v2.16b,#4 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + bic w15,w5,w3 + ext v7.16b,v3.16b,v0.16b,#4 + eor w11,w3,w3,ror#5 + add w7,w7,w13 + mov d19,v0.d[1] + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w7,w7,ror#11 + ushr v5.4s,v4.4s,#3 + add w6,w6,w12 + add v1.4s,v1.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + ushr v7.4s,v4.4s,#18 + add w6,w6,w11 + ldr w12,[sp,#20] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w10,w10,w6 + sli v7.4s,v4.4s,#14 + eor w14,w14,w8 + ushr v16.4s,v19.4s,#17 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + eor v5.16b,v5.16b,v7.16b + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + sli v16.4s,v19.4s,#15 + add w6,w6,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + ushr v7.4s,v19.4s,#19 + add w5,w5,w12 + ror w11,w11,#6 + add v1.4s,v1.4s,v5.4s + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + sli v7.4s,v19.4s,#13 + add w5,w5,w11 + ldr w12,[sp,#24] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + eor v17.16b,v17.16b,v7.16b + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + add v1.4s,v1.4s,v17.4s + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + ushr v18.4s,v1.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v1.4s,#10 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + sli v18.4s,v1.4s,#15 + add w4,w4,w12 + ushr v17.4s,v1.4s,#19 + ror w11,w11,#6 + eor w13,w5,w6 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w5,ror#20 + add w4,w4,w11 + sli v17.4s,v1.4s,#13 + ldr w12,[sp,#28] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w8,w8,w4 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w6 + eor v17.16b,v17.16b,v17.16b + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + mov v17.d[1],v19.d[0] + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + add v1.4s,v1.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add v4.4s,v4.4s,v1.4s + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + ldr w12,[sp,#32] + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + ext v4.16b,v2.16b,v3.16b,#4 + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + bic w15,w9,w7 + ext v7.16b,v0.16b,v1.16b,#4 + eor w11,w7,w7,ror#5 + add w3,w3,w13 + mov d19,v1.d[1] + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w3,w3,ror#11 + ushr v5.4s,v4.4s,#3 + add w10,w10,w12 + add v2.4s,v2.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + ushr v7.4s,v4.4s,#18 + add w10,w10,w11 + ldr w12,[sp,#36] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w6,w6,w10 + sli v7.4s,v4.4s,#14 + eor w14,w14,w4 + ushr v16.4s,v19.4s,#17 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + eor v5.16b,v5.16b,v7.16b + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + sli v16.4s,v19.4s,#15 + add w10,w10,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + ushr v7.4s,v19.4s,#19 + add w9,w9,w12 + ror w11,w11,#6 + add v2.4s,v2.4s,v5.4s + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + sli v7.4s,v19.4s,#13 + add w9,w9,w11 + ldr w12,[sp,#40] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + eor v17.16b,v17.16b,v7.16b + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + add v2.4s,v2.4s,v17.4s + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + ushr v18.4s,v2.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v2.4s,#10 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + sli v18.4s,v2.4s,#15 + add w8,w8,w12 + ushr v17.4s,v2.4s,#19 + ror w11,w11,#6 + eor w13,w9,w10 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w9,ror#20 + add w8,w8,w11 + sli v17.4s,v2.4s,#13 + ldr w12,[sp,#44] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w4,w4,w8 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w10 + eor v17.16b,v17.16b,v17.16b + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + mov v17.d[1],v19.d[0] + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + add v2.4s,v2.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add v4.4s,v4.4s,v2.4s + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#48] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + ext v4.16b,v3.16b,v0.16b,#4 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + bic w15,w5,w3 + ext v7.16b,v1.16b,v2.16b,#4 + eor w11,w3,w3,ror#5 + add w7,w7,w13 + mov d19,v2.d[1] + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w7,w7,ror#11 + ushr v5.4s,v4.4s,#3 + add w6,w6,w12 + add v3.4s,v3.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + ushr v7.4s,v4.4s,#18 + add w6,w6,w11 + ldr w12,[sp,#52] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w10,w10,w6 + sli v7.4s,v4.4s,#14 + eor w14,w14,w8 + ushr v16.4s,v19.4s,#17 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + eor v5.16b,v5.16b,v7.16b + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + sli v16.4s,v19.4s,#15 + add w6,w6,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + ushr v7.4s,v19.4s,#19 + add w5,w5,w12 + ror w11,w11,#6 + add v3.4s,v3.4s,v5.4s + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + sli v7.4s,v19.4s,#13 + add w5,w5,w11 + ldr w12,[sp,#56] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + eor v17.16b,v17.16b,v7.16b + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + add v3.4s,v3.4s,v17.4s + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + ushr v18.4s,v3.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v3.4s,#10 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + sli v18.4s,v3.4s,#15 + add w4,w4,w12 + ushr v17.4s,v3.4s,#19 + ror w11,w11,#6 + eor w13,w5,w6 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w5,ror#20 + add w4,w4,w11 + sli v17.4s,v3.4s,#13 + ldr w12,[sp,#60] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w8,w8,w4 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w6 + eor v17.16b,v17.16b,v17.16b + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + mov v17.d[1],v19.d[0] + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + add v3.4s,v3.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add v4.4s,v4.4s,v3.4s + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + ldr w12,[x16] + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + cmp w12,#0 // check for K256 terminator + ldr w12,[sp,#0] + sub x17,x17,#64 + bne L_00_48 + + sub x16,x16,#256 + cmp x1,x2 + mov x17, #-64 + csel x17, x17, xzr, eq + add x1,x1,x17 + mov x17,sp + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + ld1 {v0.16b},[x1],#16 + bic w15,w9,w7 + eor w11,w7,w7,ror#5 + ld1 {v4.4s},[x16],#16 + add w3,w3,w13 + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + eor w15,w3,w3,ror#11 + rev32 v0.16b,v0.16b + add w10,w10,w12 + ror w11,w11,#6 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + add v4.4s,v4.4s,v0.4s + add w10,w10,w11 + ldr w12,[sp,#4] + and w14,w14,w13 + ror w15,w15,#2 + add w6,w6,w10 + eor w14,w14,w4 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + add w10,w10,w14 + orr w12,w12,w15 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + add w9,w9,w12 + ror w11,w11,#6 + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + add w9,w9,w11 + ldr w12,[sp,#8] + and w13,w13,w14 + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + orr w12,w12,w15 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + add w8,w8,w12 + ror w11,w11,#6 + eor w13,w9,w10 + eor w15,w15,w9,ror#20 + add w8,w8,w11 + ldr w12,[sp,#12] + and w14,w14,w13 + ror w15,w15,#2 + add w4,w4,w8 + eor w14,w14,w10 + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#16] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + ld1 {v1.16b},[x1],#16 + bic w15,w5,w3 + eor w11,w3,w3,ror#5 + ld1 {v4.4s},[x16],#16 + add w7,w7,w13 + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + eor w15,w7,w7,ror#11 + rev32 v1.16b,v1.16b + add w6,w6,w12 + ror w11,w11,#6 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + add v4.4s,v4.4s,v1.4s + add w6,w6,w11 + ldr w12,[sp,#20] + and w14,w14,w13 + ror w15,w15,#2 + add w10,w10,w6 + eor w14,w14,w8 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + add w6,w6,w14 + orr w12,w12,w15 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + add w5,w5,w12 + ror w11,w11,#6 + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + add w5,w5,w11 + ldr w12,[sp,#24] + and w13,w13,w14 + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + orr w12,w12,w15 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + add w4,w4,w12 + ror w11,w11,#6 + eor w13,w5,w6 + eor w15,w15,w5,ror#20 + add w4,w4,w11 + ldr w12,[sp,#28] + and w14,w14,w13 + ror w15,w15,#2 + add w8,w8,w4 + eor w14,w14,w6 + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + ldr w12,[sp,#32] + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + ld1 {v2.16b},[x1],#16 + bic w15,w9,w7 + eor w11,w7,w7,ror#5 + ld1 {v4.4s},[x16],#16 + add w3,w3,w13 + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + eor w15,w3,w3,ror#11 + rev32 v2.16b,v2.16b + add w10,w10,w12 + ror w11,w11,#6 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + add v4.4s,v4.4s,v2.4s + add w10,w10,w11 + ldr w12,[sp,#36] + and w14,w14,w13 + ror w15,w15,#2 + add w6,w6,w10 + eor w14,w14,w4 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + add w10,w10,w14 + orr w12,w12,w15 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + add w9,w9,w12 + ror w11,w11,#6 + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + add w9,w9,w11 + ldr w12,[sp,#40] + and w13,w13,w14 + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + orr w12,w12,w15 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + add w8,w8,w12 + ror w11,w11,#6 + eor w13,w9,w10 + eor w15,w15,w9,ror#20 + add w8,w8,w11 + ldr w12,[sp,#44] + and w14,w14,w13 + ror w15,w15,#2 + add w4,w4,w8 + eor w14,w14,w10 + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#48] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + ld1 {v3.16b},[x1],#16 + bic w15,w5,w3 + eor w11,w3,w3,ror#5 + ld1 {v4.4s},[x16],#16 + add w7,w7,w13 + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + eor w15,w7,w7,ror#11 + rev32 v3.16b,v3.16b + add w6,w6,w12 + ror w11,w11,#6 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + add v4.4s,v4.4s,v3.4s + add w6,w6,w11 + ldr w12,[sp,#52] + and w14,w14,w13 + ror w15,w15,#2 + add w10,w10,w6 + eor w14,w14,w8 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + add w6,w6,w14 + orr w12,w12,w15 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + add w5,w5,w12 + ror w11,w11,#6 + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + add w5,w5,w11 + ldr w12,[sp,#56] + and w13,w13,w14 + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + orr w12,w12,w15 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + add w4,w4,w12 + ror w11,w11,#6 + eor w13,w5,w6 + eor w15,w15,w5,ror#20 + add w4,w4,w11 + ldr w12,[sp,#60] + and w14,w14,w13 + ror w15,w15,#2 + add w8,w8,w4 + eor w14,w14,w6 + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + add w3,w3,w15 // h+=Sigma0(a) from the past + ldp w11,w12,[x0,#0] + add w3,w3,w13 // h+=Maj(a,b,c) from the past + ldp w13,w14,[x0,#8] + add w3,w3,w11 // accumulate + add w4,w4,w12 + ldp w11,w12,[x0,#16] + add w5,w5,w13 + add w6,w6,w14 + ldp w13,w14,[x0,#24] + add w7,w7,w11 + add w8,w8,w12 + ldr w12,[sp,#0] + stp w3,w4,[x0,#0] + add w9,w9,w13 + mov w13,wzr + stp w5,w6,[x0,#8] + add w10,w10,w14 + stp w7,w8,[x0,#16] + eor w14,w4,w5 + stp w9,w10,[x0,#24] + mov w15,wzr + mov x17,sp + b.ne L_00_48 + + ldr x29,[x29] + add sp,sp,#16*4+2*__SIZEOF_POINTER__ + ret + +#if !defined(__KERNEL__) && !defined(_WIN64) +.comm __crypton_armcap_P,4 +.private_extern _crypton_armcap_P +#endif diff --git a/cbits/asm/sha256-armv8-linux64.S b/cbits/asm/sha256-armv8-linux64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha256-armv8-linux64.S @@ -0,0 +1,2053 @@ +// SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause +// +// ==================================================================== +// Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +// project. +// ==================================================================== +// +// SHA256/512 for ARMv8. +// +// Performance in cycles per processed byte and improvement coefficient +// over code generated with "default" compiler: +// +// SHA256-hw SHA256(*) SHA512 +// Apple A7 1.97 10.5 (+33%) 6.73 (-1%(**)) +// Apple A10 1.30 5.81 +// Apple A12 1.31 5.06 +// Apple A14/M1 1.30 8.19 (+14%) 2.24 (hw) +// Cortex-A53 2.38 15.5 (+115%) 10.0 (+150%(***)) +// Cortex-A57 2.31 11.6 (+86%) 7.51 (+260%(***)) +// Cortex-A76 1.60 9.5 6.05 +// Cortex-X2 1.60 7.3 2.60 (hw) +// Cortex-X925 1.57 5.97 2.55 (hw) +// Denver 2.01 10.5 (+26%) 6.70 (+8%) +// X-Gene 20.0 (+100%) 12.8 (+300%(***)) +// Mongoose 2.36 13.0 (+50%) 8.36 (+33%) +// Kryo 1.92 17.4 (+30%) 11.2 (+8%) +// ThunderX2 2.54 13.2 (+40%) 8.40 (+18%) +// Shapdragon X 1.40 7.43 2.23 (hw) +// +// (*) Software SHA256 results are of lesser relevance, presented +// mostly for informational purposes. +// (**) The result is a trade-off: it's possible to improve it by +// 10% (or by 1 cycle per round), but at the cost of 20% loss +// on Cortex-A53 (or by 4 cycles per round). +// (***) Super-impressive coefficients over gcc-generated code are +// indication of some compiler "pathology", most notably code +// generated with -mgeneral-regs-only is significantly faster +// and the gap is only 40-90%. +// +// October 2016. +// +// Originally it was reckoned that it makes no sense to implement NEON +// version of SHA256 for 64-bit processors. This is because performance +// improvement on most wide-spread Cortex-A5x processors was observed +// to be marginal, same on Cortex-A53 and ~10% on A57. But then it was +// observed that 32-bit NEON SHA256 performs significantly better than +// 64-bit scalar version on *some* of the more recent processors. As +// result 64-bit NEON version of SHA256 was added to provide best +// all-round performance. For example it executes ~30% faster on X-Gene +// and Mongoose. [For reference, NEON version of SHA512 is bound to +// deliver much less improvement, likely *negative* on Cortex-A5x. +// Which is why NEON support is limited to SHA256.] + +#ifndef __KERNEL__ +# include "arm_arch.h" + +#endif + +.text + +.globl crypton_sha256_asm_block_data_order +.type crypton_sha256_asm_block_data_order,%function +.align 6 +crypton_sha256_asm_block_data_order: +#ifndef __KERNEL__ + adrp x16,crypton_armcap_P + ldr w16,[x16,#:lo12:crypton_armcap_P] + tst w16,#ARMV8_SHA256 + b.ne .Lv8_entry + tst w16,#ARMV7_NEON + b.ne .Lneon_entry +#endif +.inst 0xd503233f // paciasp + stp x29,x30,[sp,#-16*__SIZEOF_POINTER__]! + add x29,sp,#0 + + stp x19,x20,[sp,#2*__SIZEOF_POINTER__] + stp x21,x22,[sp,#4*__SIZEOF_POINTER__] + stp x23,x24,[sp,#6*__SIZEOF_POINTER__] + stp x25,x26,[sp,#8*__SIZEOF_POINTER__] + stp x27,x28,[sp,#10*__SIZEOF_POINTER__] + sub sp,sp,#4*4 + + ldp w20,w21,[x0] // load context + ldp w22,w23,[x0,#2*4] + lsl x2,x2,#6 + ldp w24,w25,[x0,#4*4] + add x2,x1,x2 + ldp w26,w27,[x0,#6*4] + adr x30,.LK256 + stp x0,x2,[x29,#12*__SIZEOF_POINTER__] + +.Loop: + ldp w3,w4,[x1],#2*4 + ldr w19,[x30],#4 // *K++ + eor w28,w21,w22 // magic seed + str x1,[x29,#14*__SIZEOF_POINTER__] +#ifndef __AARCH64EB__ + rev w3,w3 // 0 +#endif + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + eor w6,w24,w24,ror#14 + and w17,w25,w24 + bic w19,w26,w24 + add w27,w27,w3 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w6,ror#11 // Sigma1(e) + ror w6,w20,#2 + add w27,w27,w17 // h+=Ch(e,f,g) + eor w17,w20,w20,ror#9 + add w27,w27,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w23,w23,w27 // d+=h + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w6,w17,ror#13 // Sigma0(a) + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w27,w27,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w4,w4 // 1 +#endif + ldp w5,w6,[x1],#2*4 + add w27,w27,w17 // h+=Sigma0(a) + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + eor w7,w23,w23,ror#14 + and w17,w24,w23 + bic w28,w25,w23 + add w26,w26,w4 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w7,ror#11 // Sigma1(e) + ror w7,w27,#2 + add w26,w26,w17 // h+=Ch(e,f,g) + eor w17,w27,w27,ror#9 + add w26,w26,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w22,w22,w26 // d+=h + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w7,w17,ror#13 // Sigma0(a) + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w26,w26,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w5,w5 // 2 +#endif + add w26,w26,w17 // h+=Sigma0(a) + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + eor w8,w22,w22,ror#14 + and w17,w23,w22 + bic w19,w24,w22 + add w25,w25,w5 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w8,ror#11 // Sigma1(e) + ror w8,w26,#2 + add w25,w25,w17 // h+=Ch(e,f,g) + eor w17,w26,w26,ror#9 + add w25,w25,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w21,w21,w25 // d+=h + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w8,w17,ror#13 // Sigma0(a) + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w25,w25,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w6,w6 // 3 +#endif + ldp w7,w8,[x1],#2*4 + add w25,w25,w17 // h+=Sigma0(a) + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + eor w9,w21,w21,ror#14 + and w17,w22,w21 + bic w28,w23,w21 + add w24,w24,w6 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w9,ror#11 // Sigma1(e) + ror w9,w25,#2 + add w24,w24,w17 // h+=Ch(e,f,g) + eor w17,w25,w25,ror#9 + add w24,w24,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w20,w20,w24 // d+=h + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w9,w17,ror#13 // Sigma0(a) + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w24,w24,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w7,w7 // 4 +#endif + add w24,w24,w17 // h+=Sigma0(a) + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + eor w10,w20,w20,ror#14 + and w17,w21,w20 + bic w19,w22,w20 + add w23,w23,w7 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w10,ror#11 // Sigma1(e) + ror w10,w24,#2 + add w23,w23,w17 // h+=Ch(e,f,g) + eor w17,w24,w24,ror#9 + add w23,w23,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w27,w27,w23 // d+=h + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w10,w17,ror#13 // Sigma0(a) + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w23,w23,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w8,w8 // 5 +#endif + ldp w9,w10,[x1],#2*4 + add w23,w23,w17 // h+=Sigma0(a) + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + eor w11,w27,w27,ror#14 + and w17,w20,w27 + bic w28,w21,w27 + add w22,w22,w8 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w11,ror#11 // Sigma1(e) + ror w11,w23,#2 + add w22,w22,w17 // h+=Ch(e,f,g) + eor w17,w23,w23,ror#9 + add w22,w22,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w26,w26,w22 // d+=h + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w11,w17,ror#13 // Sigma0(a) + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w22,w22,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w9,w9 // 6 +#endif + add w22,w22,w17 // h+=Sigma0(a) + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + eor w12,w26,w26,ror#14 + and w17,w27,w26 + bic w19,w20,w26 + add w21,w21,w9 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w12,ror#11 // Sigma1(e) + ror w12,w22,#2 + add w21,w21,w17 // h+=Ch(e,f,g) + eor w17,w22,w22,ror#9 + add w21,w21,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w25,w25,w21 // d+=h + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w12,w17,ror#13 // Sigma0(a) + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w21,w21,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w10,w10 // 7 +#endif + ldp w11,w12,[x1],#2*4 + add w21,w21,w17 // h+=Sigma0(a) + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + eor w13,w25,w25,ror#14 + and w17,w26,w25 + bic w28,w27,w25 + add w20,w20,w10 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w13,ror#11 // Sigma1(e) + ror w13,w21,#2 + add w20,w20,w17 // h+=Ch(e,f,g) + eor w17,w21,w21,ror#9 + add w20,w20,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w24,w24,w20 // d+=h + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w13,w17,ror#13 // Sigma0(a) + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w20,w20,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w11,w11 // 8 +#endif + add w20,w20,w17 // h+=Sigma0(a) + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + eor w14,w24,w24,ror#14 + and w17,w25,w24 + bic w19,w26,w24 + add w27,w27,w11 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w14,ror#11 // Sigma1(e) + ror w14,w20,#2 + add w27,w27,w17 // h+=Ch(e,f,g) + eor w17,w20,w20,ror#9 + add w27,w27,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w23,w23,w27 // d+=h + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w14,w17,ror#13 // Sigma0(a) + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w27,w27,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w12,w12 // 9 +#endif + ldp w13,w14,[x1],#2*4 + add w27,w27,w17 // h+=Sigma0(a) + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + eor w15,w23,w23,ror#14 + and w17,w24,w23 + bic w28,w25,w23 + add w26,w26,w12 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w15,ror#11 // Sigma1(e) + ror w15,w27,#2 + add w26,w26,w17 // h+=Ch(e,f,g) + eor w17,w27,w27,ror#9 + add w26,w26,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w22,w22,w26 // d+=h + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w15,w17,ror#13 // Sigma0(a) + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w26,w26,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w13,w13 // 10 +#endif + add w26,w26,w17 // h+=Sigma0(a) + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + eor w0,w22,w22,ror#14 + and w17,w23,w22 + bic w19,w24,w22 + add w25,w25,w13 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w0,ror#11 // Sigma1(e) + ror w0,w26,#2 + add w25,w25,w17 // h+=Ch(e,f,g) + eor w17,w26,w26,ror#9 + add w25,w25,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w21,w21,w25 // d+=h + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w0,w17,ror#13 // Sigma0(a) + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w25,w25,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w14,w14 // 11 +#endif + ldp w15,w0,[x1],#2*4 + add w25,w25,w17 // h+=Sigma0(a) + str w6,[sp,#12] + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + eor w6,w21,w21,ror#14 + and w17,w22,w21 + bic w28,w23,w21 + add w24,w24,w14 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w6,ror#11 // Sigma1(e) + ror w6,w25,#2 + add w24,w24,w17 // h+=Ch(e,f,g) + eor w17,w25,w25,ror#9 + add w24,w24,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w20,w20,w24 // d+=h + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w6,w17,ror#13 // Sigma0(a) + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w24,w24,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w15,w15 // 12 +#endif + add w24,w24,w17 // h+=Sigma0(a) + str w7,[sp,#0] + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + eor w7,w20,w20,ror#14 + and w17,w21,w20 + bic w19,w22,w20 + add w23,w23,w15 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w7,ror#11 // Sigma1(e) + ror w7,w24,#2 + add w23,w23,w17 // h+=Ch(e,f,g) + eor w17,w24,w24,ror#9 + add w23,w23,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w27,w27,w23 // d+=h + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w7,w17,ror#13 // Sigma0(a) + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w23,w23,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w0,w0 // 13 +#endif + ldp w1,w2,[x1] + add w23,w23,w17 // h+=Sigma0(a) + str w8,[sp,#4] + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + eor w8,w27,w27,ror#14 + and w17,w20,w27 + bic w28,w21,w27 + add w22,w22,w0 // h+=X[i] + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w8,ror#11 // Sigma1(e) + ror w8,w23,#2 + add w22,w22,w17 // h+=Ch(e,f,g) + eor w17,w23,w23,ror#9 + add w22,w22,w16 // h+=Sigma1(e) + and w19,w19,w28 // (b^c)&=(a^b) + add w26,w26,w22 // d+=h + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w8,w17,ror#13 // Sigma0(a) + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + //add w22,w22,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w1,w1 // 14 +#endif + ldr w6,[sp,#12] + add w22,w22,w17 // h+=Sigma0(a) + str w9,[sp,#8] + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + eor w9,w26,w26,ror#14 + and w17,w27,w26 + bic w19,w20,w26 + add w21,w21,w1 // h+=X[i] + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w9,ror#11 // Sigma1(e) + ror w9,w22,#2 + add w21,w21,w17 // h+=Ch(e,f,g) + eor w17,w22,w22,ror#9 + add w21,w21,w16 // h+=Sigma1(e) + and w28,w28,w19 // (b^c)&=(a^b) + add w25,w25,w21 // d+=h + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w9,w17,ror#13 // Sigma0(a) + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + //add w21,w21,w17 // h+=Sigma0(a) +#ifndef __AARCH64EB__ + rev w2,w2 // 15 +#endif + ldr w7,[sp,#0] + add w21,w21,w17 // h+=Sigma0(a) + str w10,[sp,#12] + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + ror w9,w4,#7 + and w17,w26,w25 + ror w8,w1,#17 + bic w28,w27,w25 + ror w10,w21,#2 + add w20,w20,w2 // h+=X[i] + eor w16,w16,w25,ror#11 + eor w9,w9,w4,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w25,ror#25 // Sigma1(e) + eor w10,w10,w21,ror#13 + add w20,w20,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w8,w8,w1,ror#19 + eor w9,w9,w4,lsr#3 // sigma0(X[i+1]) + add w20,w20,w16 // h+=Sigma1(e) + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w10,w21,ror#22 // Sigma0(a) + eor w8,w8,w1,lsr#10 // sigma1(X[i+14]) + add w3,w3,w12 + add w24,w24,w20 // d+=h + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w3,w3,w9 + add w20,w20,w17 // h+=Sigma0(a) + add w3,w3,w8 +.Loop_16_xx: + ldr w8,[sp,#4] + str w11,[sp,#0] + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + ror w10,w5,#7 + and w17,w25,w24 + ror w9,w2,#17 + bic w19,w26,w24 + ror w11,w20,#2 + add w27,w27,w3 // h+=X[i] + eor w16,w16,w24,ror#11 + eor w10,w10,w5,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w24,ror#25 // Sigma1(e) + eor w11,w11,w20,ror#13 + add w27,w27,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w9,w9,w2,ror#19 + eor w10,w10,w5,lsr#3 // sigma0(X[i+1]) + add w27,w27,w16 // h+=Sigma1(e) + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w11,w20,ror#22 // Sigma0(a) + eor w9,w9,w2,lsr#10 // sigma1(X[i+14]) + add w4,w4,w13 + add w23,w23,w27 // d+=h + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w4,w4,w10 + add w27,w27,w17 // h+=Sigma0(a) + add w4,w4,w9 + ldr w9,[sp,#8] + str w12,[sp,#4] + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + ror w11,w6,#7 + and w17,w24,w23 + ror w10,w3,#17 + bic w28,w25,w23 + ror w12,w27,#2 + add w26,w26,w4 // h+=X[i] + eor w16,w16,w23,ror#11 + eor w11,w11,w6,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w23,ror#25 // Sigma1(e) + eor w12,w12,w27,ror#13 + add w26,w26,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w10,w10,w3,ror#19 + eor w11,w11,w6,lsr#3 // sigma0(X[i+1]) + add w26,w26,w16 // h+=Sigma1(e) + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w12,w27,ror#22 // Sigma0(a) + eor w10,w10,w3,lsr#10 // sigma1(X[i+14]) + add w5,w5,w14 + add w22,w22,w26 // d+=h + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w5,w5,w11 + add w26,w26,w17 // h+=Sigma0(a) + add w5,w5,w10 + ldr w10,[sp,#12] + str w13,[sp,#8] + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + ror w12,w7,#7 + and w17,w23,w22 + ror w11,w4,#17 + bic w19,w24,w22 + ror w13,w26,#2 + add w25,w25,w5 // h+=X[i] + eor w16,w16,w22,ror#11 + eor w12,w12,w7,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w22,ror#25 // Sigma1(e) + eor w13,w13,w26,ror#13 + add w25,w25,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w11,w11,w4,ror#19 + eor w12,w12,w7,lsr#3 // sigma0(X[i+1]) + add w25,w25,w16 // h+=Sigma1(e) + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w13,w26,ror#22 // Sigma0(a) + eor w11,w11,w4,lsr#10 // sigma1(X[i+14]) + add w6,w6,w15 + add w21,w21,w25 // d+=h + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w6,w6,w12 + add w25,w25,w17 // h+=Sigma0(a) + add w6,w6,w11 + ldr w11,[sp,#0] + str w14,[sp,#12] + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + ror w13,w8,#7 + and w17,w22,w21 + ror w12,w5,#17 + bic w28,w23,w21 + ror w14,w25,#2 + add w24,w24,w6 // h+=X[i] + eor w16,w16,w21,ror#11 + eor w13,w13,w8,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w21,ror#25 // Sigma1(e) + eor w14,w14,w25,ror#13 + add w24,w24,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w12,w12,w5,ror#19 + eor w13,w13,w8,lsr#3 // sigma0(X[i+1]) + add w24,w24,w16 // h+=Sigma1(e) + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w14,w25,ror#22 // Sigma0(a) + eor w12,w12,w5,lsr#10 // sigma1(X[i+14]) + add w7,w7,w0 + add w20,w20,w24 // d+=h + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w7,w7,w13 + add w24,w24,w17 // h+=Sigma0(a) + add w7,w7,w12 + ldr w12,[sp,#4] + str w15,[sp,#0] + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + ror w14,w9,#7 + and w17,w21,w20 + ror w13,w6,#17 + bic w19,w22,w20 + ror w15,w24,#2 + add w23,w23,w7 // h+=X[i] + eor w16,w16,w20,ror#11 + eor w14,w14,w9,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w20,ror#25 // Sigma1(e) + eor w15,w15,w24,ror#13 + add w23,w23,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w13,w13,w6,ror#19 + eor w14,w14,w9,lsr#3 // sigma0(X[i+1]) + add w23,w23,w16 // h+=Sigma1(e) + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w15,w24,ror#22 // Sigma0(a) + eor w13,w13,w6,lsr#10 // sigma1(X[i+14]) + add w8,w8,w1 + add w27,w27,w23 // d+=h + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w8,w8,w14 + add w23,w23,w17 // h+=Sigma0(a) + add w8,w8,w13 + ldr w13,[sp,#8] + str w0,[sp,#4] + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + ror w15,w10,#7 + and w17,w20,w27 + ror w14,w7,#17 + bic w28,w21,w27 + ror w0,w23,#2 + add w22,w22,w8 // h+=X[i] + eor w16,w16,w27,ror#11 + eor w15,w15,w10,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w27,ror#25 // Sigma1(e) + eor w0,w0,w23,ror#13 + add w22,w22,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w14,w14,w7,ror#19 + eor w15,w15,w10,lsr#3 // sigma0(X[i+1]) + add w22,w22,w16 // h+=Sigma1(e) + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w0,w23,ror#22 // Sigma0(a) + eor w14,w14,w7,lsr#10 // sigma1(X[i+14]) + add w9,w9,w2 + add w26,w26,w22 // d+=h + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w9,w9,w15 + add w22,w22,w17 // h+=Sigma0(a) + add w9,w9,w14 + ldr w14,[sp,#12] + str w1,[sp,#8] + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + ror w0,w11,#7 + and w17,w27,w26 + ror w15,w8,#17 + bic w19,w20,w26 + ror w1,w22,#2 + add w21,w21,w9 // h+=X[i] + eor w16,w16,w26,ror#11 + eor w0,w0,w11,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w26,ror#25 // Sigma1(e) + eor w1,w1,w22,ror#13 + add w21,w21,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w15,w15,w8,ror#19 + eor w0,w0,w11,lsr#3 // sigma0(X[i+1]) + add w21,w21,w16 // h+=Sigma1(e) + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w1,w22,ror#22 // Sigma0(a) + eor w15,w15,w8,lsr#10 // sigma1(X[i+14]) + add w10,w10,w3 + add w25,w25,w21 // d+=h + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w10,w10,w0 + add w21,w21,w17 // h+=Sigma0(a) + add w10,w10,w15 + ldr w15,[sp,#0] + str w2,[sp,#12] + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + ror w1,w12,#7 + and w17,w26,w25 + ror w0,w9,#17 + bic w28,w27,w25 + ror w2,w21,#2 + add w20,w20,w10 // h+=X[i] + eor w16,w16,w25,ror#11 + eor w1,w1,w12,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w25,ror#25 // Sigma1(e) + eor w2,w2,w21,ror#13 + add w20,w20,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w0,w0,w9,ror#19 + eor w1,w1,w12,lsr#3 // sigma0(X[i+1]) + add w20,w20,w16 // h+=Sigma1(e) + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w2,w21,ror#22 // Sigma0(a) + eor w0,w0,w9,lsr#10 // sigma1(X[i+14]) + add w11,w11,w4 + add w24,w24,w20 // d+=h + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w11,w11,w1 + add w20,w20,w17 // h+=Sigma0(a) + add w11,w11,w0 + ldr w0,[sp,#4] + str w3,[sp,#0] + ror w16,w24,#6 + add w27,w27,w19 // h+=K[i] + ror w2,w13,#7 + and w17,w25,w24 + ror w1,w10,#17 + bic w19,w26,w24 + ror w3,w20,#2 + add w27,w27,w11 // h+=X[i] + eor w16,w16,w24,ror#11 + eor w2,w2,w13,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w20,w21 // a^b, b^c in next round + eor w16,w16,w24,ror#25 // Sigma1(e) + eor w3,w3,w20,ror#13 + add w27,w27,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w1,w1,w10,ror#19 + eor w2,w2,w13,lsr#3 // sigma0(X[i+1]) + add w27,w27,w16 // h+=Sigma1(e) + eor w28,w28,w21 // Maj(a,b,c) + eor w17,w3,w20,ror#22 // Sigma0(a) + eor w1,w1,w10,lsr#10 // sigma1(X[i+14]) + add w12,w12,w5 + add w23,w23,w27 // d+=h + add w27,w27,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w12,w12,w2 + add w27,w27,w17 // h+=Sigma0(a) + add w12,w12,w1 + ldr w1,[sp,#8] + str w4,[sp,#4] + ror w16,w23,#6 + add w26,w26,w28 // h+=K[i] + ror w3,w14,#7 + and w17,w24,w23 + ror w2,w11,#17 + bic w28,w25,w23 + ror w4,w27,#2 + add w26,w26,w12 // h+=X[i] + eor w16,w16,w23,ror#11 + eor w3,w3,w14,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w27,w20 // a^b, b^c in next round + eor w16,w16,w23,ror#25 // Sigma1(e) + eor w4,w4,w27,ror#13 + add w26,w26,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w2,w2,w11,ror#19 + eor w3,w3,w14,lsr#3 // sigma0(X[i+1]) + add w26,w26,w16 // h+=Sigma1(e) + eor w19,w19,w20 // Maj(a,b,c) + eor w17,w4,w27,ror#22 // Sigma0(a) + eor w2,w2,w11,lsr#10 // sigma1(X[i+14]) + add w13,w13,w6 + add w22,w22,w26 // d+=h + add w26,w26,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w13,w13,w3 + add w26,w26,w17 // h+=Sigma0(a) + add w13,w13,w2 + ldr w2,[sp,#12] + str w5,[sp,#8] + ror w16,w22,#6 + add w25,w25,w19 // h+=K[i] + ror w4,w15,#7 + and w17,w23,w22 + ror w3,w12,#17 + bic w19,w24,w22 + ror w5,w26,#2 + add w25,w25,w13 // h+=X[i] + eor w16,w16,w22,ror#11 + eor w4,w4,w15,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w26,w27 // a^b, b^c in next round + eor w16,w16,w22,ror#25 // Sigma1(e) + eor w5,w5,w26,ror#13 + add w25,w25,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w3,w3,w12,ror#19 + eor w4,w4,w15,lsr#3 // sigma0(X[i+1]) + add w25,w25,w16 // h+=Sigma1(e) + eor w28,w28,w27 // Maj(a,b,c) + eor w17,w5,w26,ror#22 // Sigma0(a) + eor w3,w3,w12,lsr#10 // sigma1(X[i+14]) + add w14,w14,w7 + add w21,w21,w25 // d+=h + add w25,w25,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w14,w14,w4 + add w25,w25,w17 // h+=Sigma0(a) + add w14,w14,w3 + ldr w3,[sp,#0] + str w6,[sp,#12] + ror w16,w21,#6 + add w24,w24,w28 // h+=K[i] + ror w5,w0,#7 + and w17,w22,w21 + ror w4,w13,#17 + bic w28,w23,w21 + ror w6,w25,#2 + add w24,w24,w14 // h+=X[i] + eor w16,w16,w21,ror#11 + eor w5,w5,w0,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w25,w26 // a^b, b^c in next round + eor w16,w16,w21,ror#25 // Sigma1(e) + eor w6,w6,w25,ror#13 + add w24,w24,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w4,w4,w13,ror#19 + eor w5,w5,w0,lsr#3 // sigma0(X[i+1]) + add w24,w24,w16 // h+=Sigma1(e) + eor w19,w19,w26 // Maj(a,b,c) + eor w17,w6,w25,ror#22 // Sigma0(a) + eor w4,w4,w13,lsr#10 // sigma1(X[i+14]) + add w15,w15,w8 + add w20,w20,w24 // d+=h + add w24,w24,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w15,w15,w5 + add w24,w24,w17 // h+=Sigma0(a) + add w15,w15,w4 + ldr w4,[sp,#4] + str w7,[sp,#0] + ror w16,w20,#6 + add w23,w23,w19 // h+=K[i] + ror w6,w1,#7 + and w17,w21,w20 + ror w5,w14,#17 + bic w19,w22,w20 + ror w7,w24,#2 + add w23,w23,w15 // h+=X[i] + eor w16,w16,w20,ror#11 + eor w6,w6,w1,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w24,w25 // a^b, b^c in next round + eor w16,w16,w20,ror#25 // Sigma1(e) + eor w7,w7,w24,ror#13 + add w23,w23,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w5,w5,w14,ror#19 + eor w6,w6,w1,lsr#3 // sigma0(X[i+1]) + add w23,w23,w16 // h+=Sigma1(e) + eor w28,w28,w25 // Maj(a,b,c) + eor w17,w7,w24,ror#22 // Sigma0(a) + eor w5,w5,w14,lsr#10 // sigma1(X[i+14]) + add w0,w0,w9 + add w27,w27,w23 // d+=h + add w23,w23,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w0,w0,w6 + add w23,w23,w17 // h+=Sigma0(a) + add w0,w0,w5 + ldr w5,[sp,#8] + str w8,[sp,#4] + ror w16,w27,#6 + add w22,w22,w28 // h+=K[i] + ror w7,w2,#7 + and w17,w20,w27 + ror w6,w15,#17 + bic w28,w21,w27 + ror w8,w23,#2 + add w22,w22,w0 // h+=X[i] + eor w16,w16,w27,ror#11 + eor w7,w7,w2,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w23,w24 // a^b, b^c in next round + eor w16,w16,w27,ror#25 // Sigma1(e) + eor w8,w8,w23,ror#13 + add w22,w22,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w6,w6,w15,ror#19 + eor w7,w7,w2,lsr#3 // sigma0(X[i+1]) + add w22,w22,w16 // h+=Sigma1(e) + eor w19,w19,w24 // Maj(a,b,c) + eor w17,w8,w23,ror#22 // Sigma0(a) + eor w6,w6,w15,lsr#10 // sigma1(X[i+14]) + add w1,w1,w10 + add w26,w26,w22 // d+=h + add w22,w22,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w1,w1,w7 + add w22,w22,w17 // h+=Sigma0(a) + add w1,w1,w6 + ldr w6,[sp,#12] + str w9,[sp,#8] + ror w16,w26,#6 + add w21,w21,w19 // h+=K[i] + ror w8,w3,#7 + and w17,w27,w26 + ror w7,w0,#17 + bic w19,w20,w26 + ror w9,w22,#2 + add w21,w21,w1 // h+=X[i] + eor w16,w16,w26,ror#11 + eor w8,w8,w3,ror#18 + orr w17,w17,w19 // Ch(e,f,g) + eor w19,w22,w23 // a^b, b^c in next round + eor w16,w16,w26,ror#25 // Sigma1(e) + eor w9,w9,w22,ror#13 + add w21,w21,w17 // h+=Ch(e,f,g) + and w28,w28,w19 // (b^c)&=(a^b) + eor w7,w7,w0,ror#19 + eor w8,w8,w3,lsr#3 // sigma0(X[i+1]) + add w21,w21,w16 // h+=Sigma1(e) + eor w28,w28,w23 // Maj(a,b,c) + eor w17,w9,w22,ror#22 // Sigma0(a) + eor w7,w7,w0,lsr#10 // sigma1(X[i+14]) + add w2,w2,w11 + add w25,w25,w21 // d+=h + add w21,w21,w28 // h+=Maj(a,b,c) + ldr w28,[x30],#4 // *K++, w19 in next round + add w2,w2,w8 + add w21,w21,w17 // h+=Sigma0(a) + add w2,w2,w7 + ldr w7,[sp,#0] + str w10,[sp,#12] + ror w16,w25,#6 + add w20,w20,w28 // h+=K[i] + ror w9,w4,#7 + and w17,w26,w25 + ror w8,w1,#17 + bic w28,w27,w25 + ror w10,w21,#2 + add w20,w20,w2 // h+=X[i] + eor w16,w16,w25,ror#11 + eor w9,w9,w4,ror#18 + orr w17,w17,w28 // Ch(e,f,g) + eor w28,w21,w22 // a^b, b^c in next round + eor w16,w16,w25,ror#25 // Sigma1(e) + eor w10,w10,w21,ror#13 + add w20,w20,w17 // h+=Ch(e,f,g) + and w19,w19,w28 // (b^c)&=(a^b) + eor w8,w8,w1,ror#19 + eor w9,w9,w4,lsr#3 // sigma0(X[i+1]) + add w20,w20,w16 // h+=Sigma1(e) + eor w19,w19,w22 // Maj(a,b,c) + eor w17,w10,w21,ror#22 // Sigma0(a) + eor w8,w8,w1,lsr#10 // sigma1(X[i+14]) + add w3,w3,w12 + add w24,w24,w20 // d+=h + add w20,w20,w19 // h+=Maj(a,b,c) + ldr w19,[x30],#4 // *K++, w28 in next round + add w3,w3,w9 + add w20,w20,w17 // h+=Sigma0(a) + add w3,w3,w8 + cbnz w19,.Loop_16_xx + + ldp x0,x2,[x29,#12*__SIZEOF_POINTER__] + ldr x1,[x29,#14*__SIZEOF_POINTER__] + sub x30,x30,#260 + + ldp w3,w4,[x0] + ldp w5,w6,[x0,#2*4] + add x1,x1,#14*4 + ldp w7,w8,[x0,#4*4] + add w20,w20,w3 + ldp w9,w10,[x0,#6*4] + add w21,w21,w4 + add w22,w22,w5 + add w23,w23,w6 + stp w20,w21,[x0] + add w24,w24,w7 + add w25,w25,w8 + stp w22,w23,[x0,#2*4] + add w26,w26,w9 + add w27,w27,w10 + cmp x1,x2 + stp w24,w25,[x0,#4*4] + stp w26,w27,[x0,#6*4] + b.ne .Loop + + ldp x19,x20,[x29,#2*__SIZEOF_POINTER__] + add sp,sp,#4*4 + ldp x21,x22,[x29,#4*__SIZEOF_POINTER__] + ldp x23,x24,[x29,#6*__SIZEOF_POINTER__] + ldp x25,x26,[x29,#8*__SIZEOF_POINTER__] + ldp x27,x28,[x29,#10*__SIZEOF_POINTER__] + ldp x29,x30,[sp],#16*__SIZEOF_POINTER__ +.inst 0xd50323bf // autiasp + ret +.size crypton_sha256_asm_block_data_order,.-crypton_sha256_asm_block_data_order + +.align 6 +.type .LK256,%object +.LK256: +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 +.long 0 //terminator +.size .LK256,.-.LK256 +.byte 83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,65,82,77,118,56,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.align 2 +.align 2 +#ifndef __KERNEL__ +.type crypton_sha256_asm_block_armv8,%function +.align 6 +crypton_sha256_asm_block_armv8: +.Lv8_entry: + stp x29,x30,[sp,#-2*__SIZEOF_POINTER__]! + add x29,sp,#0 + + ld1 {v0.4s,v1.4s},[x0] + adr x3,.LK256 + +.Loop_hw: + ld1 {v4.16b,v5.16b,v6.16b,v7.16b},[x1],#64 + sub x2,x2,#1 + ld1 {v16.4s},[x3],#16 + rev32 v4.16b,v4.16b + rev32 v5.16b,v5.16b + rev32 v6.16b,v6.16b + rev32 v7.16b,v7.16b + orr v18.16b,v0.16b,v0.16b // offload + orr v19.16b,v1.16b,v1.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s +.inst 0x5e2828a4 //sha256su0 v4.16b,v5.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.inst 0x5e0760c4 //sha256su1 v4.16b,v6.16b,v7.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s +.inst 0x5e2828c5 //sha256su0 v5.16b,v6.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.inst 0x5e0460e5 //sha256su1 v5.16b,v7.16b,v4.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v6.4s +.inst 0x5e2828e6 //sha256su0 v6.16b,v7.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.inst 0x5e056086 //sha256su1 v6.16b,v4.16b,v5.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v7.4s +.inst 0x5e282887 //sha256su0 v7.16b,v4.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.inst 0x5e0660a7 //sha256su1 v7.16b,v5.16b,v6.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s +.inst 0x5e2828a4 //sha256su0 v4.16b,v5.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.inst 0x5e0760c4 //sha256su1 v4.16b,v6.16b,v7.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s +.inst 0x5e2828c5 //sha256su0 v5.16b,v6.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.inst 0x5e0460e5 //sha256su1 v5.16b,v7.16b,v4.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v6.4s +.inst 0x5e2828e6 //sha256su0 v6.16b,v7.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.inst 0x5e056086 //sha256su1 v6.16b,v4.16b,v5.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v7.4s +.inst 0x5e282887 //sha256su0 v7.16b,v4.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.inst 0x5e0660a7 //sha256su1 v7.16b,v5.16b,v6.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s +.inst 0x5e2828a4 //sha256su0 v4.16b,v5.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.inst 0x5e0760c4 //sha256su1 v4.16b,v6.16b,v7.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s +.inst 0x5e2828c5 //sha256su0 v5.16b,v6.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.inst 0x5e0460e5 //sha256su1 v5.16b,v7.16b,v4.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v6.4s +.inst 0x5e2828e6 //sha256su0 v6.16b,v7.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s +.inst 0x5e056086 //sha256su1 v6.16b,v4.16b,v5.16b + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v7.4s +.inst 0x5e282887 //sha256su0 v7.16b,v4.16b + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s +.inst 0x5e0660a7 //sha256su1 v7.16b,v5.16b,v6.16b + ld1 {v17.4s},[x3],#16 + add v16.4s,v16.4s,v4.4s + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s + + ld1 {v16.4s},[x3],#16 + add v17.4s,v17.4s,v5.4s + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s + + ld1 {v17.4s},[x3] + add v16.4s,v16.4s,v6.4s + sub x3,x3,#64*4-16 + orr v2.16b,v0.16b,v0.16b +.inst 0x5e104020 //sha256h v0.16b,v1.16b,v16.4s +.inst 0x5e105041 //sha256h2 v1.16b,v2.16b,v16.4s + + add v17.4s,v17.4s,v7.4s + orr v2.16b,v0.16b,v0.16b +.inst 0x5e114020 //sha256h v0.16b,v1.16b,v17.4s +.inst 0x5e115041 //sha256h2 v1.16b,v2.16b,v17.4s + + add v0.4s,v0.4s,v18.4s + add v1.4s,v1.4s,v19.4s + + cbnz x2,.Loop_hw + + st1 {v0.4s,v1.4s},[x0] + + ldr x29,[sp],#2*__SIZEOF_POINTER__ + ret +.size crypton_sha256_asm_block_armv8,.-crypton_sha256_asm_block_armv8 +#endif +#ifdef __KERNEL__ +.globl crypton_sha256_asm_block_neon +#endif +.type crypton_sha256_asm_block_neon,%function +.align 4 +crypton_sha256_asm_block_neon: +.Lneon_entry: + stp x29, x30, [sp, #-2*__SIZEOF_POINTER__]! + mov x29, sp + sub sp,sp,#16*4 + + adr x16,.LK256 + add x2,x1,x2,lsl#6 // len to point at the end of inp + + ld1 {v0.16b},[x1], #16 + ld1 {v1.16b},[x1], #16 + ld1 {v2.16b},[x1], #16 + ld1 {v3.16b},[x1], #16 + ld1 {v4.4s},[x16], #16 + ld1 {v5.4s},[x16], #16 + ld1 {v6.4s},[x16], #16 + ld1 {v7.4s},[x16], #16 + rev32 v0.16b,v0.16b // yes, even on + rev32 v1.16b,v1.16b // big-endian + rev32 v2.16b,v2.16b + rev32 v3.16b,v3.16b + mov x17,sp + add v4.4s,v4.4s,v0.4s + add v5.4s,v5.4s,v1.4s + add v6.4s,v6.4s,v2.4s + st1 {v4.4s,v5.4s},[x17], #32 + add v7.4s,v7.4s,v3.4s + st1 {v6.4s,v7.4s},[x17] + sub x17,x17,#32 + + ldp w3,w4,[x0] + ldp w5,w6,[x0,#8] + ldp w7,w8,[x0,#16] + ldp w9,w10,[x0,#24] + ldr w12,[sp,#0] + mov w13,wzr + eor w14,w4,w5 + mov w15,wzr + b .L_00_48 + +.align 4 +.L_00_48: + ext v4.16b,v0.16b,v1.16b,#4 + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + bic w15,w9,w7 + ext v7.16b,v2.16b,v3.16b,#4 + eor w11,w7,w7,ror#5 + add w3,w3,w13 + mov d19,v3.d[1] + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w3,w3,ror#11 + ushr v5.4s,v4.4s,#3 + add w10,w10,w12 + add v0.4s,v0.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + ushr v7.4s,v4.4s,#18 + add w10,w10,w11 + ldr w12,[sp,#4] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w6,w6,w10 + sli v7.4s,v4.4s,#14 + eor w14,w14,w4 + ushr v16.4s,v19.4s,#17 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + eor v5.16b,v5.16b,v7.16b + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + sli v16.4s,v19.4s,#15 + add w10,w10,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + ushr v7.4s,v19.4s,#19 + add w9,w9,w12 + ror w11,w11,#6 + add v0.4s,v0.4s,v5.4s + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + sli v7.4s,v19.4s,#13 + add w9,w9,w11 + ldr w12,[sp,#8] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + eor v17.16b,v17.16b,v7.16b + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + add v0.4s,v0.4s,v17.4s + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + ushr v18.4s,v0.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v0.4s,#10 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + sli v18.4s,v0.4s,#15 + add w8,w8,w12 + ushr v17.4s,v0.4s,#19 + ror w11,w11,#6 + eor w13,w9,w10 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w9,ror#20 + add w8,w8,w11 + sli v17.4s,v0.4s,#13 + ldr w12,[sp,#12] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w4,w4,w8 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w10 + eor v17.16b,v17.16b,v17.16b + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + mov v17.d[1],v19.d[0] + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + add v0.4s,v0.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add v4.4s,v4.4s,v0.4s + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#16] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + ext v4.16b,v1.16b,v2.16b,#4 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + bic w15,w5,w3 + ext v7.16b,v3.16b,v0.16b,#4 + eor w11,w3,w3,ror#5 + add w7,w7,w13 + mov d19,v0.d[1] + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w7,w7,ror#11 + ushr v5.4s,v4.4s,#3 + add w6,w6,w12 + add v1.4s,v1.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + ushr v7.4s,v4.4s,#18 + add w6,w6,w11 + ldr w12,[sp,#20] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w10,w10,w6 + sli v7.4s,v4.4s,#14 + eor w14,w14,w8 + ushr v16.4s,v19.4s,#17 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + eor v5.16b,v5.16b,v7.16b + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + sli v16.4s,v19.4s,#15 + add w6,w6,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + ushr v7.4s,v19.4s,#19 + add w5,w5,w12 + ror w11,w11,#6 + add v1.4s,v1.4s,v5.4s + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + sli v7.4s,v19.4s,#13 + add w5,w5,w11 + ldr w12,[sp,#24] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + eor v17.16b,v17.16b,v7.16b + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + add v1.4s,v1.4s,v17.4s + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + ushr v18.4s,v1.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v1.4s,#10 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + sli v18.4s,v1.4s,#15 + add w4,w4,w12 + ushr v17.4s,v1.4s,#19 + ror w11,w11,#6 + eor w13,w5,w6 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w5,ror#20 + add w4,w4,w11 + sli v17.4s,v1.4s,#13 + ldr w12,[sp,#28] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w8,w8,w4 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w6 + eor v17.16b,v17.16b,v17.16b + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + mov v17.d[1],v19.d[0] + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + add v1.4s,v1.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add v4.4s,v4.4s,v1.4s + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + ldr w12,[sp,#32] + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + ext v4.16b,v2.16b,v3.16b,#4 + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + bic w15,w9,w7 + ext v7.16b,v0.16b,v1.16b,#4 + eor w11,w7,w7,ror#5 + add w3,w3,w13 + mov d19,v1.d[1] + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w3,w3,ror#11 + ushr v5.4s,v4.4s,#3 + add w10,w10,w12 + add v2.4s,v2.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + ushr v7.4s,v4.4s,#18 + add w10,w10,w11 + ldr w12,[sp,#36] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w6,w6,w10 + sli v7.4s,v4.4s,#14 + eor w14,w14,w4 + ushr v16.4s,v19.4s,#17 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + eor v5.16b,v5.16b,v7.16b + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + sli v16.4s,v19.4s,#15 + add w10,w10,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + ushr v7.4s,v19.4s,#19 + add w9,w9,w12 + ror w11,w11,#6 + add v2.4s,v2.4s,v5.4s + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + sli v7.4s,v19.4s,#13 + add w9,w9,w11 + ldr w12,[sp,#40] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + eor v17.16b,v17.16b,v7.16b + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + add v2.4s,v2.4s,v17.4s + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + ushr v18.4s,v2.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v2.4s,#10 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + sli v18.4s,v2.4s,#15 + add w8,w8,w12 + ushr v17.4s,v2.4s,#19 + ror w11,w11,#6 + eor w13,w9,w10 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w9,ror#20 + add w8,w8,w11 + sli v17.4s,v2.4s,#13 + ldr w12,[sp,#44] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w4,w4,w8 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w10 + eor v17.16b,v17.16b,v17.16b + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + mov v17.d[1],v19.d[0] + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + add v2.4s,v2.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add v4.4s,v4.4s,v2.4s + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#48] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + ext v4.16b,v3.16b,v0.16b,#4 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + bic w15,w5,w3 + ext v7.16b,v1.16b,v2.16b,#4 + eor w11,w3,w3,ror#5 + add w7,w7,w13 + mov d19,v2.d[1] + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + ushr v6.4s,v4.4s,#7 + eor w15,w7,w7,ror#11 + ushr v5.4s,v4.4s,#3 + add w6,w6,w12 + add v3.4s,v3.4s,v7.4s + ror w11,w11,#6 + sli v6.4s,v4.4s,#25 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + ushr v7.4s,v4.4s,#18 + add w6,w6,w11 + ldr w12,[sp,#52] + and w14,w14,w13 + eor v5.16b,v5.16b,v6.16b + ror w15,w15,#2 + add w10,w10,w6 + sli v7.4s,v4.4s,#14 + eor w14,w14,w8 + ushr v16.4s,v19.4s,#17 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + eor v5.16b,v5.16b,v7.16b + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + sli v16.4s,v19.4s,#15 + add w6,w6,w14 + orr w12,w12,w15 + ushr v17.4s,v19.4s,#10 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + ushr v7.4s,v19.4s,#19 + add w5,w5,w12 + ror w11,w11,#6 + add v3.4s,v3.4s,v5.4s + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + sli v7.4s,v19.4s,#13 + add w5,w5,w11 + ldr w12,[sp,#56] + and w13,w13,w14 + eor v17.16b,v17.16b,v16.16b + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + eor v17.16b,v17.16b,v7.16b + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + add v3.4s,v3.4s,v17.4s + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + ushr v18.4s,v3.4s,#17 + orr w12,w12,w15 + ushr v19.4s,v3.4s,#10 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + sli v18.4s,v3.4s,#15 + add w4,w4,w12 + ushr v17.4s,v3.4s,#19 + ror w11,w11,#6 + eor w13,w5,w6 + eor v19.16b,v19.16b,v18.16b + eor w15,w15,w5,ror#20 + add w4,w4,w11 + sli v17.4s,v3.4s,#13 + ldr w12,[sp,#60] + and w14,w14,w13 + ror w15,w15,#2 + ld1 {v4.4s},[x16], #16 + add w8,w8,w4 + eor v19.16b,v19.16b,v17.16b + eor w14,w14,w6 + eor v17.16b,v17.16b,v17.16b + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + mov v17.d[1],v19.d[0] + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + add v3.4s,v3.4s,v17.4s + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add v4.4s,v4.4s,v3.4s + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + ldr w12,[x16] + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + cmp w12,#0 // check for K256 terminator + ldr w12,[sp,#0] + sub x17,x17,#64 + bne .L_00_48 + + sub x16,x16,#256 + cmp x1,x2 + mov x17, #-64 + csel x17, x17, xzr, eq + add x1,x1,x17 + mov x17,sp + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + ld1 {v0.16b},[x1],#16 + bic w15,w9,w7 + eor w11,w7,w7,ror#5 + ld1 {v4.4s},[x16],#16 + add w3,w3,w13 + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + eor w15,w3,w3,ror#11 + rev32 v0.16b,v0.16b + add w10,w10,w12 + ror w11,w11,#6 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + add v4.4s,v4.4s,v0.4s + add w10,w10,w11 + ldr w12,[sp,#4] + and w14,w14,w13 + ror w15,w15,#2 + add w6,w6,w10 + eor w14,w14,w4 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + add w10,w10,w14 + orr w12,w12,w15 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + add w9,w9,w12 + ror w11,w11,#6 + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + add w9,w9,w11 + ldr w12,[sp,#8] + and w13,w13,w14 + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + orr w12,w12,w15 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + add w8,w8,w12 + ror w11,w11,#6 + eor w13,w9,w10 + eor w15,w15,w9,ror#20 + add w8,w8,w11 + ldr w12,[sp,#12] + and w14,w14,w13 + ror w15,w15,#2 + add w4,w4,w8 + eor w14,w14,w10 + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#16] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + ld1 {v1.16b},[x1],#16 + bic w15,w5,w3 + eor w11,w3,w3,ror#5 + ld1 {v4.4s},[x16],#16 + add w7,w7,w13 + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + eor w15,w7,w7,ror#11 + rev32 v1.16b,v1.16b + add w6,w6,w12 + ror w11,w11,#6 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + add v4.4s,v4.4s,v1.4s + add w6,w6,w11 + ldr w12,[sp,#20] + and w14,w14,w13 + ror w15,w15,#2 + add w10,w10,w6 + eor w14,w14,w8 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + add w6,w6,w14 + orr w12,w12,w15 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + add w5,w5,w12 + ror w11,w11,#6 + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + add w5,w5,w11 + ldr w12,[sp,#24] + and w13,w13,w14 + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + orr w12,w12,w15 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + add w4,w4,w12 + ror w11,w11,#6 + eor w13,w5,w6 + eor w15,w15,w5,ror#20 + add w4,w4,w11 + ldr w12,[sp,#28] + and w14,w14,w13 + ror w15,w15,#2 + add w8,w8,w4 + eor w14,w14,w6 + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + ldr w12,[sp,#32] + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + add w10,w10,w12 + add w3,w3,w15 + and w12,w8,w7 + ld1 {v2.16b},[x1],#16 + bic w15,w9,w7 + eor w11,w7,w7,ror#5 + ld1 {v4.4s},[x16],#16 + add w3,w3,w13 + orr w12,w12,w15 + eor w11,w11,w7,ror#19 + eor w15,w3,w3,ror#11 + rev32 v2.16b,v2.16b + add w10,w10,w12 + ror w11,w11,#6 + eor w13,w3,w4 + eor w15,w15,w3,ror#20 + add v4.4s,v4.4s,v2.4s + add w10,w10,w11 + ldr w12,[sp,#36] + and w14,w14,w13 + ror w15,w15,#2 + add w6,w6,w10 + eor w14,w14,w4 + add w9,w9,w12 + add w10,w10,w15 + and w12,w7,w6 + bic w15,w8,w6 + eor w11,w6,w6,ror#5 + add w10,w10,w14 + orr w12,w12,w15 + eor w11,w11,w6,ror#19 + eor w15,w10,w10,ror#11 + add w9,w9,w12 + ror w11,w11,#6 + eor w14,w10,w3 + eor w15,w15,w10,ror#20 + add w9,w9,w11 + ldr w12,[sp,#40] + and w13,w13,w14 + ror w15,w15,#2 + add w5,w5,w9 + eor w13,w13,w3 + add w8,w8,w12 + add w9,w9,w15 + and w12,w6,w5 + bic w15,w7,w5 + eor w11,w5,w5,ror#5 + add w9,w9,w13 + orr w12,w12,w15 + eor w11,w11,w5,ror#19 + eor w15,w9,w9,ror#11 + add w8,w8,w12 + ror w11,w11,#6 + eor w13,w9,w10 + eor w15,w15,w9,ror#20 + add w8,w8,w11 + ldr w12,[sp,#44] + and w14,w14,w13 + ror w15,w15,#2 + add w4,w4,w8 + eor w14,w14,w10 + add w7,w7,w12 + add w8,w8,w15 + and w12,w5,w4 + bic w15,w6,w4 + eor w11,w4,w4,ror#5 + add w8,w8,w14 + orr w12,w12,w15 + eor w11,w11,w4,ror#19 + eor w15,w8,w8,ror#11 + add w7,w7,w12 + ror w11,w11,#6 + eor w14,w8,w9 + eor w15,w15,w8,ror#20 + add w7,w7,w11 + ldr w12,[sp,#48] + and w13,w13,w14 + ror w15,w15,#2 + add w3,w3,w7 + eor w13,w13,w9 + st1 {v4.4s},[x17], #16 + add w6,w6,w12 + add w7,w7,w15 + and w12,w4,w3 + ld1 {v3.16b},[x1],#16 + bic w15,w5,w3 + eor w11,w3,w3,ror#5 + ld1 {v4.4s},[x16],#16 + add w7,w7,w13 + orr w12,w12,w15 + eor w11,w11,w3,ror#19 + eor w15,w7,w7,ror#11 + rev32 v3.16b,v3.16b + add w6,w6,w12 + ror w11,w11,#6 + eor w13,w7,w8 + eor w15,w15,w7,ror#20 + add v4.4s,v4.4s,v3.4s + add w6,w6,w11 + ldr w12,[sp,#52] + and w14,w14,w13 + ror w15,w15,#2 + add w10,w10,w6 + eor w14,w14,w8 + add w5,w5,w12 + add w6,w6,w15 + and w12,w3,w10 + bic w15,w4,w10 + eor w11,w10,w10,ror#5 + add w6,w6,w14 + orr w12,w12,w15 + eor w11,w11,w10,ror#19 + eor w15,w6,w6,ror#11 + add w5,w5,w12 + ror w11,w11,#6 + eor w14,w6,w7 + eor w15,w15,w6,ror#20 + add w5,w5,w11 + ldr w12,[sp,#56] + and w13,w13,w14 + ror w15,w15,#2 + add w9,w9,w5 + eor w13,w13,w7 + add w4,w4,w12 + add w5,w5,w15 + and w12,w10,w9 + bic w15,w3,w9 + eor w11,w9,w9,ror#5 + add w5,w5,w13 + orr w12,w12,w15 + eor w11,w11,w9,ror#19 + eor w15,w5,w5,ror#11 + add w4,w4,w12 + ror w11,w11,#6 + eor w13,w5,w6 + eor w15,w15,w5,ror#20 + add w4,w4,w11 + ldr w12,[sp,#60] + and w14,w14,w13 + ror w15,w15,#2 + add w8,w8,w4 + eor w14,w14,w6 + add w3,w3,w12 + add w4,w4,w15 + and w12,w9,w8 + bic w15,w10,w8 + eor w11,w8,w8,ror#5 + add w4,w4,w14 + orr w12,w12,w15 + eor w11,w11,w8,ror#19 + eor w15,w4,w4,ror#11 + add w3,w3,w12 + ror w11,w11,#6 + eor w14,w4,w5 + eor w15,w15,w4,ror#20 + add w3,w3,w11 + and w13,w13,w14 + ror w15,w15,#2 + add w7,w7,w3 + eor w13,w13,w5 + st1 {v4.4s},[x17], #16 + add w3,w3,w15 // h+=Sigma0(a) from the past + ldp w11,w12,[x0,#0] + add w3,w3,w13 // h+=Maj(a,b,c) from the past + ldp w13,w14,[x0,#8] + add w3,w3,w11 // accumulate + add w4,w4,w12 + ldp w11,w12,[x0,#16] + add w5,w5,w13 + add w6,w6,w14 + ldp w13,w14,[x0,#24] + add w7,w7,w11 + add w8,w8,w12 + ldr w12,[sp,#0] + stp w3,w4,[x0,#0] + add w9,w9,w13 + mov w13,wzr + stp w5,w6,[x0,#8] + add w10,w10,w14 + stp w7,w8,[x0,#16] + eor w14,w4,w5 + stp w9,w10,[x0,#24] + mov w15,wzr + mov x17,sp + b.ne .L_00_48 + + ldr x29,[x29] + add sp,sp,#16*4+2*__SIZEOF_POINTER__ + ret +.size crypton_sha256_asm_block_neon,.-crypton_sha256_asm_block_neon +#if !defined(__KERNEL__) && !defined(_WIN64) +.comm crypton_armcap_P,4,4 +.hidden crypton_armcap_P +#endif + +.section .note.GNU-stack,"",%progbits diff --git a/cbits/asm/sha256-x86_64-elf.S b/cbits/asm/sha256-x86_64-elf.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha256-x86_64-elf.S @@ -0,0 +1,5463 @@ +.text + + +.globl crypton_sha256_asm_block_data_order +.type crypton_sha256_asm_block_data_order,@function +.align 16 +crypton_sha256_asm_block_data_order: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + leaq crypton_ia32cap_P(%rip),%rax + movl 0(%rax),%r9d + movl 4(%rax),%r10d + movl 8(%rax),%eax + testl $536870912,%eax + jnz .Lshaext_shortcut + andl $296,%eax + cmpl $296,%eax + je .Lavx2_shortcut + andl $1073741824,%r9d + andl $268435968,%r10d + orl %r9d,%r10d + cmpl $1342177792,%r10d + je .Lavx_shortcut + testl $512,%r10d + jnz .Lssse3_shortcut + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $64+24,%rsp + +.cfi_def_cfa %rsp,144 + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,64+0(%rsp) + movq %rsi,64+8(%rsp) + movq %rdx,64+16(%rsp) + + movl 0(%rdi),%eax + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + jmp .Lloop + +.align 16 +.Lloop: + movl %ebx,%edi + leaq K256(%rip),%rbp + xorl %ecx,%edi + movl 0(%rsi),%r12d + movl %r8d,%r13d + movl %eax,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,0(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + addl %r14d,%r11d + movl 4(%rsi),%r12d + movl %edx,%r13d + movl %r11d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,4(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + addl %r14d,%r10d + movl 8(%rsi),%r12d + movl %ecx,%r13d + movl %r10d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,8(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + addl %r14d,%r9d + movl 12(%rsi),%r12d + movl %ebx,%r13d + movl %r9d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,12(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + addl %r14d,%r8d + movl 16(%rsi),%r12d + movl %eax,%r13d + movl %r8d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,16(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + addl %r14d,%edx + movl 20(%rsi),%r12d + movl %r11d,%r13d + movl %edx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,20(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + addl %r14d,%ecx + movl 24(%rsi),%r12d + movl %r10d,%r13d + movl %ecx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,24(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + addl %r14d,%ebx + movl 28(%rsi),%r12d + movl %r9d,%r13d + movl %ebx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,28(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + addl %r14d,%eax + movl 32(%rsi),%r12d + movl %r8d,%r13d + movl %eax,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,32(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + addl %r14d,%r11d + movl 36(%rsi),%r12d + movl %edx,%r13d + movl %r11d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,36(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + addl %r14d,%r10d + movl 40(%rsi),%r12d + movl %ecx,%r13d + movl %r10d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,40(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + addl %r14d,%r9d + movl 44(%rsi),%r12d + movl %ebx,%r13d + movl %r9d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,44(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + addl %r14d,%r8d + movl 48(%rsi),%r12d + movl %eax,%r13d + movl %r8d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,48(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + addl %r14d,%edx + movl 52(%rsi),%r12d + movl %r11d,%r13d + movl %edx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,52(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + addl %r14d,%ecx + movl 56(%rsi),%r12d + movl %r10d,%r13d + movl %ecx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,56(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + addl %r14d,%ebx + movl 60(%rsi),%r12d + movl %r9d,%r13d + movl %ebx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,60(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + jmp .Lrounds_16_xx +.align 16 +.Lrounds_16_xx: + movl 4(%rsp),%r13d + movl 56(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%eax + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 36(%rsp),%r12d + + addl 0(%rsp),%r12d + movl %r8d,%r13d + addl %r15d,%r12d + movl %eax,%r14d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,0(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + movl 8(%rsp),%r13d + movl 60(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r11d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 40(%rsp),%r12d + + addl 4(%rsp),%r12d + movl %edx,%r13d + addl %edi,%r12d + movl %r11d,%r14d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,4(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + movl 12(%rsp),%r13d + movl 0(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r10d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 44(%rsp),%r12d + + addl 8(%rsp),%r12d + movl %ecx,%r13d + addl %r15d,%r12d + movl %r10d,%r14d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,8(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + movl 16(%rsp),%r13d + movl 4(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r9d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 48(%rsp),%r12d + + addl 12(%rsp),%r12d + movl %ebx,%r13d + addl %edi,%r12d + movl %r9d,%r14d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,12(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + movl 20(%rsp),%r13d + movl 8(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r8d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 52(%rsp),%r12d + + addl 16(%rsp),%r12d + movl %eax,%r13d + addl %r15d,%r12d + movl %r8d,%r14d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,16(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + movl 24(%rsp),%r13d + movl 12(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%edx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 56(%rsp),%r12d + + addl 20(%rsp),%r12d + movl %r11d,%r13d + addl %edi,%r12d + movl %edx,%r14d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,20(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + movl 28(%rsp),%r13d + movl 16(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ecx + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 60(%rsp),%r12d + + addl 24(%rsp),%r12d + movl %r10d,%r13d + addl %r15d,%r12d + movl %ecx,%r14d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,24(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + movl 32(%rsp),%r13d + movl 20(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ebx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 0(%rsp),%r12d + + addl 28(%rsp),%r12d + movl %r9d,%r13d + addl %edi,%r12d + movl %ebx,%r14d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,28(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + movl 36(%rsp),%r13d + movl 24(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%eax + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 4(%rsp),%r12d + + addl 32(%rsp),%r12d + movl %r8d,%r13d + addl %r15d,%r12d + movl %eax,%r14d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,32(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + movl 40(%rsp),%r13d + movl 28(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r11d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 8(%rsp),%r12d + + addl 36(%rsp),%r12d + movl %edx,%r13d + addl %edi,%r12d + movl %r11d,%r14d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,36(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + movl 44(%rsp),%r13d + movl 32(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r10d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 12(%rsp),%r12d + + addl 40(%rsp),%r12d + movl %ecx,%r13d + addl %r15d,%r12d + movl %r10d,%r14d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,40(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + movl 48(%rsp),%r13d + movl 36(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r9d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 16(%rsp),%r12d + + addl 44(%rsp),%r12d + movl %ebx,%r13d + addl %edi,%r12d + movl %r9d,%r14d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,44(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + movl 52(%rsp),%r13d + movl 40(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r8d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 20(%rsp),%r12d + + addl 48(%rsp),%r12d + movl %eax,%r13d + addl %r15d,%r12d + movl %r8d,%r14d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,48(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + movl 56(%rsp),%r13d + movl 44(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%edx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 24(%rsp),%r12d + + addl 52(%rsp),%r12d + movl %r11d,%r13d + addl %edi,%r12d + movl %edx,%r14d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,52(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + movl 60(%rsp),%r13d + movl 48(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ecx + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 28(%rsp),%r12d + + addl 56(%rsp),%r12d + movl %r10d,%r13d + addl %r15d,%r12d + movl %ecx,%r14d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,56(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + movl 0(%rsp),%r13d + movl 52(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ebx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 32(%rsp),%r12d + + addl 60(%rsp),%r12d + movl %r9d,%r13d + addl %edi,%r12d + movl %ebx,%r14d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,60(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + cmpb $0,3(%rbp) + jnz .Lrounds_16_xx + + movq 64+0(%rsp),%rdi + addl %r14d,%eax + leaq 64(%rsi),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + cmpq 64+16(%rsp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb .Lloop + + leaq 64+24+48(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq 64+24(%rsp),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbx + movq -8(%r11),%rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + leaq (%r11),%rsp + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha256_asm_block_data_order,.-crypton_sha256_asm_block_data_order +.align 64 +.type K256,@object +K256: +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 + +.long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f +.long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f +.long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff +.long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff +.long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 +.long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 +.byte 83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.type crypton_sha256_asm_block_data_order_shaext,@function +.align 64 +crypton_sha256_asm_block_data_order_shaext: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lshaext_shortcut: + + leaq K256+128(%rip),%rcx + movdqu (%rdi),%xmm1 + movdqu 16(%rdi),%xmm2 + movdqa 512-128(%rcx),%xmm7 + + pshufd $0x1b,%xmm1,%xmm0 + pshufd $0xb1,%xmm1,%xmm1 + pshufd $0x1b,%xmm2,%xmm2 + movdqa %xmm7,%xmm8 +.byte 102,15,58,15,202,8 + punpcklqdq %xmm0,%xmm2 + jmp .Loop_shaext + +.align 16 +.Loop_shaext: + movdqu (%rsi),%xmm3 + movdqu 16(%rsi),%xmm4 + movdqu 32(%rsi),%xmm5 +.byte 102,15,56,0,223 + movdqu 48(%rsi),%xmm6 + + movdqa 0-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 102,15,56,0,231 + movdqa %xmm2,%xmm10 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + nop + movdqa %xmm1,%xmm9 +.byte 15,56,203,202 + + movdqa 32-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 102,15,56,0,239 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + leaq 64(%rsi),%rsi +.byte 15,56,204,220 +.byte 15,56,203,202 + + movdqa 64-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 102,15,56,0,247 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + + movdqa 96-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 128-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 160-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 + nop + paddd %xmm7,%xmm6 +.byte 15,56,204,220 +.byte 15,56,203,202 + movdqa 192-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,205,245 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + movdqa 224-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 256-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 288-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 + nop + paddd %xmm7,%xmm6 +.byte 15,56,204,220 +.byte 15,56,203,202 + movdqa 320-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,205,245 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + movdqa 352-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 384-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 416-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 +.byte 15,56,203,202 + paddd %xmm7,%xmm6 + + movdqa 448-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 +.byte 15,56,205,245 + movdqa %xmm8,%xmm7 +.byte 15,56,203,202 + + movdqa 480-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 + nop +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + decq %rdx + nop +.byte 15,56,203,202 + + paddd %xmm10,%xmm2 + paddd %xmm9,%xmm1 + jnz .Loop_shaext + + pshufd $0xb1,%xmm2,%xmm2 + pshufd $0x1b,%xmm1,%xmm7 + pshufd $0xb1,%xmm1,%xmm1 + punpckhqdq %xmm2,%xmm1 +.byte 102,15,58,15,215,8 + + movdqu %xmm1,(%rdi) + movdqu %xmm2,16(%rdi) +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp + + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha256_asm_block_data_order_shaext,.-crypton_sha256_asm_block_data_order_shaext +.type crypton_sha256_asm_block_data_order_ssse3,@function +.align 64 +crypton_sha256_asm_block_data_order_ssse3: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lssse3_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -64(%rsp),%rsp + movl 0(%rdi),%eax + andq $-64,%rsp + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + + + jmp .Lloop_ssse3 +.align 16 +.Lloop_ssse3: + movdqa K256+512(%rip),%xmm7 + movq %rsi,-56(%rbp) + movdqu 0(%rsi),%xmm0 + movdqu 16(%rsi),%xmm1 + movdqu 32(%rsi),%xmm2 +.byte 102,15,56,0,199 + movdqu 48(%rsi),%xmm3 + leaq K256(%rip),%rsi +.byte 102,15,56,0,207 + movdqa 0(%rsi),%xmm4 + movdqa 32(%rsi),%xmm5 +.byte 102,15,56,0,215 + paddd %xmm0,%xmm4 + movdqa 64(%rsi),%xmm6 +.byte 102,15,56,0,223 + movdqa 96(%rsi),%xmm7 + paddd %xmm1,%xmm5 + paddd %xmm2,%xmm6 + paddd %xmm3,%xmm7 + movdqa %xmm4,0(%rsp) + movl %eax,%r14d + movdqa %xmm5,16(%rsp) + movl %ebx,%edi + movdqa %xmm6,32(%rsp) + xorl %ecx,%edi + movdqa %xmm7,48(%rsp) + movl %r8d,%r13d + jmp .Lssse3_00_47 + +.align 16 +.Lssse3_00_47: + subq $-128,%rsi + rorl $14,%r13d + movdqa %xmm1,%xmm4 + movl %r14d,%eax + movl %r9d,%r12d + movdqa %xmm3,%xmm7 + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d +.byte 102,15,58,15,224,4 + andl %r8d,%r12d + xorl %r8d,%r13d +.byte 102,15,58,15,250,4 + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %ebx,%r15d + addl %r12d,%r11d + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + paddd %xmm7,%xmm0 + rorl $2,%r14d + addl %r11d,%edx + psrld $7,%xmm6 + addl %edi,%r11d + movl %edx,%r13d + pshufd $250,%xmm3,%xmm7 + addl %r11d,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%r11d + movl %r8d,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %r11d,%r14d + pxor %xmm5,%xmm4 + andl %edx,%r12d + xorl %edx,%r13d + pslld $11,%xmm5 + addl 4(%rsp),%r10d + movl %r11d,%edi + pxor %xmm6,%xmm4 + xorl %r9d,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %eax,%edi + addl %r12d,%r10d + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + psrld $10,%xmm7 + addl %r13d,%r10d + xorl %eax,%r15d + paddd %xmm4,%xmm0 + rorl $2,%r14d + addl %r10d,%ecx + psrlq $17,%xmm6 + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %ecx,%r13d + xorl %r8d,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + psrldq $8,%xmm7 + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + paddd %xmm7,%xmm0 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + pshufd $80,%xmm0,%xmm7 + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + movdqa %xmm7,%xmm6 + addl %edi,%r9d + movl %ebx,%r13d + psrld $10,%xmm7 + addl %r9d,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%r9d + movl %ecx,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + psrlq $2,%xmm6 + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + pxor %xmm6,%xmm7 + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %r10d,%edi + addl %r12d,%r8d + movdqa 0(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + paddd %xmm7,%xmm0 + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + paddd %xmm0,%xmm6 + movl %eax,%r13d + addl %r8d,%r14d + movdqa %xmm6,0(%rsp) + rorl $14,%r13d + movdqa %xmm2,%xmm4 + movl %r14d,%r8d + movl %ebx,%r12d + movdqa %xmm0,%xmm7 + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d +.byte 102,15,58,15,225,4 + andl %eax,%r12d + xorl %eax,%r13d +.byte 102,15,58,15,251,4 + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %r9d,%r15d + addl %r12d,%edx + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + paddd %xmm7,%xmm1 + rorl $2,%r14d + addl %edx,%r11d + psrld $7,%xmm6 + addl %edi,%edx + movl %r11d,%r13d + pshufd $250,%xmm0,%xmm7 + addl %edx,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%edx + movl %eax,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %edx,%r14d + pxor %xmm5,%xmm4 + andl %r11d,%r12d + xorl %r11d,%r13d + pslld $11,%xmm5 + addl 20(%rsp),%ecx + movl %edx,%edi + pxor %xmm6,%xmm4 + xorl %ebx,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %r8d,%edi + addl %r12d,%ecx + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + psrld $10,%xmm7 + addl %r13d,%ecx + xorl %r8d,%r15d + paddd %xmm4,%xmm1 + rorl $2,%r14d + addl %ecx,%r10d + psrlq $17,%xmm6 + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %r10d,%r13d + xorl %eax,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + psrldq $8,%xmm7 + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + paddd %xmm7,%xmm1 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + pshufd $80,%xmm1,%xmm7 + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + movdqa %xmm7,%xmm6 + addl %edi,%ebx + movl %r9d,%r13d + psrld $10,%xmm7 + addl %ebx,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%ebx + movl %r10d,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + psrlq $2,%xmm6 + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + pxor %xmm6,%xmm7 + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %ecx,%edi + addl %r12d,%eax + movdqa 32(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + paddd %xmm7,%xmm1 + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + paddd %xmm1,%xmm6 + movl %r8d,%r13d + addl %eax,%r14d + movdqa %xmm6,16(%rsp) + rorl $14,%r13d + movdqa %xmm3,%xmm4 + movl %r14d,%eax + movl %r9d,%r12d + movdqa %xmm1,%xmm7 + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d +.byte 102,15,58,15,226,4 + andl %r8d,%r12d + xorl %r8d,%r13d +.byte 102,15,58,15,248,4 + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %ebx,%r15d + addl %r12d,%r11d + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + paddd %xmm7,%xmm2 + rorl $2,%r14d + addl %r11d,%edx + psrld $7,%xmm6 + addl %edi,%r11d + movl %edx,%r13d + pshufd $250,%xmm1,%xmm7 + addl %r11d,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%r11d + movl %r8d,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %r11d,%r14d + pxor %xmm5,%xmm4 + andl %edx,%r12d + xorl %edx,%r13d + pslld $11,%xmm5 + addl 36(%rsp),%r10d + movl %r11d,%edi + pxor %xmm6,%xmm4 + xorl %r9d,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %eax,%edi + addl %r12d,%r10d + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + psrld $10,%xmm7 + addl %r13d,%r10d + xorl %eax,%r15d + paddd %xmm4,%xmm2 + rorl $2,%r14d + addl %r10d,%ecx + psrlq $17,%xmm6 + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %ecx,%r13d + xorl %r8d,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + psrldq $8,%xmm7 + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + paddd %xmm7,%xmm2 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + pshufd $80,%xmm2,%xmm7 + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + movdqa %xmm7,%xmm6 + addl %edi,%r9d + movl %ebx,%r13d + psrld $10,%xmm7 + addl %r9d,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%r9d + movl %ecx,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + psrlq $2,%xmm6 + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + pxor %xmm6,%xmm7 + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %r10d,%edi + addl %r12d,%r8d + movdqa 64(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + paddd %xmm7,%xmm2 + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + paddd %xmm2,%xmm6 + movl %eax,%r13d + addl %r8d,%r14d + movdqa %xmm6,32(%rsp) + rorl $14,%r13d + movdqa %xmm0,%xmm4 + movl %r14d,%r8d + movl %ebx,%r12d + movdqa %xmm2,%xmm7 + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d +.byte 102,15,58,15,227,4 + andl %eax,%r12d + xorl %eax,%r13d +.byte 102,15,58,15,249,4 + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %r9d,%r15d + addl %r12d,%edx + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + paddd %xmm7,%xmm3 + rorl $2,%r14d + addl %edx,%r11d + psrld $7,%xmm6 + addl %edi,%edx + movl %r11d,%r13d + pshufd $250,%xmm2,%xmm7 + addl %edx,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%edx + movl %eax,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %edx,%r14d + pxor %xmm5,%xmm4 + andl %r11d,%r12d + xorl %r11d,%r13d + pslld $11,%xmm5 + addl 52(%rsp),%ecx + movl %edx,%edi + pxor %xmm6,%xmm4 + xorl %ebx,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %r8d,%edi + addl %r12d,%ecx + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + psrld $10,%xmm7 + addl %r13d,%ecx + xorl %r8d,%r15d + paddd %xmm4,%xmm3 + rorl $2,%r14d + addl %ecx,%r10d + psrlq $17,%xmm6 + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %r10d,%r13d + xorl %eax,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + psrldq $8,%xmm7 + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + paddd %xmm7,%xmm3 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + pshufd $80,%xmm3,%xmm7 + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + movdqa %xmm7,%xmm6 + addl %edi,%ebx + movl %r9d,%r13d + psrld $10,%xmm7 + addl %ebx,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%ebx + movl %r10d,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + psrlq $2,%xmm6 + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + pxor %xmm6,%xmm7 + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %ecx,%edi + addl %r12d,%eax + movdqa 96(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + paddd %xmm7,%xmm3 + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + paddd %xmm3,%xmm6 + movl %r8d,%r13d + addl %eax,%r14d + movdqa %xmm6,48(%rsp) + cmpb $0,131(%rsi) + jne .Lssse3_00_47 + rorl $14,%r13d + movl %r14d,%eax + movl %r9d,%r12d + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + rorl $6,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + rorl $2,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + rorl $14,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + rorl $11,%r14d + xorl %eax,%edi + addl %r12d,%r10d + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + rorl $2,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + rorl $14,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + rorl $6,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + rorl $14,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + rorl $6,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + rorl $2,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + rorl $14,%r13d + movl %r14d,%edx + movl %eax,%r12d + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + rorl $11,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + rorl $2,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + rorl $14,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + xorl %ecx,%edi + addl %r12d,%eax + rorl $6,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + rorl $14,%r13d + movl %r14d,%eax + movl %r9d,%r12d + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + rorl $6,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + rorl $2,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + rorl $14,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + rorl $11,%r14d + xorl %eax,%edi + addl %r12d,%r10d + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + rorl $2,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + rorl $14,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + rorl $6,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + rorl $14,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + rorl $6,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + rorl $2,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + rorl $14,%r13d + movl %r14d,%edx + movl %eax,%r12d + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + rorl $11,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + rorl $2,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + rorl $14,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + xorl %ecx,%edi + addl %r12d,%eax + rorl $6,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + movq -64(%rbp),%rdi + movl %r14d,%eax + movq -56(%rbp),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + leaq 64(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb .Lloop_ssse3 + + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha256_asm_block_data_order_ssse3,.-crypton_sha256_asm_block_data_order_ssse3 +.type crypton_sha256_asm_block_data_order_avx,@function +.align 64 +crypton_sha256_asm_block_data_order_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lavx_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -64(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movl 0(%rdi),%eax + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + vmovdqa K256+512+32(%rip),%xmm8 + vmovdqa K256+512+64(%rip),%xmm9 + jmp .Lloop_avx +.align 16 +.Lloop_avx: + vmovdqa K256+512(%rip),%xmm7 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm7,%xmm0,%xmm0 + leaq K256(%rip),%rsi + vpshufb %xmm7,%xmm1,%xmm1 + vpshufb %xmm7,%xmm2,%xmm2 + vpaddd 0(%rsi),%xmm0,%xmm4 + vpshufb %xmm7,%xmm3,%xmm3 + vpaddd 32(%rsi),%xmm1,%xmm5 + vpaddd 64(%rsi),%xmm2,%xmm6 + vpaddd 96(%rsi),%xmm3,%xmm7 + vmovdqa %xmm4,0(%rsp) + movl %eax,%r14d + vmovdqa %xmm5,16(%rsp) + movl %ebx,%edi + vmovdqa %xmm6,32(%rsp) + xorl %ecx,%edi + vmovdqa %xmm7,48(%rsp) + movl %r8d,%r13d + jmp .Lavx_00_47 + +.align 16 +.Lavx_00_47: + subq $-128,%rsi + vpalignr $4,%xmm0,%xmm1,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + vpalignr $4,%xmm2,%xmm3,%xmm7 + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + vpaddd %xmm7,%xmm0,%xmm0 + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + vpshufd $250,%xmm3,%xmm7 + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + vpsrld $11,%xmm6,%xmm6 + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + vpaddd %xmm4,%xmm0,%xmm0 + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + vpxor %xmm7,%xmm6,%xmm6 + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + vpaddd %xmm6,%xmm0,%xmm0 + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + vpshufd $80,%xmm0,%xmm7 + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + vpxor %xmm7,%xmm6,%xmm6 + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + vpaddd %xmm6,%xmm0,%xmm0 + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + vpaddd 0(%rsi),%xmm0,%xmm6 + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + vmovdqa %xmm6,0(%rsp) + vpalignr $4,%xmm1,%xmm2,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + vpalignr $4,%xmm3,%xmm0,%xmm7 + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + vpaddd %xmm7,%xmm1,%xmm1 + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + vpshufd $250,%xmm0,%xmm7 + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + vpsrld $11,%xmm6,%xmm6 + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + vpaddd %xmm4,%xmm1,%xmm1 + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + vpxor %xmm7,%xmm6,%xmm6 + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + vpaddd %xmm6,%xmm1,%xmm1 + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + vpshufd $80,%xmm1,%xmm7 + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + vpxor %xmm7,%xmm6,%xmm6 + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + vpaddd %xmm6,%xmm1,%xmm1 + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + vpaddd 32(%rsi),%xmm1,%xmm6 + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + vmovdqa %xmm6,16(%rsp) + vpalignr $4,%xmm2,%xmm3,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + vpalignr $4,%xmm0,%xmm1,%xmm7 + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + vpaddd %xmm7,%xmm2,%xmm2 + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + vpshufd $250,%xmm1,%xmm7 + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + vpsrld $11,%xmm6,%xmm6 + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + vpaddd %xmm4,%xmm2,%xmm2 + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + vpxor %xmm7,%xmm6,%xmm6 + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + vpaddd %xmm6,%xmm2,%xmm2 + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + vpshufd $80,%xmm2,%xmm7 + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + vpxor %xmm7,%xmm6,%xmm6 + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + vpaddd %xmm6,%xmm2,%xmm2 + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + vpaddd 64(%rsi),%xmm2,%xmm6 + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + vmovdqa %xmm6,32(%rsp) + vpalignr $4,%xmm3,%xmm0,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + vpalignr $4,%xmm1,%xmm2,%xmm7 + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + vpaddd %xmm7,%xmm3,%xmm3 + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + vpshufd $250,%xmm2,%xmm7 + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + vpsrld $11,%xmm6,%xmm6 + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + vpaddd %xmm4,%xmm3,%xmm3 + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + vpxor %xmm7,%xmm6,%xmm6 + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + vpaddd %xmm6,%xmm3,%xmm3 + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + vpshufd $80,%xmm3,%xmm7 + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + vpxor %xmm7,%xmm6,%xmm6 + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + vpaddd %xmm6,%xmm3,%xmm3 + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + vpaddd 96(%rsi),%xmm3,%xmm6 + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + vmovdqa %xmm6,48(%rsp) + cmpb $0,131(%rsi) + jne .Lavx_00_47 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + movq -64(%rbp),%rdi + movl %r14d,%eax + movq -56(%rbp),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + leaq 64(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb .Lloop_avx + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha256_asm_block_data_order_avx,.-crypton_sha256_asm_block_data_order_avx +.type crypton_sha256_asm_block_data_order_avx2,@function +.align 64 +crypton_sha256_asm_block_data_order_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lavx2_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -64(%rsp),%rsp + vzeroupper + andq $-64,%rsp + subq $-64,%rsi + movl 0(%rdi),%eax + movq %rsi,%r12 + movl 4(%rdi),%ebx + cmpq %rdx,%rsi + movl 8(%rdi),%ecx + cmoveq %rsp,%r12 + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + vmovdqa K256+512+32(%rip),%ymm8 + vmovdqa K256+512+64(%rip),%ymm9 + jmp .Loop_avx2 +.align 16 +.Loop_avx2: + vmovdqa K256+512(%rip),%ymm7 + movq %rsi,-56(%rbp) + vmovdqu -64+0(%rsi),%xmm0 + vmovdqu -64+16(%rsi),%xmm1 + vmovdqu -64+32(%rsi),%xmm2 + vmovdqu -64+48(%rsi),%xmm3 + leaq K256(%rip),%rsi + vinserti128 $1,(%r12),%ymm0,%ymm0 + vinserti128 $1,16(%r12),%ymm1,%ymm1 + vpshufb %ymm7,%ymm0,%ymm0 + vinserti128 $1,32(%r12),%ymm2,%ymm2 + vpshufb %ymm7,%ymm1,%ymm1 + vinserti128 $1,48(%r12),%ymm3,%ymm3 + + vpshufb %ymm7,%ymm2,%ymm2 + vpaddd 0(%rsi),%ymm0,%ymm4 + vpshufb %ymm7,%ymm3,%ymm3 + vpaddd 32(%rsi),%ymm1,%ymm5 + vpaddd 64(%rsi),%ymm2,%ymm6 + vpaddd 96(%rsi),%ymm3,%ymm7 + vmovdqa %ymm4,0(%rsp) + xorl %r14d,%r14d + vmovdqa %ymm5,32(%rsp) + leaq -64(%rsp),%rsp + movl %ebx,%edi + vmovdqa %ymm6,0(%rsp) + xorl %ecx,%edi + vmovdqa %ymm7,32(%rsp) + movl %r9d,%r12d + subq $-32*4,%rsi + jmp .Lavx2_00_47 + +.align 16 +.Lavx2_00_47: + leaq -64(%rsp),%rsp + vpalignr $4,%ymm0,%ymm1,%ymm4 + addl 0+128(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + vpalignr $4,%ymm2,%ymm3,%ymm7 + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + vpsrld $7,%ymm4,%ymm6 + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + vpaddd %ymm7,%ymm0,%ymm0 + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + vpshufd $250,%ymm3,%ymm7 + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 4+128(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + vpslld $11,%ymm5,%ymm5 + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + vpaddd %ymm4,%ymm0,%ymm0 + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 8+128(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + vpxor %ymm7,%ymm6,%ymm6 + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + vpaddd %ymm6,%ymm0,%ymm0 + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + vpshufd $80,%ymm0,%ymm7 + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 12+128(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + vpxor %ymm7,%ymm6,%ymm6 + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + vpaddd %ymm6,%ymm0,%ymm0 + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + vpaddd 0(%rsi),%ymm0,%ymm6 + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + vmovdqa %ymm6,0(%rsp) + vpalignr $4,%ymm1,%ymm2,%ymm4 + addl 32+128(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + vpalignr $4,%ymm3,%ymm0,%ymm7 + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + vpsrld $7,%ymm4,%ymm6 + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + vpaddd %ymm7,%ymm1,%ymm1 + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + vpshufd $250,%ymm0,%ymm7 + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 36+128(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + vpslld $11,%ymm5,%ymm5 + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + vpaddd %ymm4,%ymm1,%ymm1 + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 40+128(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + vpxor %ymm7,%ymm6,%ymm6 + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + vpaddd %ymm6,%ymm1,%ymm1 + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + vpshufd $80,%ymm1,%ymm7 + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 44+128(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + vpxor %ymm7,%ymm6,%ymm6 + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + vpaddd %ymm6,%ymm1,%ymm1 + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + vpaddd 32(%rsi),%ymm1,%ymm6 + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + vmovdqa %ymm6,32(%rsp) + leaq -64(%rsp),%rsp + vpalignr $4,%ymm2,%ymm3,%ymm4 + addl 0+128(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + vpalignr $4,%ymm0,%ymm1,%ymm7 + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + vpsrld $7,%ymm4,%ymm6 + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + vpaddd %ymm7,%ymm2,%ymm2 + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + vpshufd $250,%ymm1,%ymm7 + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 4+128(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + vpslld $11,%ymm5,%ymm5 + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + vpaddd %ymm4,%ymm2,%ymm2 + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 8+128(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + vpxor %ymm7,%ymm6,%ymm6 + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + vpaddd %ymm6,%ymm2,%ymm2 + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + vpshufd $80,%ymm2,%ymm7 + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 12+128(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + vpxor %ymm7,%ymm6,%ymm6 + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + vpaddd %ymm6,%ymm2,%ymm2 + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + vpaddd 64(%rsi),%ymm2,%ymm6 + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + vmovdqa %ymm6,0(%rsp) + vpalignr $4,%ymm3,%ymm0,%ymm4 + addl 32+128(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + vpalignr $4,%ymm1,%ymm2,%ymm7 + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + vpsrld $7,%ymm4,%ymm6 + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + vpaddd %ymm7,%ymm3,%ymm3 + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + vpshufd $250,%ymm2,%ymm7 + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 36+128(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + vpslld $11,%ymm5,%ymm5 + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + vpaddd %ymm4,%ymm3,%ymm3 + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 40+128(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + vpxor %ymm7,%ymm6,%ymm6 + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + vpaddd %ymm6,%ymm3,%ymm3 + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + vpshufd $80,%ymm3,%ymm7 + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 44+128(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + vpxor %ymm7,%ymm6,%ymm6 + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + vpaddd %ymm6,%ymm3,%ymm3 + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + vpaddd 96(%rsi),%ymm3,%ymm6 + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + vmovdqa %ymm6,32(%rsp) + leaq 128(%rsi),%rsi + cmpb $0,3(%rsi) + jne .Lavx2_00_47 + addl 0+64(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4+64(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8+64(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12+64(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32+64(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36+64(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40+64(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44+64(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + addl 0(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + movq -64(%rbp),%rdi + addl %r14d,%eax + movl -56(%rbp),%r12d + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + + cmpl -48(%rbp),%r12d + je .Ldone_avx2 + + leaq 448(%rsp),%rsi + xorl %r14d,%r14d + movl %ebx,%edi + xorl %ecx,%edi + movl %r9d,%r12d + jmp .Lower_avx2 +.align 16 +.Lower_avx2: + addl 0+16(%rsi),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4+16(%rsi),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8+16(%rsi),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12+16(%rsi),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32+16(%rsi),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36+16(%rsi),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40+16(%rsi),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44+16(%rsi),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + leaq -64(%rsi),%rsi + cmpq %rsp,%rsi + jae .Lower_avx2 + + movq -64(%rbp),%rdi + addl %r14d,%eax + movq -56(%rbp),%rsi + leaq 448(%rsp),%rsp + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + leaq 128(%rsi),%rsi + addl 24(%rdi),%r10d + movq %rsi,%r12 + addl 28(%rdi),%r11d + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + cmoveq %rsp,%r12 + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + + jbe .Loop_avx2 + +.Ldone_avx2: + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha256_asm_block_data_order_avx2,.-crypton_sha256_asm_block_data_order_avx2 + +.section .note.gnu.property,"a",@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align 8 +2: + +.section .note.GNU-stack,"",@progbits diff --git a/cbits/asm/sha256-x86_64-macosx.S b/cbits/asm/sha256-x86_64-macosx.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha256-x86_64-macosx.S @@ -0,0 +1,5454 @@ +.text + + +.globl _crypton_sha256_asm_block_data_order + +.p2align 4 +_crypton_sha256_asm_block_data_order: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + leaq _crypton_ia32cap_P(%rip),%rax + movl 0(%rax),%r9d + movl 4(%rax),%r10d + movl 8(%rax),%eax + testl $536870912,%eax + jnz L$shaext_shortcut + andl $296,%eax + cmpl $296,%eax + je L$avx2_shortcut + andl $1073741824,%r9d + andl $268435968,%r10d + orl %r9d,%r10d + cmpl $1342177792,%r10d + je L$avx_shortcut + testl $512,%r10d + jnz L$ssse3_shortcut + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $64+24,%rsp + +.cfi_def_cfa %rsp,144 + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,64+0(%rsp) + movq %rsi,64+8(%rsp) + movq %rdx,64+16(%rsp) + + movl 0(%rdi),%eax + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + jmp L$loop + +.p2align 4 +L$loop: + movl %ebx,%edi + leaq K256(%rip),%rbp + xorl %ecx,%edi + movl 0(%rsi),%r12d + movl %r8d,%r13d + movl %eax,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,0(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + addl %r14d,%r11d + movl 4(%rsi),%r12d + movl %edx,%r13d + movl %r11d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,4(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + addl %r14d,%r10d + movl 8(%rsi),%r12d + movl %ecx,%r13d + movl %r10d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,8(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + addl %r14d,%r9d + movl 12(%rsi),%r12d + movl %ebx,%r13d + movl %r9d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,12(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + addl %r14d,%r8d + movl 16(%rsi),%r12d + movl %eax,%r13d + movl %r8d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,16(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + addl %r14d,%edx + movl 20(%rsi),%r12d + movl %r11d,%r13d + movl %edx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,20(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + addl %r14d,%ecx + movl 24(%rsi),%r12d + movl %r10d,%r13d + movl %ecx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,24(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + addl %r14d,%ebx + movl 28(%rsi),%r12d + movl %r9d,%r13d + movl %ebx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,28(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + addl %r14d,%eax + movl 32(%rsi),%r12d + movl %r8d,%r13d + movl %eax,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,32(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + addl %r14d,%r11d + movl 36(%rsi),%r12d + movl %edx,%r13d + movl %r11d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,36(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + addl %r14d,%r10d + movl 40(%rsi),%r12d + movl %ecx,%r13d + movl %r10d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,40(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + addl %r14d,%r9d + movl 44(%rsi),%r12d + movl %ebx,%r13d + movl %r9d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,44(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + addl %r14d,%r8d + movl 48(%rsi),%r12d + movl %eax,%r13d + movl %r8d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,48(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + addl %r14d,%edx + movl 52(%rsi),%r12d + movl %r11d,%r13d + movl %edx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,52(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + addl %r14d,%ecx + movl 56(%rsi),%r12d + movl %r10d,%r13d + movl %ecx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,56(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + addl %r14d,%ebx + movl 60(%rsi),%r12d + movl %r9d,%r13d + movl %ebx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,60(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + jmp L$rounds_16_xx +.p2align 4 +L$rounds_16_xx: + movl 4(%rsp),%r13d + movl 56(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%eax + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 36(%rsp),%r12d + + addl 0(%rsp),%r12d + movl %r8d,%r13d + addl %r15d,%r12d + movl %eax,%r14d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,0(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + movl 8(%rsp),%r13d + movl 60(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r11d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 40(%rsp),%r12d + + addl 4(%rsp),%r12d + movl %edx,%r13d + addl %edi,%r12d + movl %r11d,%r14d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,4(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + movl 12(%rsp),%r13d + movl 0(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r10d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 44(%rsp),%r12d + + addl 8(%rsp),%r12d + movl %ecx,%r13d + addl %r15d,%r12d + movl %r10d,%r14d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,8(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + movl 16(%rsp),%r13d + movl 4(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r9d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 48(%rsp),%r12d + + addl 12(%rsp),%r12d + movl %ebx,%r13d + addl %edi,%r12d + movl %r9d,%r14d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,12(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + movl 20(%rsp),%r13d + movl 8(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r8d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 52(%rsp),%r12d + + addl 16(%rsp),%r12d + movl %eax,%r13d + addl %r15d,%r12d + movl %r8d,%r14d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,16(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + movl 24(%rsp),%r13d + movl 12(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%edx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 56(%rsp),%r12d + + addl 20(%rsp),%r12d + movl %r11d,%r13d + addl %edi,%r12d + movl %edx,%r14d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,20(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + movl 28(%rsp),%r13d + movl 16(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ecx + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 60(%rsp),%r12d + + addl 24(%rsp),%r12d + movl %r10d,%r13d + addl %r15d,%r12d + movl %ecx,%r14d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,24(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + movl 32(%rsp),%r13d + movl 20(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ebx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 0(%rsp),%r12d + + addl 28(%rsp),%r12d + movl %r9d,%r13d + addl %edi,%r12d + movl %ebx,%r14d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,28(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + movl 36(%rsp),%r13d + movl 24(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%eax + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 4(%rsp),%r12d + + addl 32(%rsp),%r12d + movl %r8d,%r13d + addl %r15d,%r12d + movl %eax,%r14d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,32(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + movl 40(%rsp),%r13d + movl 28(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r11d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 8(%rsp),%r12d + + addl 36(%rsp),%r12d + movl %edx,%r13d + addl %edi,%r12d + movl %r11d,%r14d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,36(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + movl 44(%rsp),%r13d + movl 32(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r10d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 12(%rsp),%r12d + + addl 40(%rsp),%r12d + movl %ecx,%r13d + addl %r15d,%r12d + movl %r10d,%r14d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,40(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + movl 48(%rsp),%r13d + movl 36(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r9d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 16(%rsp),%r12d + + addl 44(%rsp),%r12d + movl %ebx,%r13d + addl %edi,%r12d + movl %r9d,%r14d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,44(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + movl 52(%rsp),%r13d + movl 40(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r8d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 20(%rsp),%r12d + + addl 48(%rsp),%r12d + movl %eax,%r13d + addl %r15d,%r12d + movl %r8d,%r14d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,48(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + movl 56(%rsp),%r13d + movl 44(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%edx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 24(%rsp),%r12d + + addl 52(%rsp),%r12d + movl %r11d,%r13d + addl %edi,%r12d + movl %edx,%r14d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,52(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + movl 60(%rsp),%r13d + movl 48(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ecx + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 28(%rsp),%r12d + + addl 56(%rsp),%r12d + movl %r10d,%r13d + addl %r15d,%r12d + movl %ecx,%r14d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,56(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + movl 0(%rsp),%r13d + movl 52(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ebx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 32(%rsp),%r12d + + addl 60(%rsp),%r12d + movl %r9d,%r13d + addl %edi,%r12d + movl %ebx,%r14d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,60(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + cmpb $0,3(%rbp) + jnz L$rounds_16_xx + + movq 64+0(%rsp),%rdi + addl %r14d,%eax + leaq 64(%rsi),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + cmpq 64+16(%rsp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb L$loop + + leaq 64+24+48(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq 64+24(%rsp),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbx + movq -8(%r11),%rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + leaq (%r11),%rsp + .byte 0xf3,0xc3 +.cfi_endproc + +.p2align 6 + +K256: +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 + +.long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f +.long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f +.long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff +.long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff +.long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 +.long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 +.byte 83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 + +.p2align 6 +crypton_sha256_asm_block_data_order_shaext: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$shaext_shortcut: + + leaq K256+128(%rip),%rcx + movdqu (%rdi),%xmm1 + movdqu 16(%rdi),%xmm2 + movdqa 512-128(%rcx),%xmm7 + + pshufd $0x1b,%xmm1,%xmm0 + pshufd $0xb1,%xmm1,%xmm1 + pshufd $0x1b,%xmm2,%xmm2 + movdqa %xmm7,%xmm8 +.byte 102,15,58,15,202,8 + punpcklqdq %xmm0,%xmm2 + jmp L$oop_shaext + +.p2align 4 +L$oop_shaext: + movdqu (%rsi),%xmm3 + movdqu 16(%rsi),%xmm4 + movdqu 32(%rsi),%xmm5 +.byte 102,15,56,0,223 + movdqu 48(%rsi),%xmm6 + + movdqa 0-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 102,15,56,0,231 + movdqa %xmm2,%xmm10 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + nop + movdqa %xmm1,%xmm9 +.byte 15,56,203,202 + + movdqa 32-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 102,15,56,0,239 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + leaq 64(%rsi),%rsi +.byte 15,56,204,220 +.byte 15,56,203,202 + + movdqa 64-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 102,15,56,0,247 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + + movdqa 96-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 128-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 160-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 + nop + paddd %xmm7,%xmm6 +.byte 15,56,204,220 +.byte 15,56,203,202 + movdqa 192-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,205,245 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + movdqa 224-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 256-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 288-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 + nop + paddd %xmm7,%xmm6 +.byte 15,56,204,220 +.byte 15,56,203,202 + movdqa 320-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,205,245 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + movdqa 352-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 384-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 416-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 +.byte 15,56,203,202 + paddd %xmm7,%xmm6 + + movdqa 448-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 +.byte 15,56,205,245 + movdqa %xmm8,%xmm7 +.byte 15,56,203,202 + + movdqa 480-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 + nop +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + decq %rdx + nop +.byte 15,56,203,202 + + paddd %xmm10,%xmm2 + paddd %xmm9,%xmm1 + jnz L$oop_shaext + + pshufd $0xb1,%xmm2,%xmm2 + pshufd $0x1b,%xmm1,%xmm7 + pshufd $0xb1,%xmm1,%xmm1 + punpckhqdq %xmm2,%xmm1 +.byte 102,15,58,15,215,8 + + movdqu %xmm1,(%rdi) + movdqu %xmm2,16(%rdi) +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp + + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 6 +crypton_sha256_asm_block_data_order_ssse3: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$ssse3_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -64(%rsp),%rsp + movl 0(%rdi),%eax + andq $-64,%rsp + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + + + jmp L$loop_ssse3 +.p2align 4 +L$loop_ssse3: + movdqa K256+512(%rip),%xmm7 + movq %rsi,-56(%rbp) + movdqu 0(%rsi),%xmm0 + movdqu 16(%rsi),%xmm1 + movdqu 32(%rsi),%xmm2 +.byte 102,15,56,0,199 + movdqu 48(%rsi),%xmm3 + leaq K256(%rip),%rsi +.byte 102,15,56,0,207 + movdqa 0(%rsi),%xmm4 + movdqa 32(%rsi),%xmm5 +.byte 102,15,56,0,215 + paddd %xmm0,%xmm4 + movdqa 64(%rsi),%xmm6 +.byte 102,15,56,0,223 + movdqa 96(%rsi),%xmm7 + paddd %xmm1,%xmm5 + paddd %xmm2,%xmm6 + paddd %xmm3,%xmm7 + movdqa %xmm4,0(%rsp) + movl %eax,%r14d + movdqa %xmm5,16(%rsp) + movl %ebx,%edi + movdqa %xmm6,32(%rsp) + xorl %ecx,%edi + movdqa %xmm7,48(%rsp) + movl %r8d,%r13d + jmp L$ssse3_00_47 + +.p2align 4 +L$ssse3_00_47: + subq $-128,%rsi + rorl $14,%r13d + movdqa %xmm1,%xmm4 + movl %r14d,%eax + movl %r9d,%r12d + movdqa %xmm3,%xmm7 + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d +.byte 102,15,58,15,224,4 + andl %r8d,%r12d + xorl %r8d,%r13d +.byte 102,15,58,15,250,4 + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %ebx,%r15d + addl %r12d,%r11d + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + paddd %xmm7,%xmm0 + rorl $2,%r14d + addl %r11d,%edx + psrld $7,%xmm6 + addl %edi,%r11d + movl %edx,%r13d + pshufd $250,%xmm3,%xmm7 + addl %r11d,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%r11d + movl %r8d,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %r11d,%r14d + pxor %xmm5,%xmm4 + andl %edx,%r12d + xorl %edx,%r13d + pslld $11,%xmm5 + addl 4(%rsp),%r10d + movl %r11d,%edi + pxor %xmm6,%xmm4 + xorl %r9d,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %eax,%edi + addl %r12d,%r10d + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + psrld $10,%xmm7 + addl %r13d,%r10d + xorl %eax,%r15d + paddd %xmm4,%xmm0 + rorl $2,%r14d + addl %r10d,%ecx + psrlq $17,%xmm6 + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %ecx,%r13d + xorl %r8d,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + psrldq $8,%xmm7 + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + paddd %xmm7,%xmm0 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + pshufd $80,%xmm0,%xmm7 + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + movdqa %xmm7,%xmm6 + addl %edi,%r9d + movl %ebx,%r13d + psrld $10,%xmm7 + addl %r9d,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%r9d + movl %ecx,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + psrlq $2,%xmm6 + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + pxor %xmm6,%xmm7 + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %r10d,%edi + addl %r12d,%r8d + movdqa 0(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + paddd %xmm7,%xmm0 + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + paddd %xmm0,%xmm6 + movl %eax,%r13d + addl %r8d,%r14d + movdqa %xmm6,0(%rsp) + rorl $14,%r13d + movdqa %xmm2,%xmm4 + movl %r14d,%r8d + movl %ebx,%r12d + movdqa %xmm0,%xmm7 + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d +.byte 102,15,58,15,225,4 + andl %eax,%r12d + xorl %eax,%r13d +.byte 102,15,58,15,251,4 + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %r9d,%r15d + addl %r12d,%edx + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + paddd %xmm7,%xmm1 + rorl $2,%r14d + addl %edx,%r11d + psrld $7,%xmm6 + addl %edi,%edx + movl %r11d,%r13d + pshufd $250,%xmm0,%xmm7 + addl %edx,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%edx + movl %eax,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %edx,%r14d + pxor %xmm5,%xmm4 + andl %r11d,%r12d + xorl %r11d,%r13d + pslld $11,%xmm5 + addl 20(%rsp),%ecx + movl %edx,%edi + pxor %xmm6,%xmm4 + xorl %ebx,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %r8d,%edi + addl %r12d,%ecx + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + psrld $10,%xmm7 + addl %r13d,%ecx + xorl %r8d,%r15d + paddd %xmm4,%xmm1 + rorl $2,%r14d + addl %ecx,%r10d + psrlq $17,%xmm6 + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %r10d,%r13d + xorl %eax,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + psrldq $8,%xmm7 + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + paddd %xmm7,%xmm1 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + pshufd $80,%xmm1,%xmm7 + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + movdqa %xmm7,%xmm6 + addl %edi,%ebx + movl %r9d,%r13d + psrld $10,%xmm7 + addl %ebx,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%ebx + movl %r10d,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + psrlq $2,%xmm6 + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + pxor %xmm6,%xmm7 + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %ecx,%edi + addl %r12d,%eax + movdqa 32(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + paddd %xmm7,%xmm1 + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + paddd %xmm1,%xmm6 + movl %r8d,%r13d + addl %eax,%r14d + movdqa %xmm6,16(%rsp) + rorl $14,%r13d + movdqa %xmm3,%xmm4 + movl %r14d,%eax + movl %r9d,%r12d + movdqa %xmm1,%xmm7 + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d +.byte 102,15,58,15,226,4 + andl %r8d,%r12d + xorl %r8d,%r13d +.byte 102,15,58,15,248,4 + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %ebx,%r15d + addl %r12d,%r11d + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + paddd %xmm7,%xmm2 + rorl $2,%r14d + addl %r11d,%edx + psrld $7,%xmm6 + addl %edi,%r11d + movl %edx,%r13d + pshufd $250,%xmm1,%xmm7 + addl %r11d,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%r11d + movl %r8d,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %r11d,%r14d + pxor %xmm5,%xmm4 + andl %edx,%r12d + xorl %edx,%r13d + pslld $11,%xmm5 + addl 36(%rsp),%r10d + movl %r11d,%edi + pxor %xmm6,%xmm4 + xorl %r9d,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %eax,%edi + addl %r12d,%r10d + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + psrld $10,%xmm7 + addl %r13d,%r10d + xorl %eax,%r15d + paddd %xmm4,%xmm2 + rorl $2,%r14d + addl %r10d,%ecx + psrlq $17,%xmm6 + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %ecx,%r13d + xorl %r8d,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + psrldq $8,%xmm7 + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + paddd %xmm7,%xmm2 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + pshufd $80,%xmm2,%xmm7 + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + movdqa %xmm7,%xmm6 + addl %edi,%r9d + movl %ebx,%r13d + psrld $10,%xmm7 + addl %r9d,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%r9d + movl %ecx,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + psrlq $2,%xmm6 + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + pxor %xmm6,%xmm7 + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %r10d,%edi + addl %r12d,%r8d + movdqa 64(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + paddd %xmm7,%xmm2 + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + paddd %xmm2,%xmm6 + movl %eax,%r13d + addl %r8d,%r14d + movdqa %xmm6,32(%rsp) + rorl $14,%r13d + movdqa %xmm0,%xmm4 + movl %r14d,%r8d + movl %ebx,%r12d + movdqa %xmm2,%xmm7 + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d +.byte 102,15,58,15,227,4 + andl %eax,%r12d + xorl %eax,%r13d +.byte 102,15,58,15,249,4 + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %r9d,%r15d + addl %r12d,%edx + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + paddd %xmm7,%xmm3 + rorl $2,%r14d + addl %edx,%r11d + psrld $7,%xmm6 + addl %edi,%edx + movl %r11d,%r13d + pshufd $250,%xmm2,%xmm7 + addl %edx,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%edx + movl %eax,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %edx,%r14d + pxor %xmm5,%xmm4 + andl %r11d,%r12d + xorl %r11d,%r13d + pslld $11,%xmm5 + addl 52(%rsp),%ecx + movl %edx,%edi + pxor %xmm6,%xmm4 + xorl %ebx,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %r8d,%edi + addl %r12d,%ecx + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + psrld $10,%xmm7 + addl %r13d,%ecx + xorl %r8d,%r15d + paddd %xmm4,%xmm3 + rorl $2,%r14d + addl %ecx,%r10d + psrlq $17,%xmm6 + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %r10d,%r13d + xorl %eax,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + psrldq $8,%xmm7 + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + paddd %xmm7,%xmm3 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + pshufd $80,%xmm3,%xmm7 + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + movdqa %xmm7,%xmm6 + addl %edi,%ebx + movl %r9d,%r13d + psrld $10,%xmm7 + addl %ebx,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%ebx + movl %r10d,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + psrlq $2,%xmm6 + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + pxor %xmm6,%xmm7 + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %ecx,%edi + addl %r12d,%eax + movdqa 96(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + paddd %xmm7,%xmm3 + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + paddd %xmm3,%xmm6 + movl %r8d,%r13d + addl %eax,%r14d + movdqa %xmm6,48(%rsp) + cmpb $0,131(%rsi) + jne L$ssse3_00_47 + rorl $14,%r13d + movl %r14d,%eax + movl %r9d,%r12d + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + rorl $6,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + rorl $2,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + rorl $14,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + rorl $11,%r14d + xorl %eax,%edi + addl %r12d,%r10d + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + rorl $2,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + rorl $14,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + rorl $6,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + rorl $14,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + rorl $6,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + rorl $2,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + rorl $14,%r13d + movl %r14d,%edx + movl %eax,%r12d + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + rorl $11,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + rorl $2,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + rorl $14,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + xorl %ecx,%edi + addl %r12d,%eax + rorl $6,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + rorl $14,%r13d + movl %r14d,%eax + movl %r9d,%r12d + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + rorl $6,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + rorl $2,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + rorl $14,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + rorl $11,%r14d + xorl %eax,%edi + addl %r12d,%r10d + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + rorl $2,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + rorl $14,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + rorl $6,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + rorl $14,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + rorl $6,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + rorl $2,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + rorl $14,%r13d + movl %r14d,%edx + movl %eax,%r12d + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + rorl $11,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + rorl $2,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + rorl $14,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + xorl %ecx,%edi + addl %r12d,%eax + rorl $6,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + movq -64(%rbp),%rdi + movl %r14d,%eax + movq -56(%rbp),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + leaq 64(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb L$loop_ssse3 + + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 6 +crypton_sha256_asm_block_data_order_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$avx_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -64(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movl 0(%rdi),%eax + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + vmovdqa K256+512+32(%rip),%xmm8 + vmovdqa K256+512+64(%rip),%xmm9 + jmp L$loop_avx +.p2align 4 +L$loop_avx: + vmovdqa K256+512(%rip),%xmm7 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm7,%xmm0,%xmm0 + leaq K256(%rip),%rsi + vpshufb %xmm7,%xmm1,%xmm1 + vpshufb %xmm7,%xmm2,%xmm2 + vpaddd 0(%rsi),%xmm0,%xmm4 + vpshufb %xmm7,%xmm3,%xmm3 + vpaddd 32(%rsi),%xmm1,%xmm5 + vpaddd 64(%rsi),%xmm2,%xmm6 + vpaddd 96(%rsi),%xmm3,%xmm7 + vmovdqa %xmm4,0(%rsp) + movl %eax,%r14d + vmovdqa %xmm5,16(%rsp) + movl %ebx,%edi + vmovdqa %xmm6,32(%rsp) + xorl %ecx,%edi + vmovdqa %xmm7,48(%rsp) + movl %r8d,%r13d + jmp L$avx_00_47 + +.p2align 4 +L$avx_00_47: + subq $-128,%rsi + vpalignr $4,%xmm0,%xmm1,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + vpalignr $4,%xmm2,%xmm3,%xmm7 + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + vpaddd %xmm7,%xmm0,%xmm0 + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + vpshufd $250,%xmm3,%xmm7 + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + vpsrld $11,%xmm6,%xmm6 + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + vpaddd %xmm4,%xmm0,%xmm0 + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + vpxor %xmm7,%xmm6,%xmm6 + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + vpaddd %xmm6,%xmm0,%xmm0 + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + vpshufd $80,%xmm0,%xmm7 + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + vpxor %xmm7,%xmm6,%xmm6 + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + vpaddd %xmm6,%xmm0,%xmm0 + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + vpaddd 0(%rsi),%xmm0,%xmm6 + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + vmovdqa %xmm6,0(%rsp) + vpalignr $4,%xmm1,%xmm2,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + vpalignr $4,%xmm3,%xmm0,%xmm7 + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + vpaddd %xmm7,%xmm1,%xmm1 + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + vpshufd $250,%xmm0,%xmm7 + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + vpsrld $11,%xmm6,%xmm6 + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + vpaddd %xmm4,%xmm1,%xmm1 + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + vpxor %xmm7,%xmm6,%xmm6 + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + vpaddd %xmm6,%xmm1,%xmm1 + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + vpshufd $80,%xmm1,%xmm7 + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + vpxor %xmm7,%xmm6,%xmm6 + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + vpaddd %xmm6,%xmm1,%xmm1 + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + vpaddd 32(%rsi),%xmm1,%xmm6 + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + vmovdqa %xmm6,16(%rsp) + vpalignr $4,%xmm2,%xmm3,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + vpalignr $4,%xmm0,%xmm1,%xmm7 + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + vpaddd %xmm7,%xmm2,%xmm2 + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + vpshufd $250,%xmm1,%xmm7 + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + vpsrld $11,%xmm6,%xmm6 + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + vpaddd %xmm4,%xmm2,%xmm2 + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + vpxor %xmm7,%xmm6,%xmm6 + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + vpaddd %xmm6,%xmm2,%xmm2 + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + vpshufd $80,%xmm2,%xmm7 + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + vpxor %xmm7,%xmm6,%xmm6 + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + vpaddd %xmm6,%xmm2,%xmm2 + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + vpaddd 64(%rsi),%xmm2,%xmm6 + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + vmovdqa %xmm6,32(%rsp) + vpalignr $4,%xmm3,%xmm0,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + vpalignr $4,%xmm1,%xmm2,%xmm7 + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + vpaddd %xmm7,%xmm3,%xmm3 + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + vpshufd $250,%xmm2,%xmm7 + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + vpsrld $11,%xmm6,%xmm6 + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + vpaddd %xmm4,%xmm3,%xmm3 + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + vpxor %xmm7,%xmm6,%xmm6 + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + vpaddd %xmm6,%xmm3,%xmm3 + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + vpshufd $80,%xmm3,%xmm7 + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + vpxor %xmm7,%xmm6,%xmm6 + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + vpaddd %xmm6,%xmm3,%xmm3 + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + vpaddd 96(%rsi),%xmm3,%xmm6 + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + vmovdqa %xmm6,48(%rsp) + cmpb $0,131(%rsi) + jne L$avx_00_47 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + movq -64(%rbp),%rdi + movl %r14d,%eax + movq -56(%rbp),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + leaq 64(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb L$loop_avx + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 6 +crypton_sha256_asm_block_data_order_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$avx2_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -64(%rsp),%rsp + vzeroupper + andq $-64,%rsp + subq $-64,%rsi + movl 0(%rdi),%eax + movq %rsi,%r12 + movl 4(%rdi),%ebx + cmpq %rdx,%rsi + movl 8(%rdi),%ecx + cmoveq %rsp,%r12 + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + vmovdqa K256+512+32(%rip),%ymm8 + vmovdqa K256+512+64(%rip),%ymm9 + jmp L$oop_avx2 +.p2align 4 +L$oop_avx2: + vmovdqa K256+512(%rip),%ymm7 + movq %rsi,-56(%rbp) + vmovdqu -64+0(%rsi),%xmm0 + vmovdqu -64+16(%rsi),%xmm1 + vmovdqu -64+32(%rsi),%xmm2 + vmovdqu -64+48(%rsi),%xmm3 + leaq K256(%rip),%rsi + vinserti128 $1,(%r12),%ymm0,%ymm0 + vinserti128 $1,16(%r12),%ymm1,%ymm1 + vpshufb %ymm7,%ymm0,%ymm0 + vinserti128 $1,32(%r12),%ymm2,%ymm2 + vpshufb %ymm7,%ymm1,%ymm1 + vinserti128 $1,48(%r12),%ymm3,%ymm3 + + vpshufb %ymm7,%ymm2,%ymm2 + vpaddd 0(%rsi),%ymm0,%ymm4 + vpshufb %ymm7,%ymm3,%ymm3 + vpaddd 32(%rsi),%ymm1,%ymm5 + vpaddd 64(%rsi),%ymm2,%ymm6 + vpaddd 96(%rsi),%ymm3,%ymm7 + vmovdqa %ymm4,0(%rsp) + xorl %r14d,%r14d + vmovdqa %ymm5,32(%rsp) + leaq -64(%rsp),%rsp + movl %ebx,%edi + vmovdqa %ymm6,0(%rsp) + xorl %ecx,%edi + vmovdqa %ymm7,32(%rsp) + movl %r9d,%r12d + subq $-32*4,%rsi + jmp L$avx2_00_47 + +.p2align 4 +L$avx2_00_47: + leaq -64(%rsp),%rsp + vpalignr $4,%ymm0,%ymm1,%ymm4 + addl 0+128(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + vpalignr $4,%ymm2,%ymm3,%ymm7 + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + vpsrld $7,%ymm4,%ymm6 + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + vpaddd %ymm7,%ymm0,%ymm0 + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + vpshufd $250,%ymm3,%ymm7 + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 4+128(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + vpslld $11,%ymm5,%ymm5 + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + vpaddd %ymm4,%ymm0,%ymm0 + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 8+128(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + vpxor %ymm7,%ymm6,%ymm6 + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + vpaddd %ymm6,%ymm0,%ymm0 + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + vpshufd $80,%ymm0,%ymm7 + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 12+128(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + vpxor %ymm7,%ymm6,%ymm6 + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + vpaddd %ymm6,%ymm0,%ymm0 + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + vpaddd 0(%rsi),%ymm0,%ymm6 + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + vmovdqa %ymm6,0(%rsp) + vpalignr $4,%ymm1,%ymm2,%ymm4 + addl 32+128(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + vpalignr $4,%ymm3,%ymm0,%ymm7 + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + vpsrld $7,%ymm4,%ymm6 + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + vpaddd %ymm7,%ymm1,%ymm1 + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + vpshufd $250,%ymm0,%ymm7 + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 36+128(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + vpslld $11,%ymm5,%ymm5 + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + vpaddd %ymm4,%ymm1,%ymm1 + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 40+128(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + vpxor %ymm7,%ymm6,%ymm6 + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + vpaddd %ymm6,%ymm1,%ymm1 + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + vpshufd $80,%ymm1,%ymm7 + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 44+128(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + vpxor %ymm7,%ymm6,%ymm6 + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + vpaddd %ymm6,%ymm1,%ymm1 + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + vpaddd 32(%rsi),%ymm1,%ymm6 + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + vmovdqa %ymm6,32(%rsp) + leaq -64(%rsp),%rsp + vpalignr $4,%ymm2,%ymm3,%ymm4 + addl 0+128(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + vpalignr $4,%ymm0,%ymm1,%ymm7 + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + vpsrld $7,%ymm4,%ymm6 + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + vpaddd %ymm7,%ymm2,%ymm2 + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + vpshufd $250,%ymm1,%ymm7 + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 4+128(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + vpslld $11,%ymm5,%ymm5 + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + vpaddd %ymm4,%ymm2,%ymm2 + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 8+128(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + vpxor %ymm7,%ymm6,%ymm6 + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + vpaddd %ymm6,%ymm2,%ymm2 + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + vpshufd $80,%ymm2,%ymm7 + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 12+128(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + vpxor %ymm7,%ymm6,%ymm6 + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + vpaddd %ymm6,%ymm2,%ymm2 + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + vpaddd 64(%rsi),%ymm2,%ymm6 + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + vmovdqa %ymm6,0(%rsp) + vpalignr $4,%ymm3,%ymm0,%ymm4 + addl 32+128(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + vpalignr $4,%ymm1,%ymm2,%ymm7 + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + vpsrld $7,%ymm4,%ymm6 + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + vpaddd %ymm7,%ymm3,%ymm3 + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + vpshufd $250,%ymm2,%ymm7 + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 36+128(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + vpslld $11,%ymm5,%ymm5 + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + vpaddd %ymm4,%ymm3,%ymm3 + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 40+128(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + vpxor %ymm7,%ymm6,%ymm6 + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + vpaddd %ymm6,%ymm3,%ymm3 + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + vpshufd $80,%ymm3,%ymm7 + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 44+128(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + vpxor %ymm7,%ymm6,%ymm6 + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + vpaddd %ymm6,%ymm3,%ymm3 + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + vpaddd 96(%rsi),%ymm3,%ymm6 + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + vmovdqa %ymm6,32(%rsp) + leaq 128(%rsi),%rsi + cmpb $0,3(%rsi) + jne L$avx2_00_47 + addl 0+64(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4+64(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8+64(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12+64(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32+64(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36+64(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40+64(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44+64(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + addl 0(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + movq -64(%rbp),%rdi + addl %r14d,%eax + movl -56(%rbp),%r12d + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + + cmpl -48(%rbp),%r12d + je L$done_avx2 + + leaq 448(%rsp),%rsi + xorl %r14d,%r14d + movl %ebx,%edi + xorl %ecx,%edi + movl %r9d,%r12d + jmp L$ower_avx2 +.p2align 4 +L$ower_avx2: + addl 0+16(%rsi),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4+16(%rsi),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8+16(%rsi),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12+16(%rsi),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32+16(%rsi),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36+16(%rsi),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40+16(%rsi),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44+16(%rsi),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + leaq -64(%rsi),%rsi + cmpq %rsp,%rsi + jae L$ower_avx2 + + movq -64(%rbp),%rdi + addl %r14d,%eax + movq -56(%rbp),%rsi + leaq 448(%rsp),%rsp + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + leaq 128(%rsi),%rsi + addl 24(%rdi),%r10d + movq %rsi,%r12 + addl 28(%rdi),%r11d + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + cmoveq %rsp,%r12 + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + + jbe L$oop_avx2 + +L$done_avx2: + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + diff --git a/cbits/asm/sha256-x86_64-mingw64.S b/cbits/asm/sha256-x86_64-mingw64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha256-x86_64-mingw64.S @@ -0,0 +1,5731 @@ +.text + + +.globl crypton_sha256_asm_block_data_order +.def crypton_sha256_asm_block_data_order; .scl 2; .type 32; .endef +.p2align 4 +crypton_sha256_asm_block_data_order: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha256_asm_block_data_order: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + leaq crypton_ia32cap_P(%rip),%rax + movl 0(%rax),%r9d + movl 4(%rax),%r10d + movl 8(%rax),%eax + testl $536870912,%eax + jnz .Lshaext_shortcut + andl $296,%eax + cmpl $296,%eax + je .Lavx2_shortcut + andl $1073741824,%r9d + andl $268435968,%r10d + orl %r9d,%r10d + cmpl $1342177792,%r10d + je .Lavx_shortcut + testl $512,%r10d + jnz .Lssse3_shortcut + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $64+24,%rsp + + +.LSEH_body_crypton_sha256_asm_block_data_order: + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,64+0(%rsp) + movq %rsi,64+8(%rsp) + movq %rdx,64+16(%rsp) + + movl 0(%rdi),%eax + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + jmp .Lloop + +.p2align 4 +.Lloop: + movl %ebx,%edi + leaq K256(%rip),%rbp + xorl %ecx,%edi + movl 0(%rsi),%r12d + movl %r8d,%r13d + movl %eax,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,0(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + addl %r14d,%r11d + movl 4(%rsi),%r12d + movl %edx,%r13d + movl %r11d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,4(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + addl %r14d,%r10d + movl 8(%rsi),%r12d + movl %ecx,%r13d + movl %r10d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,8(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + addl %r14d,%r9d + movl 12(%rsi),%r12d + movl %ebx,%r13d + movl %r9d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,12(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + addl %r14d,%r8d + movl 16(%rsi),%r12d + movl %eax,%r13d + movl %r8d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,16(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + addl %r14d,%edx + movl 20(%rsi),%r12d + movl %r11d,%r13d + movl %edx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,20(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + addl %r14d,%ecx + movl 24(%rsi),%r12d + movl %r10d,%r13d + movl %ecx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,24(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + addl %r14d,%ebx + movl 28(%rsi),%r12d + movl %r9d,%r13d + movl %ebx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,28(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + addl %r14d,%eax + movl 32(%rsi),%r12d + movl %r8d,%r13d + movl %eax,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,32(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + addl %r14d,%r11d + movl 36(%rsi),%r12d + movl %edx,%r13d + movl %r11d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,36(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + addl %r14d,%r10d + movl 40(%rsi),%r12d + movl %ecx,%r13d + movl %r10d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,40(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + addl %r14d,%r9d + movl 44(%rsi),%r12d + movl %ebx,%r13d + movl %r9d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,44(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + addl %r14d,%r8d + movl 48(%rsi),%r12d + movl %eax,%r13d + movl %r8d,%r14d + bswapl %r12d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,48(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + addl %r14d,%edx + movl 52(%rsi),%r12d + movl %r11d,%r13d + movl %edx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,52(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + addl %r14d,%ecx + movl 56(%rsi),%r12d + movl %r10d,%r13d + movl %ecx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,56(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + addl %r14d,%ebx + movl 60(%rsi),%r12d + movl %r9d,%r13d + movl %ebx,%r14d + bswapl %r12d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,60(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + jmp .Lrounds_16_xx +.p2align 4 +.Lrounds_16_xx: + movl 4(%rsp),%r13d + movl 56(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%eax + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 36(%rsp),%r12d + + addl 0(%rsp),%r12d + movl %r8d,%r13d + addl %r15d,%r12d + movl %eax,%r14d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,0(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + movl 8(%rsp),%r13d + movl 60(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r11d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 40(%rsp),%r12d + + addl 4(%rsp),%r12d + movl %edx,%r13d + addl %edi,%r12d + movl %r11d,%r14d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,4(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + movl 12(%rsp),%r13d + movl 0(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r10d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 44(%rsp),%r12d + + addl 8(%rsp),%r12d + movl %ecx,%r13d + addl %r15d,%r12d + movl %r10d,%r14d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,8(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + movl 16(%rsp),%r13d + movl 4(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r9d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 48(%rsp),%r12d + + addl 12(%rsp),%r12d + movl %ebx,%r13d + addl %edi,%r12d + movl %r9d,%r14d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,12(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + movl 20(%rsp),%r13d + movl 8(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r8d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 52(%rsp),%r12d + + addl 16(%rsp),%r12d + movl %eax,%r13d + addl %r15d,%r12d + movl %r8d,%r14d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,16(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + movl 24(%rsp),%r13d + movl 12(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%edx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 56(%rsp),%r12d + + addl 20(%rsp),%r12d + movl %r11d,%r13d + addl %edi,%r12d + movl %edx,%r14d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,20(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + movl 28(%rsp),%r13d + movl 16(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ecx + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 60(%rsp),%r12d + + addl 24(%rsp),%r12d + movl %r10d,%r13d + addl %r15d,%r12d + movl %ecx,%r14d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,24(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + movl 32(%rsp),%r13d + movl 20(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ebx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 0(%rsp),%r12d + + addl 28(%rsp),%r12d + movl %r9d,%r13d + addl %edi,%r12d + movl %ebx,%r14d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,28(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + movl 36(%rsp),%r13d + movl 24(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%eax + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 4(%rsp),%r12d + + addl 32(%rsp),%r12d + movl %r8d,%r13d + addl %r15d,%r12d + movl %eax,%r14d + rorl $14,%r13d + movl %r9d,%r15d + + xorl %r8d,%r13d + rorl $9,%r14d + xorl %r10d,%r15d + + movl %r12d,32(%rsp) + xorl %eax,%r14d + andl %r8d,%r15d + + rorl $5,%r13d + addl %r11d,%r12d + xorl %r10d,%r15d + + rorl $11,%r14d + xorl %r8d,%r13d + addl %r15d,%r12d + + movl %eax,%r15d + addl (%rbp),%r12d + xorl %eax,%r14d + + xorl %ebx,%r15d + rorl $6,%r13d + movl %ebx,%r11d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r11d + addl %r12d,%edx + addl %r12d,%r11d + + leaq 4(%rbp),%rbp + movl 40(%rsp),%r13d + movl 28(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r11d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 8(%rsp),%r12d + + addl 36(%rsp),%r12d + movl %edx,%r13d + addl %edi,%r12d + movl %r11d,%r14d + rorl $14,%r13d + movl %r8d,%edi + + xorl %edx,%r13d + rorl $9,%r14d + xorl %r9d,%edi + + movl %r12d,36(%rsp) + xorl %r11d,%r14d + andl %edx,%edi + + rorl $5,%r13d + addl %r10d,%r12d + xorl %r9d,%edi + + rorl $11,%r14d + xorl %edx,%r13d + addl %edi,%r12d + + movl %r11d,%edi + addl (%rbp),%r12d + xorl %r11d,%r14d + + xorl %eax,%edi + rorl $6,%r13d + movl %eax,%r10d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r10d + addl %r12d,%ecx + addl %r12d,%r10d + + leaq 4(%rbp),%rbp + movl 44(%rsp),%r13d + movl 32(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r10d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 12(%rsp),%r12d + + addl 40(%rsp),%r12d + movl %ecx,%r13d + addl %r15d,%r12d + movl %r10d,%r14d + rorl $14,%r13d + movl %edx,%r15d + + xorl %ecx,%r13d + rorl $9,%r14d + xorl %r8d,%r15d + + movl %r12d,40(%rsp) + xorl %r10d,%r14d + andl %ecx,%r15d + + rorl $5,%r13d + addl %r9d,%r12d + xorl %r8d,%r15d + + rorl $11,%r14d + xorl %ecx,%r13d + addl %r15d,%r12d + + movl %r10d,%r15d + addl (%rbp),%r12d + xorl %r10d,%r14d + + xorl %r11d,%r15d + rorl $6,%r13d + movl %r11d,%r9d + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%r9d + addl %r12d,%ebx + addl %r12d,%r9d + + leaq 4(%rbp),%rbp + movl 48(%rsp),%r13d + movl 36(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r9d + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 16(%rsp),%r12d + + addl 44(%rsp),%r12d + movl %ebx,%r13d + addl %edi,%r12d + movl %r9d,%r14d + rorl $14,%r13d + movl %ecx,%edi + + xorl %ebx,%r13d + rorl $9,%r14d + xorl %edx,%edi + + movl %r12d,44(%rsp) + xorl %r9d,%r14d + andl %ebx,%edi + + rorl $5,%r13d + addl %r8d,%r12d + xorl %edx,%edi + + rorl $11,%r14d + xorl %ebx,%r13d + addl %edi,%r12d + + movl %r9d,%edi + addl (%rbp),%r12d + xorl %r9d,%r14d + + xorl %r10d,%edi + rorl $6,%r13d + movl %r10d,%r8d + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%r8d + addl %r12d,%eax + addl %r12d,%r8d + + leaq 20(%rbp),%rbp + movl 52(%rsp),%r13d + movl 40(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%r8d + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 20(%rsp),%r12d + + addl 48(%rsp),%r12d + movl %eax,%r13d + addl %r15d,%r12d + movl %r8d,%r14d + rorl $14,%r13d + movl %ebx,%r15d + + xorl %eax,%r13d + rorl $9,%r14d + xorl %ecx,%r15d + + movl %r12d,48(%rsp) + xorl %r8d,%r14d + andl %eax,%r15d + + rorl $5,%r13d + addl %edx,%r12d + xorl %ecx,%r15d + + rorl $11,%r14d + xorl %eax,%r13d + addl %r15d,%r12d + + movl %r8d,%r15d + addl (%rbp),%r12d + xorl %r8d,%r14d + + xorl %r9d,%r15d + rorl $6,%r13d + movl %r9d,%edx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%edx + addl %r12d,%r11d + addl %r12d,%edx + + leaq 4(%rbp),%rbp + movl 56(%rsp),%r13d + movl 44(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%edx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 24(%rsp),%r12d + + addl 52(%rsp),%r12d + movl %r11d,%r13d + addl %edi,%r12d + movl %edx,%r14d + rorl $14,%r13d + movl %eax,%edi + + xorl %r11d,%r13d + rorl $9,%r14d + xorl %ebx,%edi + + movl %r12d,52(%rsp) + xorl %edx,%r14d + andl %r11d,%edi + + rorl $5,%r13d + addl %ecx,%r12d + xorl %ebx,%edi + + rorl $11,%r14d + xorl %r11d,%r13d + addl %edi,%r12d + + movl %edx,%edi + addl (%rbp),%r12d + xorl %edx,%r14d + + xorl %r8d,%edi + rorl $6,%r13d + movl %r8d,%ecx + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%ecx + addl %r12d,%r10d + addl %r12d,%ecx + + leaq 4(%rbp),%rbp + movl 60(%rsp),%r13d + movl 48(%rsp),%r15d + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ecx + movl %r15d,%r14d + rorl $2,%r15d + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%r15d + shrl $10,%r14d + + rorl $17,%r15d + xorl %r13d,%r12d + xorl %r14d,%r15d + addl 28(%rsp),%r12d + + addl 56(%rsp),%r12d + movl %r10d,%r13d + addl %r15d,%r12d + movl %ecx,%r14d + rorl $14,%r13d + movl %r11d,%r15d + + xorl %r10d,%r13d + rorl $9,%r14d + xorl %eax,%r15d + + movl %r12d,56(%rsp) + xorl %ecx,%r14d + andl %r10d,%r15d + + rorl $5,%r13d + addl %ebx,%r12d + xorl %eax,%r15d + + rorl $11,%r14d + xorl %r10d,%r13d + addl %r15d,%r12d + + movl %ecx,%r15d + addl (%rbp),%r12d + xorl %ecx,%r14d + + xorl %edx,%r15d + rorl $6,%r13d + movl %edx,%ebx + + andl %r15d,%edi + rorl $2,%r14d + addl %r13d,%r12d + + xorl %edi,%ebx + addl %r12d,%r9d + addl %r12d,%ebx + + leaq 4(%rbp),%rbp + movl 0(%rsp),%r13d + movl 52(%rsp),%edi + + movl %r13d,%r12d + rorl $11,%r13d + addl %r14d,%ebx + movl %edi,%r14d + rorl $2,%edi + + xorl %r12d,%r13d + shrl $3,%r12d + rorl $7,%r13d + xorl %r14d,%edi + shrl $10,%r14d + + rorl $17,%edi + xorl %r13d,%r12d + xorl %r14d,%edi + addl 32(%rsp),%r12d + + addl 60(%rsp),%r12d + movl %r9d,%r13d + addl %edi,%r12d + movl %ebx,%r14d + rorl $14,%r13d + movl %r10d,%edi + + xorl %r9d,%r13d + rorl $9,%r14d + xorl %r11d,%edi + + movl %r12d,60(%rsp) + xorl %ebx,%r14d + andl %r9d,%edi + + rorl $5,%r13d + addl %eax,%r12d + xorl %r11d,%edi + + rorl $11,%r14d + xorl %r9d,%r13d + addl %edi,%r12d + + movl %ebx,%edi + addl (%rbp),%r12d + xorl %ebx,%r14d + + xorl %ecx,%edi + rorl $6,%r13d + movl %ecx,%eax + + andl %edi,%r15d + rorl $2,%r14d + addl %r13d,%r12d + + xorl %r15d,%eax + addl %r12d,%r8d + addl %r12d,%eax + + leaq 20(%rbp),%rbp + cmpb $0,3(%rbp) + jnz .Lrounds_16_xx + + movq 64+0(%rsp),%rdi + addl %r14d,%eax + leaq 64(%rsi),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + cmpq 64+16(%rsp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb .Lloop + + leaq 64+24+48(%rsp),%r11 + + movq 64+24(%rsp),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbx + movq -8(%r11),%rbp +.LSEH_epilogue_crypton_sha256_asm_block_data_order: + mov 8(%r11),%rdi + mov 16(%r11),%rsi + + leaq (%r11),%rsp + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha256_asm_block_data_order: +.p2align 6 + +K256: +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 +.long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 + +.long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f +.long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f +.long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff +.long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff +.long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 +.long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 +.byte 83,72,65,50,53,54,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.def crypton_sha256_asm_block_data_order_shaext; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha256_asm_block_data_order_shaext: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha256_asm_block_data_order_shaext: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lshaext_shortcut: + subq $0x50,%rsp + + movaps %xmm6,-80(%rbp) + movaps %xmm7,-64(%rbp) + movaps %xmm8,-48(%rbp) + movaps %xmm9,-32(%rbp) + movaps %xmm10,-16(%rbp) + +.LSEH_body_crypton_sha256_asm_block_data_order_shaext: + + leaq K256+128(%rip),%rcx + movdqu (%rdi),%xmm1 + movdqu 16(%rdi),%xmm2 + movdqa 512-128(%rcx),%xmm7 + + pshufd $0x1b,%xmm1,%xmm0 + pshufd $0xb1,%xmm1,%xmm1 + pshufd $0x1b,%xmm2,%xmm2 + movdqa %xmm7,%xmm8 +.byte 102,15,58,15,202,8 + punpcklqdq %xmm0,%xmm2 + jmp .Loop_shaext + +.p2align 4 +.Loop_shaext: + movdqu (%rsi),%xmm3 + movdqu 16(%rsi),%xmm4 + movdqu 32(%rsi),%xmm5 +.byte 102,15,56,0,223 + movdqu 48(%rsi),%xmm6 + + movdqa 0-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 102,15,56,0,231 + movdqa %xmm2,%xmm10 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + nop + movdqa %xmm1,%xmm9 +.byte 15,56,203,202 + + movdqa 32-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 102,15,56,0,239 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + leaq 64(%rsi),%rsi +.byte 15,56,204,220 +.byte 15,56,203,202 + + movdqa 64-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 102,15,56,0,247 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + + movdqa 96-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 128-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 160-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 + nop + paddd %xmm7,%xmm6 +.byte 15,56,204,220 +.byte 15,56,203,202 + movdqa 192-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,205,245 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + movdqa 224-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 256-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 288-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 + nop + paddd %xmm7,%xmm6 +.byte 15,56,204,220 +.byte 15,56,203,202 + movdqa 320-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,205,245 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm6,%xmm7 +.byte 102,15,58,15,253,4 + nop + paddd %xmm7,%xmm3 +.byte 15,56,204,229 +.byte 15,56,203,202 + movdqa 352-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 +.byte 15,56,205,222 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm3,%xmm7 +.byte 102,15,58,15,254,4 + nop + paddd %xmm7,%xmm4 +.byte 15,56,204,238 +.byte 15,56,203,202 + movdqa 384-128(%rcx),%xmm0 + paddd %xmm3,%xmm0 +.byte 15,56,205,227 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm4,%xmm7 +.byte 102,15,58,15,251,4 + nop + paddd %xmm7,%xmm5 +.byte 15,56,204,243 +.byte 15,56,203,202 + movdqa 416-128(%rcx),%xmm0 + paddd %xmm4,%xmm0 +.byte 15,56,205,236 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + movdqa %xmm5,%xmm7 +.byte 102,15,58,15,252,4 +.byte 15,56,203,202 + paddd %xmm7,%xmm6 + + movdqa 448-128(%rcx),%xmm0 + paddd %xmm5,%xmm0 +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 +.byte 15,56,205,245 + movdqa %xmm8,%xmm7 +.byte 15,56,203,202 + + movdqa 480-128(%rcx),%xmm0 + paddd %xmm6,%xmm0 + nop +.byte 15,56,203,209 + pshufd $0x0e,%xmm0,%xmm0 + decq %rdx + nop +.byte 15,56,203,202 + + paddd %xmm10,%xmm2 + paddd %xmm9,%xmm1 + jnz .Loop_shaext + + pshufd $0xb1,%xmm2,%xmm2 + pshufd $0x1b,%xmm1,%xmm7 + pshufd $0xb1,%xmm1,%xmm1 + punpckhqdq %xmm2,%xmm1 +.byte 102,15,58,15,215,8 + + movdqu %xmm1,(%rdi) + movdqu %xmm2,16(%rdi) + movaps -80(%rbp),%xmm6 + movaps -64(%rbp),%xmm7 + movaps -48(%rbp),%xmm8 + movaps -32(%rbp),%xmm9 + movaps -16(%rbp),%xmm10 + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha256_asm_block_data_order_shaext: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha256_asm_block_data_order_shaext: +.def crypton_sha256_asm_block_data_order_ssse3; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha256_asm_block_data_order_ssse3: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha256_asm_block_data_order_ssse3: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lssse3_shortcut: + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $88,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + movaps %xmm6,-128(%rbp) + movaps %xmm7,-112(%rbp) + movaps %xmm8,-96(%rbp) + movaps %xmm9,-80(%rbp) + +.LSEH_body_crypton_sha256_asm_block_data_order_ssse3: + + + leaq -64(%rsp),%rsp + movl 0(%rdi),%eax + andq $-64,%rsp + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + + + jmp .Lloop_ssse3 +.p2align 4 +.Lloop_ssse3: + movdqa K256+512(%rip),%xmm7 + movq %rsi,-56(%rbp) + movdqu 0(%rsi),%xmm0 + movdqu 16(%rsi),%xmm1 + movdqu 32(%rsi),%xmm2 +.byte 102,15,56,0,199 + movdqu 48(%rsi),%xmm3 + leaq K256(%rip),%rsi +.byte 102,15,56,0,207 + movdqa 0(%rsi),%xmm4 + movdqa 32(%rsi),%xmm5 +.byte 102,15,56,0,215 + paddd %xmm0,%xmm4 + movdqa 64(%rsi),%xmm6 +.byte 102,15,56,0,223 + movdqa 96(%rsi),%xmm7 + paddd %xmm1,%xmm5 + paddd %xmm2,%xmm6 + paddd %xmm3,%xmm7 + movdqa %xmm4,0(%rsp) + movl %eax,%r14d + movdqa %xmm5,16(%rsp) + movl %ebx,%edi + movdqa %xmm6,32(%rsp) + xorl %ecx,%edi + movdqa %xmm7,48(%rsp) + movl %r8d,%r13d + jmp .Lssse3_00_47 + +.p2align 4 +.Lssse3_00_47: + subq $-128,%rsi + rorl $14,%r13d + movdqa %xmm1,%xmm4 + movl %r14d,%eax + movl %r9d,%r12d + movdqa %xmm3,%xmm7 + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d +.byte 102,15,58,15,224,4 + andl %r8d,%r12d + xorl %r8d,%r13d +.byte 102,15,58,15,250,4 + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %ebx,%r15d + addl %r12d,%r11d + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + paddd %xmm7,%xmm0 + rorl $2,%r14d + addl %r11d,%edx + psrld $7,%xmm6 + addl %edi,%r11d + movl %edx,%r13d + pshufd $250,%xmm3,%xmm7 + addl %r11d,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%r11d + movl %r8d,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %r11d,%r14d + pxor %xmm5,%xmm4 + andl %edx,%r12d + xorl %edx,%r13d + pslld $11,%xmm5 + addl 4(%rsp),%r10d + movl %r11d,%edi + pxor %xmm6,%xmm4 + xorl %r9d,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %eax,%edi + addl %r12d,%r10d + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + psrld $10,%xmm7 + addl %r13d,%r10d + xorl %eax,%r15d + paddd %xmm4,%xmm0 + rorl $2,%r14d + addl %r10d,%ecx + psrlq $17,%xmm6 + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %ecx,%r13d + xorl %r8d,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + psrldq $8,%xmm7 + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + paddd %xmm7,%xmm0 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + pshufd $80,%xmm0,%xmm7 + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + movdqa %xmm7,%xmm6 + addl %edi,%r9d + movl %ebx,%r13d + psrld $10,%xmm7 + addl %r9d,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%r9d + movl %ecx,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + psrlq $2,%xmm6 + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + pxor %xmm6,%xmm7 + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %r10d,%edi + addl %r12d,%r8d + movdqa 0(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + paddd %xmm7,%xmm0 + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + paddd %xmm0,%xmm6 + movl %eax,%r13d + addl %r8d,%r14d + movdqa %xmm6,0(%rsp) + rorl $14,%r13d + movdqa %xmm2,%xmm4 + movl %r14d,%r8d + movl %ebx,%r12d + movdqa %xmm0,%xmm7 + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d +.byte 102,15,58,15,225,4 + andl %eax,%r12d + xorl %eax,%r13d +.byte 102,15,58,15,251,4 + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %r9d,%r15d + addl %r12d,%edx + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + paddd %xmm7,%xmm1 + rorl $2,%r14d + addl %edx,%r11d + psrld $7,%xmm6 + addl %edi,%edx + movl %r11d,%r13d + pshufd $250,%xmm0,%xmm7 + addl %edx,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%edx + movl %eax,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %edx,%r14d + pxor %xmm5,%xmm4 + andl %r11d,%r12d + xorl %r11d,%r13d + pslld $11,%xmm5 + addl 20(%rsp),%ecx + movl %edx,%edi + pxor %xmm6,%xmm4 + xorl %ebx,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %r8d,%edi + addl %r12d,%ecx + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + psrld $10,%xmm7 + addl %r13d,%ecx + xorl %r8d,%r15d + paddd %xmm4,%xmm1 + rorl $2,%r14d + addl %ecx,%r10d + psrlq $17,%xmm6 + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %r10d,%r13d + xorl %eax,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + psrldq $8,%xmm7 + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + paddd %xmm7,%xmm1 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + pshufd $80,%xmm1,%xmm7 + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + movdqa %xmm7,%xmm6 + addl %edi,%ebx + movl %r9d,%r13d + psrld $10,%xmm7 + addl %ebx,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%ebx + movl %r10d,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + psrlq $2,%xmm6 + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + pxor %xmm6,%xmm7 + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %ecx,%edi + addl %r12d,%eax + movdqa 32(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + paddd %xmm7,%xmm1 + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + paddd %xmm1,%xmm6 + movl %r8d,%r13d + addl %eax,%r14d + movdqa %xmm6,16(%rsp) + rorl $14,%r13d + movdqa %xmm3,%xmm4 + movl %r14d,%eax + movl %r9d,%r12d + movdqa %xmm1,%xmm7 + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d +.byte 102,15,58,15,226,4 + andl %r8d,%r12d + xorl %r8d,%r13d +.byte 102,15,58,15,248,4 + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %ebx,%r15d + addl %r12d,%r11d + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + paddd %xmm7,%xmm2 + rorl $2,%r14d + addl %r11d,%edx + psrld $7,%xmm6 + addl %edi,%r11d + movl %edx,%r13d + pshufd $250,%xmm1,%xmm7 + addl %r11d,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%r11d + movl %r8d,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %r11d,%r14d + pxor %xmm5,%xmm4 + andl %edx,%r12d + xorl %edx,%r13d + pslld $11,%xmm5 + addl 36(%rsp),%r10d + movl %r11d,%edi + pxor %xmm6,%xmm4 + xorl %r9d,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %eax,%edi + addl %r12d,%r10d + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + psrld $10,%xmm7 + addl %r13d,%r10d + xorl %eax,%r15d + paddd %xmm4,%xmm2 + rorl $2,%r14d + addl %r10d,%ecx + psrlq $17,%xmm6 + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %ecx,%r13d + xorl %r8d,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + psrldq $8,%xmm7 + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + paddd %xmm7,%xmm2 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + pshufd $80,%xmm2,%xmm7 + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + movdqa %xmm7,%xmm6 + addl %edi,%r9d + movl %ebx,%r13d + psrld $10,%xmm7 + addl %r9d,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%r9d + movl %ecx,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + psrlq $2,%xmm6 + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + pxor %xmm6,%xmm7 + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %r10d,%edi + addl %r12d,%r8d + movdqa 64(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + paddd %xmm7,%xmm2 + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + paddd %xmm2,%xmm6 + movl %eax,%r13d + addl %r8d,%r14d + movdqa %xmm6,32(%rsp) + rorl $14,%r13d + movdqa %xmm0,%xmm4 + movl %r14d,%r8d + movl %ebx,%r12d + movdqa %xmm2,%xmm7 + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d +.byte 102,15,58,15,227,4 + andl %eax,%r12d + xorl %eax,%r13d +.byte 102,15,58,15,249,4 + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + movdqa %xmm4,%xmm5 + xorl %r9d,%r15d + addl %r12d,%edx + movdqa %xmm4,%xmm6 + rorl $6,%r13d + andl %r15d,%edi + psrld $3,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + paddd %xmm7,%xmm3 + rorl $2,%r14d + addl %edx,%r11d + psrld $7,%xmm6 + addl %edi,%edx + movl %r11d,%r13d + pshufd $250,%xmm2,%xmm7 + addl %edx,%r14d + rorl $14,%r13d + pslld $14,%xmm5 + movl %r14d,%edx + movl %eax,%r12d + pxor %xmm6,%xmm4 + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + psrld $11,%xmm6 + xorl %edx,%r14d + pxor %xmm5,%xmm4 + andl %r11d,%r12d + xorl %r11d,%r13d + pslld $11,%xmm5 + addl 52(%rsp),%ecx + movl %edx,%edi + pxor %xmm6,%xmm4 + xorl %ebx,%r12d + rorl $11,%r14d + movdqa %xmm7,%xmm6 + xorl %r8d,%edi + addl %r12d,%ecx + pxor %xmm5,%xmm4 + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + psrld $10,%xmm7 + addl %r13d,%ecx + xorl %r8d,%r15d + paddd %xmm4,%xmm3 + rorl $2,%r14d + addl %ecx,%r10d + psrlq $17,%xmm6 + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + pxor %xmm6,%xmm7 + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + psrlq $2,%xmm6 + xorl %r10d,%r13d + xorl %eax,%r12d + pxor %xmm6,%xmm7 + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + pshufd $128,%xmm7,%xmm7 + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + psrldq $8,%xmm7 + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + paddd %xmm7,%xmm3 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + pshufd $80,%xmm3,%xmm7 + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + movdqa %xmm7,%xmm6 + addl %edi,%ebx + movl %r9d,%r13d + psrld $10,%xmm7 + addl %ebx,%r14d + rorl $14,%r13d + psrlq $17,%xmm6 + movl %r14d,%ebx + movl %r10d,%r12d + pxor %xmm6,%xmm7 + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + psrlq $2,%xmm6 + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + pxor %xmm6,%xmm7 + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + pshufd $8,%xmm7,%xmm7 + xorl %ecx,%edi + addl %r12d,%eax + movdqa 96(%rsi),%xmm6 + rorl $6,%r13d + andl %edi,%r15d + pslldq $8,%xmm7 + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + paddd %xmm7,%xmm3 + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + paddd %xmm3,%xmm6 + movl %r8d,%r13d + addl %eax,%r14d + movdqa %xmm6,48(%rsp) + cmpb $0,131(%rsi) + jne .Lssse3_00_47 + rorl $14,%r13d + movl %r14d,%eax + movl %r9d,%r12d + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + rorl $6,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + rorl $2,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + rorl $14,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + rorl $11,%r14d + xorl %eax,%edi + addl %r12d,%r10d + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + rorl $2,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + rorl $14,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + rorl $6,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + rorl $14,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + rorl $6,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + rorl $2,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + rorl $14,%r13d + movl %r14d,%edx + movl %eax,%r12d + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + rorl $11,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + rorl $2,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + rorl $14,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + xorl %ecx,%edi + addl %r12d,%eax + rorl $6,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + rorl $14,%r13d + movl %r14d,%eax + movl %r9d,%r12d + rorl $9,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + rorl $5,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + rorl $11,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + rorl $6,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + rorl $2,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + rorl $14,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + rorl $9,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + rorl $5,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + rorl $11,%r14d + xorl %eax,%edi + addl %r12d,%r10d + rorl $6,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + rorl $2,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + rorl $14,%r13d + movl %r14d,%r10d + movl %edx,%r12d + rorl $9,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + rorl $5,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + rorl $11,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + rorl $6,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + rorl $2,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + rorl $14,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + rorl $9,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + rorl $5,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + rorl $11,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + rorl $6,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + rorl $2,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + rorl $14,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + rorl $9,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + rorl $5,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + rorl $11,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + rorl $6,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + rorl $2,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + rorl $14,%r13d + movl %r14d,%edx + movl %eax,%r12d + rorl $9,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + rorl $5,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + rorl $11,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + rorl $6,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + rorl $2,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + rorl $14,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + rorl $9,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + rorl $5,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + rorl $11,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + rorl $6,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + rorl $2,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + rorl $14,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + rorl $9,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + rorl $5,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + rorl $11,%r14d + xorl %ecx,%edi + addl %r12d,%eax + rorl $6,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + rorl $2,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + movq -64(%rbp),%rdi + movl %r14d,%eax + movq -56(%rbp),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + leaq 64(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb .Lloop_ssse3 + + movaps -128(%rbp),%xmm6 + movaps -112(%rbp),%xmm7 + movaps -96(%rbp),%xmm8 + movaps -80(%rbp),%xmm9 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha256_asm_block_data_order_ssse3: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha256_asm_block_data_order_ssse3: +.def crypton_sha256_asm_block_data_order_avx; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha256_asm_block_data_order_avx: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha256_asm_block_data_order_avx: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lavx_shortcut: + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $120,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + movaps %xmm6,-160(%rbp) + movaps %xmm7,-144(%rbp) + movaps %xmm8,-128(%rbp) + movaps %xmm9,-112(%rbp) + +.LSEH_body_crypton_sha256_asm_block_data_order_avx: + + + leaq -64(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movl 0(%rdi),%eax + movl 4(%rdi),%ebx + movl 8(%rdi),%ecx + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + vmovdqa K256+512+32(%rip),%xmm8 + vmovdqa K256+512+64(%rip),%xmm9 + jmp .Lloop_avx +.p2align 4 +.Lloop_avx: + vmovdqa K256+512(%rip),%xmm7 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm7,%xmm0,%xmm0 + leaq K256(%rip),%rsi + vpshufb %xmm7,%xmm1,%xmm1 + vpshufb %xmm7,%xmm2,%xmm2 + vpaddd 0(%rsi),%xmm0,%xmm4 + vpshufb %xmm7,%xmm3,%xmm3 + vpaddd 32(%rsi),%xmm1,%xmm5 + vpaddd 64(%rsi),%xmm2,%xmm6 + vpaddd 96(%rsi),%xmm3,%xmm7 + vmovdqa %xmm4,0(%rsp) + movl %eax,%r14d + vmovdqa %xmm5,16(%rsp) + movl %ebx,%edi + vmovdqa %xmm6,32(%rsp) + xorl %ecx,%edi + vmovdqa %xmm7,48(%rsp) + movl %r8d,%r13d + jmp .Lavx_00_47 + +.p2align 4 +.Lavx_00_47: + subq $-128,%rsi + vpalignr $4,%xmm0,%xmm1,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + vpalignr $4,%xmm2,%xmm3,%xmm7 + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + vpaddd %xmm7,%xmm0,%xmm0 + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + vpshufd $250,%xmm3,%xmm7 + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + vpsrld $11,%xmm6,%xmm6 + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + vpaddd %xmm4,%xmm0,%xmm0 + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + vpxor %xmm7,%xmm6,%xmm6 + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + vpaddd %xmm6,%xmm0,%xmm0 + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + vpshufd $80,%xmm0,%xmm7 + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + vpxor %xmm7,%xmm6,%xmm6 + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + vpaddd %xmm6,%xmm0,%xmm0 + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + vpaddd 0(%rsi),%xmm0,%xmm6 + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + vmovdqa %xmm6,0(%rsp) + vpalignr $4,%xmm1,%xmm2,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + vpalignr $4,%xmm3,%xmm0,%xmm7 + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + vpaddd %xmm7,%xmm1,%xmm1 + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + vpshufd $250,%xmm0,%xmm7 + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + vpsrld $11,%xmm6,%xmm6 + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + vpaddd %xmm4,%xmm1,%xmm1 + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + vpxor %xmm7,%xmm6,%xmm6 + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + vpaddd %xmm6,%xmm1,%xmm1 + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + vpshufd $80,%xmm1,%xmm7 + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + vpxor %xmm7,%xmm6,%xmm6 + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + vpaddd %xmm6,%xmm1,%xmm1 + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + vpaddd 32(%rsi),%xmm1,%xmm6 + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + vmovdqa %xmm6,16(%rsp) + vpalignr $4,%xmm2,%xmm3,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + vpalignr $4,%xmm0,%xmm1,%xmm7 + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + vpaddd %xmm7,%xmm2,%xmm2 + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + vpshufd $250,%xmm1,%xmm7 + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + vpsrld $11,%xmm6,%xmm6 + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + vpaddd %xmm4,%xmm2,%xmm2 + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + vpxor %xmm7,%xmm6,%xmm6 + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + vpaddd %xmm6,%xmm2,%xmm2 + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + vpshufd $80,%xmm2,%xmm7 + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + vpxor %xmm7,%xmm6,%xmm6 + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + vpaddd %xmm6,%xmm2,%xmm2 + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + vpaddd 64(%rsi),%xmm2,%xmm6 + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + vmovdqa %xmm6,32(%rsp) + vpalignr $4,%xmm3,%xmm0,%xmm4 + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + vpalignr $4,%xmm1,%xmm2,%xmm7 + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + vpsrld $7,%xmm4,%xmm6 + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + vpaddd %xmm7,%xmm3,%xmm3 + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + vpsrld $3,%xmm4,%xmm7 + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + vpslld $14,%xmm4,%xmm5 + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + vpxor %xmm6,%xmm7,%xmm4 + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + vpshufd $250,%xmm2,%xmm7 + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + vpsrld $11,%xmm6,%xmm6 + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + vpxor %xmm5,%xmm4,%xmm4 + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + vpslld $11,%xmm5,%xmm5 + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + vpxor %xmm6,%xmm4,%xmm4 + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + vpsrld $10,%xmm7,%xmm6 + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + vpxor %xmm5,%xmm4,%xmm4 + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + vpsrlq $17,%xmm7,%xmm7 + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + vpaddd %xmm4,%xmm3,%xmm3 + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + vpxor %xmm7,%xmm6,%xmm6 + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + vpsrlq $2,%xmm7,%xmm7 + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + vpxor %xmm7,%xmm6,%xmm6 + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + vpshufb %xmm8,%xmm6,%xmm6 + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + vpaddd %xmm6,%xmm3,%xmm3 + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + vpshufd $80,%xmm3,%xmm7 + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + vpsrld $10,%xmm7,%xmm6 + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + vpsrlq $17,%xmm7,%xmm7 + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + vpxor %xmm7,%xmm6,%xmm6 + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + vpsrlq $2,%xmm7,%xmm7 + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + vpxor %xmm7,%xmm6,%xmm6 + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + vpshufb %xmm9,%xmm6,%xmm6 + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + vpaddd %xmm6,%xmm3,%xmm3 + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + vpaddd 96(%rsi),%xmm3,%xmm6 + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + vmovdqa %xmm6,48(%rsp) + cmpb $0,131(%rsi) + jne .Lavx_00_47 + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 0(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 4(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 8(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 12(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 16(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 20(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 24(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 28(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%eax + movl %r9d,%r12d + shrdl $9,%r14d,%r14d + xorl %r8d,%r13d + xorl %r10d,%r12d + shrdl $5,%r13d,%r13d + xorl %eax,%r14d + andl %r8d,%r12d + xorl %r8d,%r13d + addl 32(%rsp),%r11d + movl %eax,%r15d + xorl %r10d,%r12d + shrdl $11,%r14d,%r14d + xorl %ebx,%r15d + addl %r12d,%r11d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %eax,%r14d + addl %r13d,%r11d + xorl %ebx,%edi + shrdl $2,%r14d,%r14d + addl %r11d,%edx + addl %edi,%r11d + movl %edx,%r13d + addl %r11d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r11d + movl %r8d,%r12d + shrdl $9,%r14d,%r14d + xorl %edx,%r13d + xorl %r9d,%r12d + shrdl $5,%r13d,%r13d + xorl %r11d,%r14d + andl %edx,%r12d + xorl %edx,%r13d + addl 36(%rsp),%r10d + movl %r11d,%edi + xorl %r9d,%r12d + shrdl $11,%r14d,%r14d + xorl %eax,%edi + addl %r12d,%r10d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r11d,%r14d + addl %r13d,%r10d + xorl %eax,%r15d + shrdl $2,%r14d,%r14d + addl %r10d,%ecx + addl %r15d,%r10d + movl %ecx,%r13d + addl %r10d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r10d + movl %edx,%r12d + shrdl $9,%r14d,%r14d + xorl %ecx,%r13d + xorl %r8d,%r12d + shrdl $5,%r13d,%r13d + xorl %r10d,%r14d + andl %ecx,%r12d + xorl %ecx,%r13d + addl 40(%rsp),%r9d + movl %r10d,%r15d + xorl %r8d,%r12d + shrdl $11,%r14d,%r14d + xorl %r11d,%r15d + addl %r12d,%r9d + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r10d,%r14d + addl %r13d,%r9d + xorl %r11d,%edi + shrdl $2,%r14d,%r14d + addl %r9d,%ebx + addl %edi,%r9d + movl %ebx,%r13d + addl %r9d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r9d + movl %ecx,%r12d + shrdl $9,%r14d,%r14d + xorl %ebx,%r13d + xorl %edx,%r12d + shrdl $5,%r13d,%r13d + xorl %r9d,%r14d + andl %ebx,%r12d + xorl %ebx,%r13d + addl 44(%rsp),%r8d + movl %r9d,%edi + xorl %edx,%r12d + shrdl $11,%r14d,%r14d + xorl %r10d,%edi + addl %r12d,%r8d + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %r9d,%r14d + addl %r13d,%r8d + xorl %r10d,%r15d + shrdl $2,%r14d,%r14d + addl %r8d,%eax + addl %r15d,%r8d + movl %eax,%r13d + addl %r8d,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%r8d + movl %ebx,%r12d + shrdl $9,%r14d,%r14d + xorl %eax,%r13d + xorl %ecx,%r12d + shrdl $5,%r13d,%r13d + xorl %r8d,%r14d + andl %eax,%r12d + xorl %eax,%r13d + addl 48(%rsp),%edx + movl %r8d,%r15d + xorl %ecx,%r12d + shrdl $11,%r14d,%r14d + xorl %r9d,%r15d + addl %r12d,%edx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %r8d,%r14d + addl %r13d,%edx + xorl %r9d,%edi + shrdl $2,%r14d,%r14d + addl %edx,%r11d + addl %edi,%edx + movl %r11d,%r13d + addl %edx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%edx + movl %eax,%r12d + shrdl $9,%r14d,%r14d + xorl %r11d,%r13d + xorl %ebx,%r12d + shrdl $5,%r13d,%r13d + xorl %edx,%r14d + andl %r11d,%r12d + xorl %r11d,%r13d + addl 52(%rsp),%ecx + movl %edx,%edi + xorl %ebx,%r12d + shrdl $11,%r14d,%r14d + xorl %r8d,%edi + addl %r12d,%ecx + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %edx,%r14d + addl %r13d,%ecx + xorl %r8d,%r15d + shrdl $2,%r14d,%r14d + addl %ecx,%r10d + addl %r15d,%ecx + movl %r10d,%r13d + addl %ecx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ecx + movl %r11d,%r12d + shrdl $9,%r14d,%r14d + xorl %r10d,%r13d + xorl %eax,%r12d + shrdl $5,%r13d,%r13d + xorl %ecx,%r14d + andl %r10d,%r12d + xorl %r10d,%r13d + addl 56(%rsp),%ebx + movl %ecx,%r15d + xorl %eax,%r12d + shrdl $11,%r14d,%r14d + xorl %edx,%r15d + addl %r12d,%ebx + shrdl $6,%r13d,%r13d + andl %r15d,%edi + xorl %ecx,%r14d + addl %r13d,%ebx + xorl %edx,%edi + shrdl $2,%r14d,%r14d + addl %ebx,%r9d + addl %edi,%ebx + movl %r9d,%r13d + addl %ebx,%r14d + shrdl $14,%r13d,%r13d + movl %r14d,%ebx + movl %r10d,%r12d + shrdl $9,%r14d,%r14d + xorl %r9d,%r13d + xorl %r11d,%r12d + shrdl $5,%r13d,%r13d + xorl %ebx,%r14d + andl %r9d,%r12d + xorl %r9d,%r13d + addl 60(%rsp),%eax + movl %ebx,%edi + xorl %r11d,%r12d + shrdl $11,%r14d,%r14d + xorl %ecx,%edi + addl %r12d,%eax + shrdl $6,%r13d,%r13d + andl %edi,%r15d + xorl %ebx,%r14d + addl %r13d,%eax + xorl %ecx,%r15d + shrdl $2,%r14d,%r14d + addl %eax,%r8d + addl %r15d,%eax + movl %r8d,%r13d + addl %eax,%r14d + movq -64(%rbp),%rdi + movl %r14d,%eax + movq -56(%rbp),%rsi + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + leaq 64(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + jb .Lloop_avx + + vzeroupper + movaps -160(%rbp),%xmm6 + movaps -144(%rbp),%xmm7 + movaps -128(%rbp),%xmm8 + movaps -112(%rbp),%xmm9 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha256_asm_block_data_order_avx: +.def crypton_sha256_asm_block_data_order_avx2; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha256_asm_block_data_order_avx2: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha256_asm_block_data_order_avx2: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lavx2_shortcut: + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $120,%rsp + + leaq (%rsi,%rdx,4),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + movaps %xmm6,-160(%rbp) + movaps %xmm7,-144(%rbp) + movaps %xmm8,-128(%rbp) + movaps %xmm9,-112(%rbp) + +.LSEH_body_crypton_sha256_asm_block_data_order_avx2: + + + leaq -64(%rsp),%rsp + vzeroupper + andq $-64,%rsp + subq $-64,%rsi + movl 0(%rdi),%eax + movq %rsi,%r12 + movl 4(%rdi),%ebx + cmpq %rdx,%rsi + movl 8(%rdi),%ecx + cmoveq %rsp,%r12 + movl 12(%rdi),%edx + movl 16(%rdi),%r8d + movl 20(%rdi),%r9d + movl 24(%rdi),%r10d + movl 28(%rdi),%r11d + vmovdqa K256+512+32(%rip),%ymm8 + vmovdqa K256+512+64(%rip),%ymm9 + jmp .Loop_avx2 +.p2align 4 +.Loop_avx2: + vmovdqa K256+512(%rip),%ymm7 + movq %rsi,-56(%rbp) + vmovdqu -64+0(%rsi),%xmm0 + vmovdqu -64+16(%rsi),%xmm1 + vmovdqu -64+32(%rsi),%xmm2 + vmovdqu -64+48(%rsi),%xmm3 + leaq K256(%rip),%rsi + vinserti128 $1,(%r12),%ymm0,%ymm0 + vinserti128 $1,16(%r12),%ymm1,%ymm1 + vpshufb %ymm7,%ymm0,%ymm0 + vinserti128 $1,32(%r12),%ymm2,%ymm2 + vpshufb %ymm7,%ymm1,%ymm1 + vinserti128 $1,48(%r12),%ymm3,%ymm3 + + vpshufb %ymm7,%ymm2,%ymm2 + vpaddd 0(%rsi),%ymm0,%ymm4 + vpshufb %ymm7,%ymm3,%ymm3 + vpaddd 32(%rsi),%ymm1,%ymm5 + vpaddd 64(%rsi),%ymm2,%ymm6 + vpaddd 96(%rsi),%ymm3,%ymm7 + vmovdqa %ymm4,0(%rsp) + xorl %r14d,%r14d + vmovdqa %ymm5,32(%rsp) + leaq -64(%rsp),%rsp + movl %ebx,%edi + vmovdqa %ymm6,0(%rsp) + xorl %ecx,%edi + vmovdqa %ymm7,32(%rsp) + movl %r9d,%r12d + subq $-32*4,%rsi + jmp .Lavx2_00_47 + +.p2align 4 +.Lavx2_00_47: + leaq -64(%rsp),%rsp + vpalignr $4,%ymm0,%ymm1,%ymm4 + addl 0+128(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + vpalignr $4,%ymm2,%ymm3,%ymm7 + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + vpsrld $7,%ymm4,%ymm6 + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + vpaddd %ymm7,%ymm0,%ymm0 + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + vpshufd $250,%ymm3,%ymm7 + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 4+128(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + vpslld $11,%ymm5,%ymm5 + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + vpaddd %ymm4,%ymm0,%ymm0 + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 8+128(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + vpxor %ymm7,%ymm6,%ymm6 + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + vpaddd %ymm6,%ymm0,%ymm0 + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + vpshufd $80,%ymm0,%ymm7 + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 12+128(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + vpxor %ymm7,%ymm6,%ymm6 + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + vpaddd %ymm6,%ymm0,%ymm0 + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + vpaddd 0(%rsi),%ymm0,%ymm6 + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + vmovdqa %ymm6,0(%rsp) + vpalignr $4,%ymm1,%ymm2,%ymm4 + addl 32+128(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + vpalignr $4,%ymm3,%ymm0,%ymm7 + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + vpsrld $7,%ymm4,%ymm6 + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + vpaddd %ymm7,%ymm1,%ymm1 + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + vpshufd $250,%ymm0,%ymm7 + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 36+128(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + vpslld $11,%ymm5,%ymm5 + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + vpaddd %ymm4,%ymm1,%ymm1 + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 40+128(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + vpxor %ymm7,%ymm6,%ymm6 + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + vpaddd %ymm6,%ymm1,%ymm1 + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + vpshufd $80,%ymm1,%ymm7 + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 44+128(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + vpxor %ymm7,%ymm6,%ymm6 + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + vpaddd %ymm6,%ymm1,%ymm1 + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + vpaddd 32(%rsi),%ymm1,%ymm6 + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + vmovdqa %ymm6,32(%rsp) + leaq -64(%rsp),%rsp + vpalignr $4,%ymm2,%ymm3,%ymm4 + addl 0+128(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + vpalignr $4,%ymm0,%ymm1,%ymm7 + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + vpsrld $7,%ymm4,%ymm6 + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + vpaddd %ymm7,%ymm2,%ymm2 + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + vpshufd $250,%ymm1,%ymm7 + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 4+128(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + vpslld $11,%ymm5,%ymm5 + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + vpaddd %ymm4,%ymm2,%ymm2 + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 8+128(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + vpxor %ymm7,%ymm6,%ymm6 + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + vpaddd %ymm6,%ymm2,%ymm2 + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + vpshufd $80,%ymm2,%ymm7 + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 12+128(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + vpxor %ymm7,%ymm6,%ymm6 + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + vpaddd %ymm6,%ymm2,%ymm2 + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + vpaddd 64(%rsi),%ymm2,%ymm6 + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + vmovdqa %ymm6,0(%rsp) + vpalignr $4,%ymm3,%ymm0,%ymm4 + addl 32+128(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + vpalignr $4,%ymm1,%ymm2,%ymm7 + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + vpsrld $7,%ymm4,%ymm6 + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + vpaddd %ymm7,%ymm3,%ymm3 + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + vpsrld $3,%ymm4,%ymm7 + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + vpslld $14,%ymm4,%ymm5 + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + vpxor %ymm6,%ymm7,%ymm4 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + vpshufd $250,%ymm2,%ymm7 + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + vpsrld $11,%ymm6,%ymm6 + addl 36+128(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + vpxor %ymm5,%ymm4,%ymm4 + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + vpslld $11,%ymm5,%ymm5 + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + vpxor %ymm6,%ymm4,%ymm4 + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + vpsrld $10,%ymm7,%ymm6 + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + vpxor %ymm5,%ymm4,%ymm4 + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + vpsrlq $17,%ymm7,%ymm7 + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + vpaddd %ymm4,%ymm3,%ymm3 + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 40+128(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + vpxor %ymm7,%ymm6,%ymm6 + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + vpshufb %ymm8,%ymm6,%ymm6 + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + vpaddd %ymm6,%ymm3,%ymm3 + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + vpshufd $80,%ymm3,%ymm7 + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + vpsrld $10,%ymm7,%ymm6 + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + vpsrlq $17,%ymm7,%ymm7 + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + vpxor %ymm7,%ymm6,%ymm6 + addl 44+128(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + vpsrlq $2,%ymm7,%ymm7 + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + vpxor %ymm7,%ymm6,%ymm6 + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + vpshufb %ymm9,%ymm6,%ymm6 + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + vpaddd %ymm6,%ymm3,%ymm3 + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + vpaddd 96(%rsi),%ymm3,%ymm6 + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + vmovdqa %ymm6,32(%rsp) + leaq 128(%rsi),%rsi + cmpb $0,3(%rsi) + jne .Lavx2_00_47 + addl 0+64(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4+64(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8+64(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12+64(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32+64(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36+64(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40+64(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44+64(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + addl 0(%rsp),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4(%rsp),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8(%rsp),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12(%rsp),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32(%rsp),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36(%rsp),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40(%rsp),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44(%rsp),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + movq -64(%rbp),%rdi + addl %r14d,%eax + movl -56(%rbp),%r12d + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + addl 24(%rdi),%r10d + addl 28(%rdi),%r11d + + movl %eax,0(%rdi) + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + + cmpl -48(%rbp),%r12d + je .Ldone_avx2 + + leaq 448(%rsp),%rsi + xorl %r14d,%r14d + movl %ebx,%edi + xorl %ecx,%edi + movl %r9d,%r12d + jmp .Lower_avx2 +.p2align 4 +.Lower_avx2: + addl 0+16(%rsi),%r11d + andl %r8d,%r12d + rorxl $25,%r8d,%r13d + rorxl $11,%r8d,%r15d + leal (%rax,%r14,1),%eax + leal (%r11,%r12,1),%r11d + andnl %r10d,%r8d,%r12d + xorl %r15d,%r13d + rorxl $6,%r8d,%r14d + leal (%r11,%r12,1),%r11d + xorl %r14d,%r13d + movl %eax,%r15d + rorxl $22,%eax,%r12d + leal (%r11,%r13,1),%r11d + xorl %ebx,%r15d + rorxl $13,%eax,%r14d + rorxl $2,%eax,%r13d + leal (%rdx,%r11,1),%edx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %ebx,%edi + xorl %r13d,%r14d + leal (%r11,%rdi,1),%r11d + movl %r8d,%r12d + addl 4+16(%rsi),%r10d + andl %edx,%r12d + rorxl $25,%edx,%r13d + rorxl $11,%edx,%edi + leal (%r11,%r14,1),%r11d + leal (%r10,%r12,1),%r10d + andnl %r9d,%edx,%r12d + xorl %edi,%r13d + rorxl $6,%edx,%r14d + leal (%r10,%r12,1),%r10d + xorl %r14d,%r13d + movl %r11d,%edi + rorxl $22,%r11d,%r12d + leal (%r10,%r13,1),%r10d + xorl %eax,%edi + rorxl $13,%r11d,%r14d + rorxl $2,%r11d,%r13d + leal (%rcx,%r10,1),%ecx + andl %edi,%r15d + xorl %r12d,%r14d + xorl %eax,%r15d + xorl %r13d,%r14d + leal (%r10,%r15,1),%r10d + movl %edx,%r12d + addl 8+16(%rsi),%r9d + andl %ecx,%r12d + rorxl $25,%ecx,%r13d + rorxl $11,%ecx,%r15d + leal (%r10,%r14,1),%r10d + leal (%r9,%r12,1),%r9d + andnl %r8d,%ecx,%r12d + xorl %r15d,%r13d + rorxl $6,%ecx,%r14d + leal (%r9,%r12,1),%r9d + xorl %r14d,%r13d + movl %r10d,%r15d + rorxl $22,%r10d,%r12d + leal (%r9,%r13,1),%r9d + xorl %r11d,%r15d + rorxl $13,%r10d,%r14d + rorxl $2,%r10d,%r13d + leal (%rbx,%r9,1),%ebx + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r11d,%edi + xorl %r13d,%r14d + leal (%r9,%rdi,1),%r9d + movl %ecx,%r12d + addl 12+16(%rsi),%r8d + andl %ebx,%r12d + rorxl $25,%ebx,%r13d + rorxl $11,%ebx,%edi + leal (%r9,%r14,1),%r9d + leal (%r8,%r12,1),%r8d + andnl %edx,%ebx,%r12d + xorl %edi,%r13d + rorxl $6,%ebx,%r14d + leal (%r8,%r12,1),%r8d + xorl %r14d,%r13d + movl %r9d,%edi + rorxl $22,%r9d,%r12d + leal (%r8,%r13,1),%r8d + xorl %r10d,%edi + rorxl $13,%r9d,%r14d + rorxl $2,%r9d,%r13d + leal (%rax,%r8,1),%eax + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r10d,%r15d + xorl %r13d,%r14d + leal (%r8,%r15,1),%r8d + movl %ebx,%r12d + addl 32+16(%rsi),%edx + andl %eax,%r12d + rorxl $25,%eax,%r13d + rorxl $11,%eax,%r15d + leal (%r8,%r14,1),%r8d + leal (%rdx,%r12,1),%edx + andnl %ecx,%eax,%r12d + xorl %r15d,%r13d + rorxl $6,%eax,%r14d + leal (%rdx,%r12,1),%edx + xorl %r14d,%r13d + movl %r8d,%r15d + rorxl $22,%r8d,%r12d + leal (%rdx,%r13,1),%edx + xorl %r9d,%r15d + rorxl $13,%r8d,%r14d + rorxl $2,%r8d,%r13d + leal (%r11,%rdx,1),%r11d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %r9d,%edi + xorl %r13d,%r14d + leal (%rdx,%rdi,1),%edx + movl %eax,%r12d + addl 36+16(%rsi),%ecx + andl %r11d,%r12d + rorxl $25,%r11d,%r13d + rorxl $11,%r11d,%edi + leal (%rdx,%r14,1),%edx + leal (%rcx,%r12,1),%ecx + andnl %ebx,%r11d,%r12d + xorl %edi,%r13d + rorxl $6,%r11d,%r14d + leal (%rcx,%r12,1),%ecx + xorl %r14d,%r13d + movl %edx,%edi + rorxl $22,%edx,%r12d + leal (%rcx,%r13,1),%ecx + xorl %r8d,%edi + rorxl $13,%edx,%r14d + rorxl $2,%edx,%r13d + leal (%r10,%rcx,1),%r10d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %r8d,%r15d + xorl %r13d,%r14d + leal (%rcx,%r15,1),%ecx + movl %r11d,%r12d + addl 40+16(%rsi),%ebx + andl %r10d,%r12d + rorxl $25,%r10d,%r13d + rorxl $11,%r10d,%r15d + leal (%rcx,%r14,1),%ecx + leal (%rbx,%r12,1),%ebx + andnl %eax,%r10d,%r12d + xorl %r15d,%r13d + rorxl $6,%r10d,%r14d + leal (%rbx,%r12,1),%ebx + xorl %r14d,%r13d + movl %ecx,%r15d + rorxl $22,%ecx,%r12d + leal (%rbx,%r13,1),%ebx + xorl %edx,%r15d + rorxl $13,%ecx,%r14d + rorxl $2,%ecx,%r13d + leal (%r9,%rbx,1),%r9d + andl %r15d,%edi + xorl %r12d,%r14d + xorl %edx,%edi + xorl %r13d,%r14d + leal (%rbx,%rdi,1),%ebx + movl %r10d,%r12d + addl 44+16(%rsi),%eax + andl %r9d,%r12d + rorxl $25,%r9d,%r13d + rorxl $11,%r9d,%edi + leal (%rbx,%r14,1),%ebx + leal (%rax,%r12,1),%eax + andnl %r11d,%r9d,%r12d + xorl %edi,%r13d + rorxl $6,%r9d,%r14d + leal (%rax,%r12,1),%eax + xorl %r14d,%r13d + movl %ebx,%edi + rorxl $22,%ebx,%r12d + leal (%rax,%r13,1),%eax + xorl %ecx,%edi + rorxl $13,%ebx,%r14d + rorxl $2,%ebx,%r13d + leal (%r8,%rax,1),%r8d + andl %edi,%r15d + xorl %r12d,%r14d + xorl %ecx,%r15d + xorl %r13d,%r14d + leal (%rax,%r15,1),%eax + movl %r9d,%r12d + leaq -64(%rsi),%rsi + cmpq %rsp,%rsi + jae .Lower_avx2 + + movq -64(%rbp),%rdi + addl %r14d,%eax + movq -56(%rbp),%rsi + leaq 448(%rsp),%rsp + + addl 0(%rdi),%eax + addl 4(%rdi),%ebx + addl 8(%rdi),%ecx + addl 12(%rdi),%edx + addl 16(%rdi),%r8d + addl 20(%rdi),%r9d + leaq 128(%rsi),%rsi + addl 24(%rdi),%r10d + movq %rsi,%r12 + addl 28(%rdi),%r11d + cmpq -48(%rbp),%rsi + + movl %eax,0(%rdi) + cmoveq %rsp,%r12 + movl %ebx,4(%rdi) + movl %ecx,8(%rdi) + movl %edx,12(%rdi) + movl %r8d,16(%rdi) + movl %r9d,20(%rdi) + movl %r10d,24(%rdi) + movl %r11d,28(%rdi) + + jbe .Loop_avx2 + +.Ldone_avx2: + vzeroupper + movaps -160(%rbp),%xmm6 + movaps -144(%rbp),%xmm7 + movaps -128(%rbp),%xmm8 + movaps -112(%rbp),%xmm9 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha256_asm_block_data_order_avx2: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha256_asm_block_data_order_avx2: +.section .pdata +.p2align 2 +.rva .LSEH_begin_crypton_sha256_asm_block_data_order +.rva .LSEH_body_crypton_sha256_asm_block_data_order +.rva .LSEH_info_crypton_sha256_asm_block_data_order_prologue + +.rva .LSEH_body_crypton_sha256_asm_block_data_order +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order +.rva .LSEH_info_crypton_sha256_asm_block_data_order_body + +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order +.rva .LSEH_end_crypton_sha256_asm_block_data_order +.rva .LSEH_info_crypton_sha256_asm_block_data_order_epilogue + +.rva .LSEH_begin_crypton_sha256_asm_block_data_order_shaext +.rva .LSEH_body_crypton_sha256_asm_block_data_order_shaext +.rva .LSEH_info_crypton_sha256_asm_block_data_order_shaext_prologue + +.rva .LSEH_body_crypton_sha256_asm_block_data_order_shaext +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_shaext +.rva .LSEH_info_crypton_sha256_asm_block_data_order_shaext_body + +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_shaext +.rva .LSEH_end_crypton_sha256_asm_block_data_order_shaext +.rva .LSEH_info_crypton_sha256_asm_block_data_order_shaext_epilogue + +.rva .LSEH_begin_crypton_sha256_asm_block_data_order_ssse3 +.rva .LSEH_body_crypton_sha256_asm_block_data_order_ssse3 +.rva .LSEH_info_crypton_sha256_asm_block_data_order_ssse3_prologue + +.rva .LSEH_body_crypton_sha256_asm_block_data_order_ssse3 +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_ssse3 +.rva .LSEH_info_crypton_sha256_asm_block_data_order_ssse3_body + +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_ssse3 +.rva .LSEH_end_crypton_sha256_asm_block_data_order_ssse3 +.rva .LSEH_info_crypton_sha256_asm_block_data_order_ssse3_epilogue + +.rva .LSEH_begin_crypton_sha256_asm_block_data_order_avx +.rva .LSEH_body_crypton_sha256_asm_block_data_order_avx +.rva .LSEH_info_crypton_sha256_asm_block_data_order_avx_prologue + +.rva .LSEH_body_crypton_sha256_asm_block_data_order_avx +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_avx +.rva .LSEH_info_crypton_sha256_asm_block_data_order_avx_body + +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_avx +.rva .LSEH_end_crypton_sha256_asm_block_data_order_avx +.rva .LSEH_info_crypton_sha256_asm_block_data_order_avx_epilogue + +.rva .LSEH_begin_crypton_sha256_asm_block_data_order_avx2 +.rva .LSEH_body_crypton_sha256_asm_block_data_order_avx2 +.rva .LSEH_info_crypton_sha256_asm_block_data_order_avx2_prologue + +.rva .LSEH_body_crypton_sha256_asm_block_data_order_avx2 +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_avx2 +.rva .LSEH_info_crypton_sha256_asm_block_data_order_avx2_body + +.rva .LSEH_epilogue_crypton_sha256_asm_block_data_order_avx2 +.rva .LSEH_end_crypton_sha256_asm_block_data_order_avx2 +.rva .LSEH_info_crypton_sha256_asm_block_data_order_avx2_epilogue + +.section .xdata +.p2align 3 +.LSEH_info_crypton_sha256_asm_block_data_order_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha256_asm_block_data_order_body: +.byte 1,0,18,0 +.byte 0x00,0xf4,0x0b,0x00 +.byte 0x00,0xe4,0x0c,0x00 +.byte 0x00,0xd4,0x0d,0x00 +.byte 0x00,0xc4,0x0e,0x00 +.byte 0x00,0x34,0x0f,0x00 +.byte 0x00,0x54,0x10,0x00 +.byte 0x00,0x74,0x12,0x00 +.byte 0x00,0x64,0x13,0x00 +.byte 0x00,0x01,0x11,0x00 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha256_asm_block_data_order_epilogue: +.byte 1,0,5,11 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0xb3 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha256_asm_block_data_order_shaext_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha256_asm_block_data_order_shaext_body: +.byte 1,0,17,85 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0x74,0x0c,0x00 +.byte 0x00,0x64,0x0d,0x00 +.byte 0x00,0x53 +.byte 0x00,0x92 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha256_asm_block_data_order_shaext_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_body: +.byte 1,0,25,133 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xf4,0x0b,0x00 +.byte 0x00,0xe4,0x0c,0x00 +.byte 0x00,0xd4,0x0d,0x00 +.byte 0x00,0xc4,0x0e,0x00 +.byte 0x00,0x34,0x0f,0x00 +.byte 0x00,0x74,0x12,0x00 +.byte 0x00,0x64,0x13,0x00 +.byte 0x00,0x53 +.byte 0x00,0xf2 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha256_asm_block_data_order_ssse3_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha256_asm_block_data_order_avx_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha256_asm_block_data_order_avx_body: +.byte 1,0,26,165 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xf4,0x0f,0x00 +.byte 0x00,0xe4,0x10,0x00 +.byte 0x00,0xd4,0x11,0x00 +.byte 0x00,0xc4,0x12,0x00 +.byte 0x00,0x34,0x13,0x00 +.byte 0x00,0x74,0x16,0x00 +.byte 0x00,0x64,0x17,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x14,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha256_asm_block_data_order_avx_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha256_asm_block_data_order_avx2_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha256_asm_block_data_order_avx2_body: +.byte 1,0,26,165 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xf4,0x0f,0x00 +.byte 0x00,0xe4,0x10,0x00 +.byte 0x00,0xd4,0x11,0x00 +.byte 0x00,0xc4,0x12,0x00 +.byte 0x00,0x34,0x13,0x00 +.byte 0x00,0x74,0x16,0x00 +.byte 0x00,0x64,0x17,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x14,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha256_asm_block_data_order_avx2_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + diff --git a/cbits/asm/sha512-armv8.pl b/cbits/asm/sha512-armv8.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/sha512-armv8.pl @@ -0,0 +1,892 @@ +#!/usr/bin/env perl +# SPDX-License-Identifier: GPL-1.0+ OR BSD-3-Clause +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# SHA256/512 for ARMv8. +# +# Performance in cycles per processed byte and improvement coefficient +# over code generated with "default" compiler: +# +# SHA256-hw SHA256(*) SHA512 +# Apple A7 1.97 10.5 (+33%) 6.73 (-1%(**)) +# Apple A10 1.30 5.81 +# Apple A12 1.31 5.06 +# Apple A14/M1 1.30 8.19 (+14%) 2.24 (hw) +# Cortex-A53 2.38 15.5 (+115%) 10.0 (+150%(***)) +# Cortex-A57 2.31 11.6 (+86%) 7.51 (+260%(***)) +# Cortex-A76 1.60 9.5 6.05 +# Cortex-X2 1.60 7.3 2.60 (hw) +# Cortex-X925 1.57 5.97 2.55 (hw) +# Denver 2.01 10.5 (+26%) 6.70 (+8%) +# X-Gene 20.0 (+100%) 12.8 (+300%(***)) +# Mongoose 2.36 13.0 (+50%) 8.36 (+33%) +# Kryo 1.92 17.4 (+30%) 11.2 (+8%) +# ThunderX2 2.54 13.2 (+40%) 8.40 (+18%) +# Shapdragon X 1.40 7.43 2.23 (hw) +# +# (*) Software SHA256 results are of lesser relevance, presented +# mostly for informational purposes. +# (**) The result is a trade-off: it's possible to improve it by +# 10% (or by 1 cycle per round), but at the cost of 20% loss +# on Cortex-A53 (or by 4 cycles per round). +# (***) Super-impressive coefficients over gcc-generated code are +# indication of some compiler "pathology", most notably code +# generated with -mgeneral-regs-only is significantly faster +# and the gap is only 40-90%. +# +# October 2016. +# +# Originally it was reckoned that it makes no sense to implement NEON +# version of SHA256 for 64-bit processors. This is because performance +# improvement on most wide-spread Cortex-A5x processors was observed +# to be marginal, same on Cortex-A53 and ~10% on A57. But then it was +# observed that 32-bit NEON SHA256 performs significantly better than +# 64-bit scalar version on *some* of the more recent processors. As +# result 64-bit NEON version of SHA256 was added to provide best +# all-round performance. For example it executes ~30% faster on X-Gene +# and Mongoose. [For reference, NEON version of SHA512 is bound to +# deliver much less improvement, likely *negative* on Cortex-A5x. +# Which is why NEON support is limited to SHA256.] + +$flavour = shift; +$output = shift; + +if ($flavour && $flavour ne "void") { + $0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; + ( $xlate="${dir}arm-xlate.pl" and -f $xlate ) or + ( $xlate="${dir}../../perlasm/arm-xlate.pl" and -f $xlate) or + die "can't locate arm-xlate.pl"; + + open STDOUT,"| \"$^X\" $xlate $flavour $output"; +} else { + open STDOUT,">$output"; +} + +if ($output =~ /512/) { + $BITS=512; + $SZ=8; + @Sigma0=(28,34,39); + @Sigma1=(14,18,41); + @sigma0=(1, 8, 7); + @sigma1=(19,61, 6); + $rounds=80; + $reg_t="x"; +} else { + $BITS=256; + $SZ=4; + @Sigma0=( 2,13,22); + @Sigma1=( 6,11,25); + @sigma0=( 7,18, 3); + @sigma1=(17,19,10); + $rounds=64; + $reg_t="w"; +} + +$func="sha${BITS}_block_data_order"; + +($ctx,$inp,$num,$Ktbl)=map("x$_",(0..2,30)); + +@X=map("$reg_t$_",(3..15,0..2)); +@V=($A,$B,$C,$D,$E,$F,$G,$H)=map("$reg_t$_",(20..27)); +($t0,$t1,$t2,$t3)=map("$reg_t$_",(16,17,19,28)); + +sub BODY_00_xx { +my ($i,$a,$b,$c,$d,$e,$f,$g,$h)=@_; +my $j=($i+1)&15; +my ($T0,$T1,$T2)=(@X[($i-8)&15],@X[($i-9)&15],@X[($i-10)&15]); + $T0=@X[$i+3] if ($i<11); + +$code.=<<___ if ($i<16); +#ifndef __AARCH64EB__ + rev @X[$i],@X[$i] // $i +#endif +___ +$code.=<<___ if ($i<13 && ($i&1)); + ldp @X[$i+1],@X[$i+2],[$inp],#2*$SZ +___ +$code.=<<___ if ($i==13); + ldp @X[14],@X[15],[$inp] +___ +$code.=<<___ if ($i>=14); + ldr @X[($i-11)&15],[sp,#`$SZ*(($i-11)%4)`] +___ +$code.=<<___ if ($i>0 && $i<16); + add $a,$a,$t1 // h+=Sigma0(a) +___ +$code.=<<___ if ($i>=11); + str @X[($i-8)&15],[sp,#`$SZ*(($i-8)%4)`] +___ +# While ARMv8 specifies merged rotate-n-logical operation such as +# 'eor x,y,z,ror#n', it was found to negatively affect performance +# on Apple A7. The reason seems to be that it requires even 'y' to +# be available earlier. This means that such merged instruction is +# not necessarily best choice on critical path... On the other hand +# Cortex-A5x handles merged instructions much better than disjoint +# rotate and logical... See (**) footnote above. +$code.=<<___ if ($i<15); + ror $t0,$e,#$Sigma1[0] + add $h,$h,$t2 // h+=K[i] + eor $T0,$e,$e,ror#`$Sigma1[2]-$Sigma1[1]` + and $t1,$f,$e + bic $t2,$g,$e + add $h,$h,@X[$i&15] // h+=X[i] + orr $t1,$t1,$t2 // Ch(e,f,g) + eor $t2,$a,$b // a^b, b^c in next round + eor $t0,$t0,$T0,ror#$Sigma1[1] // Sigma1(e) + ror $T0,$a,#$Sigma0[0] + add $h,$h,$t1 // h+=Ch(e,f,g) + eor $t1,$a,$a,ror#`$Sigma0[2]-$Sigma0[1]` + add $h,$h,$t0 // h+=Sigma1(e) + and $t3,$t3,$t2 // (b^c)&=(a^b) + add $d,$d,$h // d+=h + eor $t3,$t3,$b // Maj(a,b,c) + eor $t1,$T0,$t1,ror#$Sigma0[1] // Sigma0(a) + add $h,$h,$t3 // h+=Maj(a,b,c) + ldr $t3,[$Ktbl],#$SZ // *K++, $t2 in next round + //add $h,$h,$t1 // h+=Sigma0(a) +___ +$code.=<<___ if ($i>=15); + ror $t0,$e,#$Sigma1[0] + add $h,$h,$t2 // h+=K[i] + ror $T1,@X[($j+1)&15],#$sigma0[0] + and $t1,$f,$e + ror $T2,@X[($j+14)&15],#$sigma1[0] + bic $t2,$g,$e + ror $T0,$a,#$Sigma0[0] + add $h,$h,@X[$i&15] // h+=X[i] + eor $t0,$t0,$e,ror#$Sigma1[1] + eor $T1,$T1,@X[($j+1)&15],ror#$sigma0[1] + orr $t1,$t1,$t2 // Ch(e,f,g) + eor $t2,$a,$b // a^b, b^c in next round + eor $t0,$t0,$e,ror#$Sigma1[2] // Sigma1(e) + eor $T0,$T0,$a,ror#$Sigma0[1] + add $h,$h,$t1 // h+=Ch(e,f,g) + and $t3,$t3,$t2 // (b^c)&=(a^b) + eor $T2,$T2,@X[($j+14)&15],ror#$sigma1[1] + eor $T1,$T1,@X[($j+1)&15],lsr#$sigma0[2] // sigma0(X[i+1]) + add $h,$h,$t0 // h+=Sigma1(e) + eor $t3,$t3,$b // Maj(a,b,c) + eor $t1,$T0,$a,ror#$Sigma0[2] // Sigma0(a) + eor $T2,$T2,@X[($j+14)&15],lsr#$sigma1[2] // sigma1(X[i+14]) + add @X[$j],@X[$j],@X[($j+9)&15] + add $d,$d,$h // d+=h + add $h,$h,$t3 // h+=Maj(a,b,c) + ldr $t3,[$Ktbl],#$SZ // *K++, $t2 in next round + add @X[$j],@X[$j],$T1 + add $h,$h,$t1 // h+=Sigma0(a) + add @X[$j],@X[$j],$T2 +___ + ($t2,$t3)=($t3,$t2); +} + +$code.=<<___; +#ifndef __KERNEL__ +# include "arm_arch.h" +.extern OPENSSL_armcap_P +#endif + +.text + +.globl $func +.type $func,%function +.align 6 +$func: +#ifndef __KERNEL__ + adrp c16,OPENSSL_armcap_P + ldr w16,[c16,#:lo12:OPENSSL_armcap_P] +___ +$code.=<<___ if ($SZ==4); + tst w16,#ARMV8_SHA256 + b.ne .Lv8_entry + tst w16,#ARMV7_NEON + b.ne .Lneon_entry +___ +$code.=<<___ if ($SZ==8); + tst w16,#ARMV8_SHA512 + b.ne .Lv8_entry +___ +$code.=<<___; +#endif + .inst 0xd503233f // paciasp + stp c29,c30,[csp,#-16*__SIZEOF_POINTER__]! + add c29,csp,#0 + + stp c19,c20,[csp,#2*__SIZEOF_POINTER__] + stp c21,c22,[csp,#4*__SIZEOF_POINTER__] + stp c23,c24,[csp,#6*__SIZEOF_POINTER__] + stp c25,c26,[csp,#8*__SIZEOF_POINTER__] + stp c27,c28,[csp,#10*__SIZEOF_POINTER__] + sub csp,csp,#4*$SZ + + ldp $A,$B,[$ctx] // load context + ldp $C,$D,[$ctx,#2*$SZ] + lsl $num,$num,#`log(16*$SZ)/log(2)` + ldp $E,$F,[$ctx,#4*$SZ] + cadd $num,$inp,$num // end of input + ldp $G,$H,[$ctx,#6*$SZ] + adr $Ktbl,.LK$BITS + stp c#$ctx,c#$num,[c29,#12*__SIZEOF_POINTER__] + +.Loop: + ldp @X[0],@X[1],[$inp],#2*$SZ + ldr $t2,[$Ktbl],#$SZ // *K++ + eor $t3,$B,$C // magic seed + str c#$inp,[c29,#14*__SIZEOF_POINTER__] +___ +for ($i=0;$i<16;$i++) { &BODY_00_xx($i,@V); unshift(@V,pop(@V)); } +$code.=".Loop_16_xx:\n"; +for (;$i<32;$i++) { &BODY_00_xx($i,@V); unshift(@V,pop(@V)); } +$code.=<<___; + cbnz $t2,.Loop_16_xx + + ldp c#$ctx,c#$num,[c29,#12*__SIZEOF_POINTER__] + ldr c#$inp,[c29,#14*__SIZEOF_POINTER__] + csub $Ktbl,$Ktbl,#`$SZ*($rounds+1)` // rewind + + ldp @X[0],@X[1],[$ctx] + ldp @X[2],@X[3],[$ctx,#2*$SZ] + cadd $inp,$inp,#14*$SZ // advance input pointer + ldp @X[4],@X[5],[$ctx,#4*$SZ] + add $A,$A,@X[0] + ldp @X[6],@X[7],[$ctx,#6*$SZ] + add $B,$B,@X[1] + add $C,$C,@X[2] + add $D,$D,@X[3] + stp $A,$B,[$ctx] + add $E,$E,@X[4] + add $F,$F,@X[5] + stp $C,$D,[$ctx,#2*$SZ] + add $G,$G,@X[6] + add $H,$H,@X[7] + cmp $inp,$num + stp $E,$F,[$ctx,#4*$SZ] + stp $G,$H,[$ctx,#6*$SZ] + b.ne .Loop + + ldp c19,c20,[c29,#2*__SIZEOF_POINTER__] + add csp,csp,#4*$SZ + ldp c21,c22,[c29,#4*__SIZEOF_POINTER__] + ldp c23,c24,[c29,#6*__SIZEOF_POINTER__] + ldp c25,c26,[c29,#8*__SIZEOF_POINTER__] + ldp c27,c28,[c29,#10*__SIZEOF_POINTER__] + ldp c29,c30,[csp],#16*__SIZEOF_POINTER__ + .inst 0xd50323bf // autiasp + ret +.size $func,.-$func + +.align 6 +.type .LK$BITS,%object +.LK$BITS: +___ +$code.=<<___ if ($SZ==8); + .quad 0x428a2f98d728ae22,0x7137449123ef65cd + .quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc + .quad 0x3956c25bf348b538,0x59f111f1b605d019 + .quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 + .quad 0xd807aa98a3030242,0x12835b0145706fbe + .quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 + .quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 + .quad 0x9bdc06a725c71235,0xc19bf174cf692694 + .quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 + .quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 + .quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 + .quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 + .quad 0x983e5152ee66dfab,0xa831c66d2db43210 + .quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 + .quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 + .quad 0x06ca6351e003826f,0x142929670a0e6e70 + .quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 + .quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df + .quad 0x650a73548baf63de,0x766a0abb3c77b2a8 + .quad 0x81c2c92e47edaee6,0x92722c851482353b + .quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 + .quad 0xc24b8b70d0f89791,0xc76c51a30654be30 + .quad 0xd192e819d6ef5218,0xd69906245565a910 + .quad 0xf40e35855771202a,0x106aa07032bbd1b8 + .quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 + .quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 + .quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb + .quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 + .quad 0x748f82ee5defb2fc,0x78a5636f43172f60 + .quad 0x84c87814a1f0ab72,0x8cc702081a6439ec + .quad 0x90befffa23631e28,0xa4506cebde82bde9 + .quad 0xbef9a3f7b2c67915,0xc67178f2e372532b + .quad 0xca273eceea26619c,0xd186b8c721c0c207 + .quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 + .quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 + .quad 0x113f9804bef90dae,0x1b710b35131c471b + .quad 0x28db77f523047d84,0x32caab7b40c72493 + .quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c + .quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a + .quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + .quad 0 // terminator +___ +$code.=<<___ if ($SZ==4); + .long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 + .long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 + .long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 + .long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 + .long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc + .long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da + .long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 + .long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 + .long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 + .long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 + .long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 + .long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 + .long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 + .long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 + .long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 + .long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 + .long 0 //terminator +___ +$code.=<<___; +.size .LK$BITS,.-.LK$BITS +.asciz "SHA$BITS block transform for ARMv8, CRYPTOGAMS by \@dot-asm" +.align 2 +___ + +if ($SZ==4) { +my $Ktbl="x3"; + +my ($ABCD,$EFGH,$abcd)=map("v$_.16b",(0..2)); +my @MSG=map("v$_.16b",(4..7)); +my ($W0,$W1)=("v16.4s","v17.4s"); +my ($ABCD_SAVE,$EFGH_SAVE)=("v18.16b","v19.16b"); + +$code.=<<___; +#ifndef __KERNEL__ +.type sha256_block_armv8,%function +.align 6 +sha256_block_armv8: +.Lv8_entry: + stp c29,c30,[csp,#-2*__SIZEOF_POINTER__]! + add c29,csp,#0 + + ld1.32 {$ABCD,$EFGH},[$ctx] + adr $Ktbl,.LK256 + +.Loop_hw: + ld1 {@MSG[0]-@MSG[3]},[$inp],#64 + sub $num,$num,#1 + ld1.32 {$W0},[$Ktbl],#16 + rev32 @MSG[0],@MSG[0] + rev32 @MSG[1],@MSG[1] + rev32 @MSG[2],@MSG[2] + rev32 @MSG[3],@MSG[3] + orr $ABCD_SAVE,$ABCD,$ABCD // offload + orr $EFGH_SAVE,$EFGH,$EFGH +___ +for($i=0;$i<12;$i++) { +$code.=<<___; + ld1.32 {$W1},[$Ktbl],#16 + add.i32 $W0,$W0,@MSG[0] + sha256su0 @MSG[0],@MSG[1] + orr $abcd,$ABCD,$ABCD + sha256h $ABCD,$EFGH,$W0 + sha256h2 $EFGH,$abcd,$W0 + sha256su1 @MSG[0],@MSG[2],@MSG[3] +___ + ($W0,$W1)=($W1,$W0); push(@MSG,shift(@MSG)); +} +$code.=<<___; + ld1.32 {$W1},[$Ktbl],#16 + add.i32 $W0,$W0,@MSG[0] + orr $abcd,$ABCD,$ABCD + sha256h $ABCD,$EFGH,$W0 + sha256h2 $EFGH,$abcd,$W0 + + ld1.32 {$W0},[$Ktbl],#16 + add.i32 $W1,$W1,@MSG[1] + orr $abcd,$ABCD,$ABCD + sha256h $ABCD,$EFGH,$W1 + sha256h2 $EFGH,$abcd,$W1 + + ld1.32 {$W1},[$Ktbl] + add.i32 $W0,$W0,@MSG[2] + csub $Ktbl,$Ktbl,#$rounds*$SZ-16 // rewind + orr $abcd,$ABCD,$ABCD + sha256h $ABCD,$EFGH,$W0 + sha256h2 $EFGH,$abcd,$W0 + + add.i32 $W1,$W1,@MSG[3] + orr $abcd,$ABCD,$ABCD + sha256h $ABCD,$EFGH,$W1 + sha256h2 $EFGH,$abcd,$W1 + + add.i32 $ABCD,$ABCD,$ABCD_SAVE + add.i32 $EFGH,$EFGH,$EFGH_SAVE + + cbnz $num,.Loop_hw + + st1.32 {$ABCD,$EFGH},[$ctx] + + ldr c29,[csp],#2*__SIZEOF_POINTER__ + ret +.size sha256_block_armv8,.-sha256_block_armv8 +#endif +___ +} + +if ($SZ==4) { ######################################### NEON stuff # +# You'll surely note a lot of similarities with sha256-armv4 module, +# and of course it's not a coincidence. sha256-armv4 was used as +# initial template, but was adapted for ARMv8 instruction set and +# extensively re-tuned for all-round performance. + +my @V = ($A,$B,$C,$D,$E,$F,$G,$H) = map("w$_",(3..10)); +my ($t0,$t1,$t2,$t3,$t4) = map("w$_",(11..15)); +my $Ktbl="x16"; +my $Xfer="x17"; +my @X = map("q$_",(0..3)); +my ($T0,$T1,$T2,$T3,$T4,$T5,$T6,$T7) = map("q$_",(4..7,16..19)); +my $j=0; + +sub AUTOLOAD() # thunk [simplified] x86-style perlasm +{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://; $opcode =~ s/_/\./; + my $arg = pop; + $arg = "#$arg" if ($arg*1 eq $arg); + $code .= "\t$opcode\t".join(',',@_,$arg)."\n"; +} + +sub Dscalar { shift =~ m|[qv]([0-9]+)|?"d$1":""; } +sub Dlo { shift =~ m|[qv]([0-9]+)|?"v$1.d[0]":""; } +sub Dhi { shift =~ m|[qv]([0-9]+)|?"v$1.d[1]":""; } + +sub Xupdate() +{ use integer; + my $body = shift; + my @insns = (&$body,&$body,&$body,&$body); + my ($a,$b,$c,$d,$e,$f,$g,$h); + + &ext_8 ($T0,@X[0],@X[1],4); # X[1..4] + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &ext_8 ($T3,@X[2],@X[3],4); # X[9..12] + eval(shift(@insns)); + eval(shift(@insns)); + &mov (&Dscalar($T7),&Dhi(@X[3])); # X[14..15] + eval(shift(@insns)); + eval(shift(@insns)); + &ushr_32 ($T2,$T0,$sigma0[0]); + eval(shift(@insns)); + &ushr_32 ($T1,$T0,$sigma0[2]); + eval(shift(@insns)); + &add_32 (@X[0],@X[0],$T3); # X[0..3] += X[9..12] + eval(shift(@insns)); + &sli_32 ($T2,$T0,32-$sigma0[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &ushr_32 ($T3,$T0,$sigma0[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &eor_8 ($T1,$T1,$T2); + eval(shift(@insns)); + eval(shift(@insns)); + &sli_32 ($T3,$T0,32-$sigma0[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &ushr_32 ($T4,$T7,$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &eor_8 ($T1,$T1,$T3); # sigma0(X[1..4]) + eval(shift(@insns)); + eval(shift(@insns)); + &sli_32 ($T4,$T7,32-$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &ushr_32 ($T5,$T7,$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &ushr_32 ($T3,$T7,$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &add_32 (@X[0],@X[0],$T1); # X[0..3] += sigma0(X[1..4]) + eval(shift(@insns)); + eval(shift(@insns)); + &sli_u32 ($T3,$T7,32-$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &eor_8 ($T5,$T5,$T4); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &eor_8 ($T5,$T5,$T3); # sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &add_32 (@X[0],@X[0],$T5); # X[0..1] += sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &ushr_32 ($T6,@X[0],$sigma1[0]); + eval(shift(@insns)); + &ushr_32 ($T7,@X[0],$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &sli_32 ($T6,@X[0],32-$sigma1[0]); + eval(shift(@insns)); + &ushr_32 ($T5,@X[0],$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &eor_8 ($T7,$T7,$T6); + eval(shift(@insns)); + eval(shift(@insns)); + &sli_32 ($T5,@X[0],32-$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &ld1_32 ("{$T0}","[$Ktbl], #16"); + eval(shift(@insns)); + &eor_8 ($T7,$T7,$T5); # sigma1(X[16..17]) + eval(shift(@insns)); + eval(shift(@insns)); + &eor_8 ($T5,$T5,$T5); + eval(shift(@insns)); + eval(shift(@insns)); + &mov (&Dhi($T5), &Dlo($T7)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &add_32 (@X[0],@X[0],$T5); # X[2..3] += sigma1(X[16..17]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &add_32 ($T0,$T0,@X[0]); + while($#insns>=1) { eval(shift(@insns)); } + &st1_32 ("{$T0}","[$Xfer], #16"); + eval(shift(@insns)); + + push(@X,shift(@X)); # "rotate" X[] +} + +sub Xpreload() +{ use integer; + my $body = shift; + my @insns = (&$body,&$body,&$body,&$body); + my ($a,$b,$c,$d,$e,$f,$g,$h); + + eval(shift(@insns)); + eval(shift(@insns)); + &ld1_8 ("{@X[0]}","[$inp],#16"); + eval(shift(@insns)); + eval(shift(@insns)); + &ld1_32 ("{$T0}","[$Ktbl],#16"); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &rev32 (@X[0],@X[0]); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &add_32 ($T0,$T0,@X[0]); + foreach (@insns) { eval; } # remaining instructions + &st1_32 ("{$T0}","[$Xfer], #16"); + + push(@X,shift(@X)); # "rotate" X[] +} + +sub body_00_15 () { + ( + '($a,$b,$c,$d,$e,$f,$g,$h)=@V;'. + '&add ($h,$h,$t1)', # h+=X[i]+K[i] + '&add ($a,$a,$t4);'. # h+=Sigma0(a) from the past + '&and ($t1,$f,$e)', + '&bic ($t4,$g,$e)', + '&eor ($t0,$e,$e,"ror#".($Sigma1[1]-$Sigma1[0]))', + '&add ($a,$a,$t2)', # h+=Maj(a,b,c) from the past + '&orr ($t1,$t1,$t4)', # Ch(e,f,g) + '&eor ($t0,$t0,$e,"ror#".($Sigma1[2]-$Sigma1[0]))', # Sigma1(e) + '&eor ($t4,$a,$a,"ror#".($Sigma0[1]-$Sigma0[0]))', + '&add ($h,$h,$t1)', # h+=Ch(e,f,g) + '&ror ($t0,$t0,"#$Sigma1[0]")', + '&eor ($t2,$a,$b)', # a^b, b^c in next round + '&eor ($t4,$t4,$a,"ror#".($Sigma0[2]-$Sigma0[0]))', # Sigma0(a) + '&add ($h,$h,$t0)', # h+=Sigma1(e) + '&ldr ($t1,sprintf "[sp,#%d]",4*(($j+1)&15)) if (($j&15)!=15);'. + '&ldr ($t1,"[$Ktbl]") if ($j==15);'. + '&and ($t3,$t3,$t2)', # (b^c)&=(a^b) + '&ror ($t4,$t4,"#$Sigma0[0]")', + '&add ($d,$d,$h)', # d+=h + '&eor ($t3,$t3,$b)', # Maj(a,b,c) + '$j++; unshift(@V,pop(@V)); ($t2,$t3)=($t3,$t2);' + ) +} + +$code.=<<___; +#ifdef __KERNEL__ +.globl sha256_block_neon +#endif +.type sha256_block_neon,%function +.align 4 +sha256_block_neon: +.Lneon_entry: + stp c29, c30, [csp, #-2*__SIZEOF_POINTER__]! + mov c29, csp + sub csp,csp,#16*4 + + adr $Ktbl,.LK256 + add $num,$inp,$num,lsl#6 // len to point at the end of inp + + ld1.8 {@X[0]},[$inp], #16 + ld1.8 {@X[1]},[$inp], #16 + ld1.8 {@X[2]},[$inp], #16 + ld1.8 {@X[3]},[$inp], #16 + ld1.32 {$T0},[$Ktbl], #16 + ld1.32 {$T1},[$Ktbl], #16 + ld1.32 {$T2},[$Ktbl], #16 + ld1.32 {$T3},[$Ktbl], #16 + rev32 @X[0],@X[0] // yes, even on + rev32 @X[1],@X[1] // big-endian + rev32 @X[2],@X[2] + rev32 @X[3],@X[3] + cmov $Xfer,sp + add.32 $T0,$T0,@X[0] + add.32 $T1,$T1,@X[1] + add.32 $T2,$T2,@X[2] + st1.32 {$T0-$T1},[$Xfer], #32 + add.32 $T3,$T3,@X[3] + st1.32 {$T2-$T3},[$Xfer] + csub $Xfer,$Xfer,#32 + + ldp $A,$B,[$ctx] + ldp $C,$D,[$ctx,#8] + ldp $E,$F,[$ctx,#16] + ldp $G,$H,[$ctx,#24] + ldr $t1,[sp,#0] + mov $t2,wzr + eor $t3,$B,$C + mov $t4,wzr + b .L_00_48 + +.align 4 +.L_00_48: +___ + &Xupdate(\&body_00_15); + &Xupdate(\&body_00_15); + &Xupdate(\&body_00_15); + &Xupdate(\&body_00_15); +$code.=<<___; + cmp $t1,#0 // check for K256 terminator + ldr $t1,[sp,#0] + csub $Xfer,$Xfer,#64 + bne .L_00_48 + + csub $Ktbl,$Ktbl,#256 // rewind $Ktbl + cmp $inp,$num + mov $Xfer, #-64 + csel $Xfer, $Xfer, xzr, eq + cadd $inp,$inp,$Xfer // avoid SEGV + cmov $Xfer,sp +___ + &Xpreload(\&body_00_15); + &Xpreload(\&body_00_15); + &Xpreload(\&body_00_15); + &Xpreload(\&body_00_15); +$code.=<<___; + add $A,$A,$t4 // h+=Sigma0(a) from the past + ldp $t0,$t1,[$ctx,#0] + add $A,$A,$t2 // h+=Maj(a,b,c) from the past + ldp $t2,$t3,[$ctx,#8] + add $A,$A,$t0 // accumulate + add $B,$B,$t1 + ldp $t0,$t1,[$ctx,#16] + add $C,$C,$t2 + add $D,$D,$t3 + ldp $t2,$t3,[$ctx,#24] + add $E,$E,$t0 + add $F,$F,$t1 + ldr $t1,[sp,#0] + stp $A,$B,[$ctx,#0] + add $G,$G,$t2 + mov $t2,wzr + stp $C,$D,[$ctx,#8] + add $H,$H,$t3 + stp $E,$F,[$ctx,#16] + eor $t3,$B,$C + stp $G,$H,[$ctx,#24] + mov $t4,wzr + cmov $Xfer,sp + b.ne .L_00_48 + + ldr c29,[c29] + add csp,csp,#16*4+2*__SIZEOF_POINTER__ + ret +.size sha256_block_neon,.-sha256_block_neon +___ +} + +if ($SZ==8) { +my $Ktbl="x3"; + +my @H = map("v$_.16b",(0..4)); +my ($fg,$de,$m9_10)=map("v$_.16b",(5..7)); +my @MSG=map("v$_.16b",(16..23)); +my ($W0,$W1)=("v24.2d","v25.2d"); +my ($AB,$CD,$EF,$GH)=map("v$_.16b",(26..29)); + +$code.=<<___; +#ifndef __KERNEL__ +.type sha512_block_armv8,%function +.align 6 +sha512_block_armv8: +.Lv8_entry: + stp c29,c30,[csp,#-2*__SIZEOF_POINTER__]! + add c29,csp,#0 + + ld1 {@MSG[0]-@MSG[3]},[$inp],#64 // load input + ld1 {@MSG[4]-@MSG[7]},[$inp],#64 + + ld1.64 {@H[0]-@H[3]},[$ctx] // load context + adr $Ktbl,.LK512 + + rev64 @MSG[0],@MSG[0] + rev64 @MSG[1],@MSG[1] + rev64 @MSG[2],@MSG[2] + rev64 @MSG[3],@MSG[3] + rev64 @MSG[4],@MSG[4] + rev64 @MSG[5],@MSG[5] + rev64 @MSG[6],@MSG[6] + rev64 @MSG[7],@MSG[7] + b .Loop_hw + +.align 4 +.Loop_hw: + ld1.64 {$W0},[$Ktbl],#16 + subs $num,$num,#1 + sub c4,c#$inp,#128 + orr $AB,@H[0],@H[0] // offload + orr $CD,@H[1],@H[1] + orr $EF,@H[2],@H[2] + orr $GH,@H[3],@H[3] + csel c#$inp,c#$inp,c4,ne // conditional rewind +___ +for($i=0;$i<32;$i++) { +$code.=<<___; + add.i64 $W0,$W0,@MSG[0] + ld1.64 {$W1},[$Ktbl],#16 + ext $W0,$W0,$W0,#8 + ext $fg,@H[2],@H[3],#8 + ext $de,@H[1],@H[2],#8 + add.i64 @H[3],@H[3],$W0 // "T1 + H + K512[i]" + sha512su0 @MSG[0],@MSG[1] + ext $m9_10,@MSG[4],@MSG[5],#8 + sha512h @H[3],$fg,$de + sha512su1 @MSG[0],@MSG[7],$m9_10 + add.i64 @H[4],@H[1],@H[3] // "D + T1" + sha512h2 @H[3],$H[1],@H[0] +___ + ($W0,$W1)=($W1,$W0); push(@MSG,shift(@MSG)); + @H = (@H[3],@H[0],@H[4],@H[2],@H[1]); +} +for(;$i<40;$i++) { +$code.=<<___ if ($i<39); + ld1.64 {$W1},[$Ktbl],#16 +___ +$code.=<<___ if ($i==39); + csub $Ktbl,$Ktbl,#$rounds*$SZ // rewind +___ +$code.=<<___; + add.i64 $W0,$W0,@MSG[0] + ld1 {@MSG[0]},[$inp],#16 // load next input + ext $W0,$W0,$W0,#8 + ext $fg,@H[2],@H[3],#8 + ext $de,@H[1],@H[2],#8 + add.i64 @H[3],@H[3],$W0 // "T1 + H + K512[i]" + sha512h @H[3],$fg,$de + rev64 @MSG[0],@MSG[0] + add.i64 @H[4],@H[1],@H[3] // "D + T1" + sha512h2 @H[3],$H[1],@H[0] +___ + ($W0,$W1)=($W1,$W0); push(@MSG,shift(@MSG)); + @H = (@H[3],@H[0],@H[4],@H[2],@H[1]); +} +$code.=<<___; + add.i64 @H[0],@H[0],$AB // accumulate + add.i64 @H[1],@H[1],$CD + add.i64 @H[2],@H[2],$EF + add.i64 @H[3],@H[3],$GH + + cbnz $num,.Loop_hw + + st1.64 {@H[0]-@H[3]},[$ctx] // store context + + ldr c29,[csp],#2*__SIZEOF_POINTER__ + ret +.size sha512_block_armv8,.-sha512_block_armv8 +#endif +___ +} + +$code.=<<___; +#if !defined(__KERNEL__) && !defined(_WIN64) +.comm OPENSSL_armcap_P,4,4 +.hidden OPENSSL_armcap_P +#endif +___ + +{ my %opcode = ( + "sha256h" => 0x5e004000, "sha256h2" => 0x5e005000, + "sha256su0" => 0x5e282800, "sha256su1" => 0x5e006000 ); + + sub unsha256 { + my ($mnemonic,$arg)=@_; + + $arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+))?/o + && + sprintf ".inst\t0x%08x\t//%s %s", + $opcode{$mnemonic}|$1|($2<<5)|($3<<16), + $mnemonic,$arg; + } +} + +{ my %opcode = ( + "sha512h" => 0xce608000, "sha512h2" => 0xce608400, + "sha512su0" => 0xcec08000, "sha512su1" => 0xce608800 ); + + sub unsha512 { + my ($mnemonic,$arg)=@_; + + $arg =~ m/[qv]([0-9]+)[^,]*,\s*[qv]([0-9]+)[^,]*(?:,\s*[qv]([0-9]+))?/o + && + sprintf ".inst\t0x%08x\t//%s %s", + $opcode{$mnemonic}|$1|($2<<5)|($3<<16), + $mnemonic,$arg; + } +} + +open SELF,$0; +while() { + next if (/^#!/); + last if (!s/^#/\/\// and !/^$/); + print; +} +close SELF; + +foreach(split("\n",$code)) { + + s/\`([^\`]*)\`/eval($1)/ge; + + s/\b(sha512\w+)\s+([qv].*)/unsha512($1,$2)/ge or + s/\b(sha256\w+)\s+([qv].*)/unsha256($1,$2)/ge; + + s/\bq([0-9]+)\b/v$1.16b/g; # old->new registers + + s/\.[ui]?8(\s)/$1/; + s/\.\w?64\b// and s/\.16b/\.2d/g or + s/\.\w?32\b// and s/\.16b/\.4s/g; + m/\bext\b/ and s/\.2d/\.16b/g or + m/(ld|st)1[^\[]+\[0\]/ and s/\.4s/\.s/g; + + s/([cw])#x([0-9]+)/$1$2/g; + + print $_,"\n"; +} + +close STDOUT; diff --git a/cbits/asm/sha512-x86_64-elf.S b/cbits/asm/sha512-x86_64-elf.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha512-x86_64-elf.S @@ -0,0 +1,5727 @@ +.text + + +.globl crypton_sha512_asm_block_data_order +.type crypton_sha512_asm_block_data_order,@function +.align 16 +crypton_sha512_asm_block_data_order: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + leaq crypton_ia32cap_P(%rip),%rax + movl 0(%rax),%r9d + movl 4(%rax),%r10d + movl 8(%rax),%eax + testl $2048,%r10d + jnz .Lxop_shortcut + andl $296,%eax + cmpl $296,%eax + je .Lavx2_shortcut + andl $1073741824,%r9d + andl $268435968,%r10d + orl %r9d,%r10d + cmpl $1342177792,%r10d + je .Lavx_shortcut + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $128+24,%rsp + +.cfi_def_cfa %rsp,208 + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,128+0(%rsp) + movq %rsi,128+8(%rsp) + movq %rdx,128+16(%rsp) + + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Lloop + +.align 16 +.Lloop: + movq %rbx,%rdi + leaq K512(%rip),%rbp + xorq %rcx,%rdi + movq 0(%rsi),%r12 + movq %r8,%r13 + movq %rax,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,0(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + addq %r14,%r11 + movq 8(%rsi),%r12 + movq %rdx,%r13 + movq %r11,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,8(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + addq %r14,%r10 + movq 16(%rsi),%r12 + movq %rcx,%r13 + movq %r10,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,16(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + addq %r14,%r9 + movq 24(%rsi),%r12 + movq %rbx,%r13 + movq %r9,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,24(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + addq %r14,%r8 + movq 32(%rsi),%r12 + movq %rax,%r13 + movq %r8,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,32(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + addq %r14,%rdx + movq 40(%rsi),%r12 + movq %r11,%r13 + movq %rdx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,40(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + addq %r14,%rcx + movq 48(%rsi),%r12 + movq %r10,%r13 + movq %rcx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,48(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + addq %r14,%rbx + movq 56(%rsi),%r12 + movq %r9,%r13 + movq %rbx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,56(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + addq %r14,%rax + movq 64(%rsi),%r12 + movq %r8,%r13 + movq %rax,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,64(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + addq %r14,%r11 + movq 72(%rsi),%r12 + movq %rdx,%r13 + movq %r11,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,72(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + addq %r14,%r10 + movq 80(%rsi),%r12 + movq %rcx,%r13 + movq %r10,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,80(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + addq %r14,%r9 + movq 88(%rsi),%r12 + movq %rbx,%r13 + movq %r9,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,88(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + addq %r14,%r8 + movq 96(%rsi),%r12 + movq %rax,%r13 + movq %r8,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,96(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + addq %r14,%rdx + movq 104(%rsi),%r12 + movq %r11,%r13 + movq %rdx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,104(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + addq %r14,%rcx + movq 112(%rsi),%r12 + movq %r10,%r13 + movq %rcx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,112(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + addq %r14,%rbx + movq 120(%rsi),%r12 + movq %r9,%r13 + movq %rbx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,120(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + jmp .Lrounds_16_xx +.align 16 +.Lrounds_16_xx: + movq 8(%rsp),%r13 + movq 112(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rax + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 72(%rsp),%r12 + + addq 0(%rsp),%r12 + movq %r8,%r13 + addq %r15,%r12 + movq %rax,%r14 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,0(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + movq 16(%rsp),%r13 + movq 120(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r11 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 80(%rsp),%r12 + + addq 8(%rsp),%r12 + movq %rdx,%r13 + addq %rdi,%r12 + movq %r11,%r14 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,8(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + movq 24(%rsp),%r13 + movq 0(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r10 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 88(%rsp),%r12 + + addq 16(%rsp),%r12 + movq %rcx,%r13 + addq %r15,%r12 + movq %r10,%r14 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,16(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + movq 32(%rsp),%r13 + movq 8(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r9 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 96(%rsp),%r12 + + addq 24(%rsp),%r12 + movq %rbx,%r13 + addq %rdi,%r12 + movq %r9,%r14 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,24(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + movq 40(%rsp),%r13 + movq 16(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r8 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 104(%rsp),%r12 + + addq 32(%rsp),%r12 + movq %rax,%r13 + addq %r15,%r12 + movq %r8,%r14 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,32(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + movq 48(%rsp),%r13 + movq 24(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rdx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 112(%rsp),%r12 + + addq 40(%rsp),%r12 + movq %r11,%r13 + addq %rdi,%r12 + movq %rdx,%r14 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,40(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + movq 56(%rsp),%r13 + movq 32(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rcx + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 120(%rsp),%r12 + + addq 48(%rsp),%r12 + movq %r10,%r13 + addq %r15,%r12 + movq %rcx,%r14 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,48(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + movq 64(%rsp),%r13 + movq 40(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rbx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 0(%rsp),%r12 + + addq 56(%rsp),%r12 + movq %r9,%r13 + addq %rdi,%r12 + movq %rbx,%r14 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,56(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + movq 72(%rsp),%r13 + movq 48(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rax + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 8(%rsp),%r12 + + addq 64(%rsp),%r12 + movq %r8,%r13 + addq %r15,%r12 + movq %rax,%r14 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,64(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + movq 80(%rsp),%r13 + movq 56(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r11 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 16(%rsp),%r12 + + addq 72(%rsp),%r12 + movq %rdx,%r13 + addq %rdi,%r12 + movq %r11,%r14 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,72(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + movq 88(%rsp),%r13 + movq 64(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r10 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 24(%rsp),%r12 + + addq 80(%rsp),%r12 + movq %rcx,%r13 + addq %r15,%r12 + movq %r10,%r14 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,80(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + movq 96(%rsp),%r13 + movq 72(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r9 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 32(%rsp),%r12 + + addq 88(%rsp),%r12 + movq %rbx,%r13 + addq %rdi,%r12 + movq %r9,%r14 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,88(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + movq 104(%rsp),%r13 + movq 80(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r8 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 40(%rsp),%r12 + + addq 96(%rsp),%r12 + movq %rax,%r13 + addq %r15,%r12 + movq %r8,%r14 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,96(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + movq 112(%rsp),%r13 + movq 88(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rdx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 48(%rsp),%r12 + + addq 104(%rsp),%r12 + movq %r11,%r13 + addq %rdi,%r12 + movq %rdx,%r14 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,104(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + movq 120(%rsp),%r13 + movq 96(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rcx + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 56(%rsp),%r12 + + addq 112(%rsp),%r12 + movq %r10,%r13 + addq %r15,%r12 + movq %rcx,%r14 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,112(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + movq 0(%rsp),%r13 + movq 104(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rbx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 64(%rsp),%r12 + + addq 120(%rsp),%r12 + movq %r9,%r13 + addq %rdi,%r12 + movq %rbx,%r14 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,120(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + cmpb $0,7(%rbp) + jnz .Lrounds_16_xx + + movq 128+0(%rsp),%rdi + addq %r14,%rax + leaq 128(%rsi),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + cmpq 128+16(%rsp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb .Lloop + + leaq 128+24+48(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq 128+24(%rsp),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbx + movq -8(%r11),%rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + leaq (%r11),%rsp + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha512_asm_block_data_order,.-crypton_sha512_asm_block_data_order +.align 64 +.type K512,@object +K512: +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + +.quad 0x0001020304050607,0x08090a0b0c0d0e0f +.quad 0x0001020304050607,0x08090a0b0c0d0e0f + +K512_nodup: +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + +.byte 83,72,65,53,49,50,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.globl crypton_sha512_asm_block_data_order_shaext +.type crypton_sha512_asm_block_data_order_shaext,@function +.align 64 +crypton_sha512_asm_block_data_order_shaext: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lshaext_shortcut: + + leaq K512_nodup+128(%rip),%rcx + vmovdqu (%rdi),%ymm0 + vmovdqu 32(%rdi),%ymm1 + vmovdqa -160(%rcx),%ymm8 + + vpermq $27,%ymm0,%ymm0 + vpblendd $15,%ymm1,%ymm0,%ymm5 + vpblendd $15,%ymm0,%ymm1,%ymm6 + vpermq $225,%ymm5,%ymm5 + vpermq $75,%ymm6,%ymm6 + jmp .Loop_shaext + +.align 16 +.Loop_shaext: + vmovdqu (%rsi),%ymm0 + vmovdqu 32(%rsi),%ymm1 + vmovdqu 64(%rsi),%ymm2 + vpshufb %ymm8,%ymm0,%ymm0 + vmovdqu 96(%rsi),%ymm3 + + vpaddq 0-128(%rcx),%ymm0,%ymm4 + vpshufb %ymm8,%ymm1,%ymm1 + vmovdqa %ymm6,%ymm10 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vmovdqa %ymm5,%ymm9 +.byte 196,226,79,203,236 + + vpaddq 32-128(%rcx),%ymm1,%ymm4 + vpshufb %ymm8,%ymm2,%ymm2 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + leaq 128(%rsi),%rsi +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + + vpaddq 64-128(%rcx),%ymm2,%ymm4 + vpshufb %ymm8,%ymm3,%ymm3 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + + vpaddq 96-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 128-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 160-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 192-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 224-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 256-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 288-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 320-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 352-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 384-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 416-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 448-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 480-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 512-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 544-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 +.byte 196,226,79,203,236 + vpaddq %ymm7,%ymm3,%ymm3 + + vpaddq 576-128(%rcx),%ymm2,%ymm4 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 +.byte 196,226,127,205,218 +.byte 196,226,79,203,236 + + vpaddq 608-128(%rcx),%ymm3,%ymm4 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + decq %rdx +.byte 196,226,79,203,236 + + vpaddq %ymm10,%ymm6,%ymm6 + vpaddq %ymm9,%ymm5,%ymm5 + jnz .Loop_shaext + + vpermq $75,%ymm5,%ymm5 + vpblendd $240,%ymm6,%ymm5,%ymm1 + vpblendd $240,%ymm5,%ymm6,%ymm2 + vpermq $180,%ymm1,%ymm1 + vpermq $27,%ymm2,%ymm2 + + vmovdqu %ymm1,(%rdi) + vmovdqu %ymm2,32(%rdi) + + vzeroupper +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp + + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha512_asm_block_data_order_shaext,.-crypton_sha512_asm_block_data_order_shaext +.type crypton_sha512_asm_block_data_order_xop,@function +.align 64 +crypton_sha512_asm_block_data_order_xop: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lxop_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Lloop_xop +.align 16 +.Lloop_xop: + vmovdqa K512+1280(%rip),%xmm11 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vpshufb %xmm11,%xmm0,%xmm0 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm11,%xmm1,%xmm1 + vmovdqu 64(%rsi),%xmm4 + vpshufb %xmm11,%xmm2,%xmm2 + vmovdqu 80(%rsi),%xmm5 + vpshufb %xmm11,%xmm3,%xmm3 + vmovdqu 96(%rsi),%xmm6 + vpshufb %xmm11,%xmm4,%xmm4 + vmovdqu 112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vpshufb %xmm11,%xmm5,%xmm5 + vpaddq -128(%rsi),%xmm0,%xmm8 + vpshufb %xmm11,%xmm6,%xmm6 + vpaddq -96(%rsi),%xmm1,%xmm9 + vpshufb %xmm11,%xmm7,%xmm7 + vpaddq -64(%rsi),%xmm2,%xmm10 + vpaddq -32(%rsi),%xmm3,%xmm11 + vmovdqa %xmm8,0(%rsp) + vpaddq 0(%rsi),%xmm4,%xmm8 + vmovdqa %xmm9,16(%rsp) + vpaddq 32(%rsi),%xmm5,%xmm9 + vmovdqa %xmm10,32(%rsp) + vpaddq 64(%rsi),%xmm6,%xmm10 + vmovdqa %xmm11,48(%rsp) + vpaddq 96(%rsi),%xmm7,%xmm11 + vmovdqa %xmm8,64(%rsp) + movq %rax,%r14 + vmovdqa %xmm9,80(%rsp) + movq %rbx,%rdi + vmovdqa %xmm10,96(%rsp) + xorq %rcx,%rdi + vmovdqa %xmm11,112(%rsp) + movq %r8,%r13 + jmp .Lxop_00_47 + +.align 16 +.Lxop_00_47: + addq $256,%rsi + vpalignr $8,%xmm0,%xmm1,%xmm8 + rorq $23,%r13 + movq %r14,%rax + vpalignr $8,%xmm4,%xmm5,%xmm11 + movq %r9,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r8,%r13 + xorq %r10,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rax,%r14 + vpaddq %xmm11,%xmm0,%xmm0 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 +.byte 143,72,120,195,209,7 + xorq %r10,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,223,3 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm7,%xmm10 + addq %r11,%rdx + addq %rdi,%r11 + vpaddq %xmm8,%xmm0,%xmm0 + movq %rdx,%r13 + addq %r11,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r11 + vpxor %xmm10,%xmm11,%xmm11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + vpaddq %xmm11,%xmm0,%xmm0 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + vpaddq -128(%rsi),%xmm0,%xmm10 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,0(%rsp) + vpalignr $8,%xmm1,%xmm2,%xmm8 + rorq $23,%r13 + movq %r14,%r10 + vpalignr $8,%xmm5,%xmm6,%xmm11 + movq %rdx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rcx,%r13 + xorq %r8,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r10,%r14 + vpaddq %xmm11,%xmm1,%xmm1 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 +.byte 143,72,120,195,209,7 + xorq %r8,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,216,3 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm0,%xmm10 + addq %r9,%rbx + addq %rdi,%r9 + vpaddq %xmm8,%xmm1,%xmm1 + movq %rbx,%r13 + addq %r9,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r9 + vpxor %xmm10,%xmm11,%xmm11 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + vpaddq %xmm11,%xmm1,%xmm1 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + vpaddq -96(%rsi),%xmm1,%xmm10 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,16(%rsp) + vpalignr $8,%xmm2,%xmm3,%xmm8 + rorq $23,%r13 + movq %r14,%r8 + vpalignr $8,%xmm6,%xmm7,%xmm11 + movq %rbx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rax,%r13 + xorq %rcx,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r8,%r14 + vpaddq %xmm11,%xmm2,%xmm2 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 +.byte 143,72,120,195,209,7 + xorq %rcx,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,217,3 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm1,%xmm10 + addq %rdx,%r11 + addq %rdi,%rdx + vpaddq %xmm8,%xmm2,%xmm2 + movq %r11,%r13 + addq %rdx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rdx + vpxor %xmm10,%xmm11,%xmm11 + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + vpaddq %xmm11,%xmm2,%xmm2 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + vpaddq -64(%rsi),%xmm2,%xmm10 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,32(%rsp) + vpalignr $8,%xmm3,%xmm4,%xmm8 + rorq $23,%r13 + movq %r14,%rcx + vpalignr $8,%xmm7,%xmm0,%xmm11 + movq %r11,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r10,%r13 + xorq %rax,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rcx,%r14 + vpaddq %xmm11,%xmm3,%xmm3 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 +.byte 143,72,120,195,209,7 + xorq %rax,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,218,3 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm2,%xmm10 + addq %rbx,%r9 + addq %rdi,%rbx + vpaddq %xmm8,%xmm3,%xmm3 + movq %r9,%r13 + addq %rbx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rbx + vpxor %xmm10,%xmm11,%xmm11 + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + vpaddq %xmm11,%xmm3,%xmm3 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + vpaddq -32(%rsi),%xmm3,%xmm10 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,48(%rsp) + vpalignr $8,%xmm4,%xmm5,%xmm8 + rorq $23,%r13 + movq %r14,%rax + vpalignr $8,%xmm0,%xmm1,%xmm11 + movq %r9,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r8,%r13 + xorq %r10,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rax,%r14 + vpaddq %xmm11,%xmm4,%xmm4 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 +.byte 143,72,120,195,209,7 + xorq %r10,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,219,3 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm3,%xmm10 + addq %r11,%rdx + addq %rdi,%r11 + vpaddq %xmm8,%xmm4,%xmm4 + movq %rdx,%r13 + addq %r11,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r11 + vpxor %xmm10,%xmm11,%xmm11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + vpaddq %xmm11,%xmm4,%xmm4 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + vpaddq 0(%rsi),%xmm4,%xmm10 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,64(%rsp) + vpalignr $8,%xmm5,%xmm6,%xmm8 + rorq $23,%r13 + movq %r14,%r10 + vpalignr $8,%xmm1,%xmm2,%xmm11 + movq %rdx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rcx,%r13 + xorq %r8,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r10,%r14 + vpaddq %xmm11,%xmm5,%xmm5 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 +.byte 143,72,120,195,209,7 + xorq %r8,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,220,3 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm4,%xmm10 + addq %r9,%rbx + addq %rdi,%r9 + vpaddq %xmm8,%xmm5,%xmm5 + movq %rbx,%r13 + addq %r9,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r9 + vpxor %xmm10,%xmm11,%xmm11 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + vpaddq %xmm11,%xmm5,%xmm5 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + vpaddq 32(%rsi),%xmm5,%xmm10 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,80(%rsp) + vpalignr $8,%xmm6,%xmm7,%xmm8 + rorq $23,%r13 + movq %r14,%r8 + vpalignr $8,%xmm2,%xmm3,%xmm11 + movq %rbx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rax,%r13 + xorq %rcx,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r8,%r14 + vpaddq %xmm11,%xmm6,%xmm6 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 +.byte 143,72,120,195,209,7 + xorq %rcx,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,221,3 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm5,%xmm10 + addq %rdx,%r11 + addq %rdi,%rdx + vpaddq %xmm8,%xmm6,%xmm6 + movq %r11,%r13 + addq %rdx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rdx + vpxor %xmm10,%xmm11,%xmm11 + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + vpaddq %xmm11,%xmm6,%xmm6 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + vpaddq 64(%rsi),%xmm6,%xmm10 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,96(%rsp) + vpalignr $8,%xmm7,%xmm0,%xmm8 + rorq $23,%r13 + movq %r14,%rcx + vpalignr $8,%xmm3,%xmm4,%xmm11 + movq %r11,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r10,%r13 + xorq %rax,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rcx,%r14 + vpaddq %xmm11,%xmm7,%xmm7 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 +.byte 143,72,120,195,209,7 + xorq %rax,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,222,3 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm6,%xmm10 + addq %rbx,%r9 + addq %rdi,%rbx + vpaddq %xmm8,%xmm7,%xmm7 + movq %r9,%r13 + addq %rbx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rbx + vpxor %xmm10,%xmm11,%xmm11 + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + vpaddq %xmm11,%xmm7,%xmm7 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + vpaddq 96(%rsi),%xmm7,%xmm10 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,112(%rsp) + cmpb $0,135(%rsi) + jne .Lxop_00_47 + rorq $23,%r13 + movq %r14,%rax + movq %r9,%r12 + rorq $5,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + rorq $4,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + rorq $6,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + rorq $28,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + rorq $23,%r13 + movq %r14,%r11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + rorq $23,%r13 + movq %r14,%r10 + movq %rdx,%r12 + rorq $5,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + rorq $4,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + rorq $6,%r14 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + rorq $28,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + rorq $23,%r13 + movq %r14,%r9 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + rorq $23,%r13 + movq %r14,%r8 + movq %rbx,%r12 + rorq $5,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + rorq $4,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + rorq $6,%r14 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + rorq $28,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + rorq $23,%r13 + movq %r14,%rdx + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + rorq $23,%r13 + movq %r14,%rcx + movq %r11,%r12 + rorq $5,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + rorq $4,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + rorq $6,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + rorq $28,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + rorq $23,%r13 + movq %r14,%rbx + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + rorq $23,%r13 + movq %r14,%rax + movq %r9,%r12 + rorq $5,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + rorq $4,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + rorq $6,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + rorq $28,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + rorq $23,%r13 + movq %r14,%r11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + rorq $23,%r13 + movq %r14,%r10 + movq %rdx,%r12 + rorq $5,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + rorq $4,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + rorq $6,%r14 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + rorq $28,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + rorq $23,%r13 + movq %r14,%r9 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + rorq $23,%r13 + movq %r14,%r8 + movq %rbx,%r12 + rorq $5,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + rorq $4,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + rorq $6,%r14 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + rorq $28,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + rorq $23,%r13 + movq %r14,%rdx + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + rorq $23,%r13 + movq %r14,%rcx + movq %r11,%r12 + rorq $5,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + rorq $4,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + rorq $6,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + rorq $28,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + rorq $23,%r13 + movq %r14,%rbx + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + movq -64(%rbp),%rdi + movq %r14,%rax + movq -56(%rbp),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + leaq 128(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb .Lloop_xop + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha512_asm_block_data_order_xop,.-crypton_sha512_asm_block_data_order_xop +.type crypton_sha512_asm_block_data_order_avx,@function +.align 64 +crypton_sha512_asm_block_data_order_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lavx_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Lloop_avx +.align 16 +.Lloop_avx: + vmovdqa K512+1280(%rip),%xmm11 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vpshufb %xmm11,%xmm0,%xmm0 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm11,%xmm1,%xmm1 + vmovdqu 64(%rsi),%xmm4 + vpshufb %xmm11,%xmm2,%xmm2 + vmovdqu 80(%rsi),%xmm5 + vpshufb %xmm11,%xmm3,%xmm3 + vmovdqu 96(%rsi),%xmm6 + vpshufb %xmm11,%xmm4,%xmm4 + vmovdqu 112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vpshufb %xmm11,%xmm5,%xmm5 + vpaddq -128(%rsi),%xmm0,%xmm8 + vpshufb %xmm11,%xmm6,%xmm6 + vpaddq -96(%rsi),%xmm1,%xmm9 + vpshufb %xmm11,%xmm7,%xmm7 + vpaddq -64(%rsi),%xmm2,%xmm10 + vpaddq -32(%rsi),%xmm3,%xmm11 + vmovdqa %xmm8,0(%rsp) + vpaddq 0(%rsi),%xmm4,%xmm8 + vmovdqa %xmm9,16(%rsp) + vpaddq 32(%rsi),%xmm5,%xmm9 + vmovdqa %xmm10,32(%rsp) + vpaddq 64(%rsi),%xmm6,%xmm10 + vmovdqa %xmm11,48(%rsp) + vpaddq 96(%rsi),%xmm7,%xmm11 + vmovdqa %xmm8,64(%rsp) + movq %rax,%r14 + vmovdqa %xmm9,80(%rsp) + movq %rbx,%rdi + vmovdqa %xmm10,96(%rsp) + xorq %rcx,%rdi + vmovdqa %xmm11,112(%rsp) + movq %r8,%r13 + jmp .Lavx_00_47 + +.align 16 +.Lavx_00_47: + addq $256,%rsi + vpalignr $8,%xmm0,%xmm1,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rax + vpalignr $8,%xmm4,%xmm5,%xmm11 + movq %r9,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r8,%r13 + xorq %r10,%r12 + vpaddq %xmm11,%xmm0,%xmm0 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r8,%r12 + xorq %r8,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 0(%rsp),%r11 + movq %rax,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rbx,%r15 + addq %r12,%r11 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm7,%xmm11 + addq %r11,%rdx + addq %rdi,%r11 + vpxor %xmm9,%xmm8,%xmm8 + movq %rdx,%r13 + addq %r11,%r14 + vpsllq $3,%xmm7,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r11 + vpaddq %xmm8,%xmm0,%xmm0 + movq %r8,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm7,%xmm9 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rdx,%r12 + xorq %rdx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 8(%rsp),%r10 + movq %r11,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r9,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rax,%rdi + addq %r12,%r10 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm0,%xmm0 + xorq %r11,%r14 + addq %r13,%r10 + vpaddq -128(%rsi),%xmm0,%xmm10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,0(%rsp) + vpalignr $8,%xmm1,%xmm2,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r10 + vpalignr $8,%xmm5,%xmm6,%xmm11 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rcx,%r13 + xorq %r8,%r12 + vpaddq %xmm11,%xmm1,%xmm1 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rcx,%r12 + xorq %rcx,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 16(%rsp),%r9 + movq %r10,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r11,%r15 + addq %r12,%r9 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm0,%xmm11 + addq %r9,%rbx + addq %rdi,%r9 + vpxor %xmm9,%xmm8,%xmm8 + movq %rbx,%r13 + addq %r9,%r14 + vpsllq $3,%xmm0,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r9 + vpaddq %xmm8,%xmm1,%xmm1 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm0,%xmm9 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rbx,%r12 + xorq %rbx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 24(%rsp),%r8 + movq %r9,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r10,%rdi + addq %r12,%r8 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm1,%xmm1 + xorq %r9,%r14 + addq %r13,%r8 + vpaddq -96(%rsi),%xmm1,%xmm10 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,16(%rsp) + vpalignr $8,%xmm2,%xmm3,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r8 + vpalignr $8,%xmm6,%xmm7,%xmm11 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rax,%r13 + xorq %rcx,%r12 + vpaddq %xmm11,%xmm2,%xmm2 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rax,%r12 + xorq %rax,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 32(%rsp),%rdx + movq %r8,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r9,%r15 + addq %r12,%rdx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm1,%xmm11 + addq %rdx,%r11 + addq %rdi,%rdx + vpxor %xmm9,%xmm8,%xmm8 + movq %r11,%r13 + addq %rdx,%r14 + vpsllq $3,%xmm1,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rdx + vpaddq %xmm8,%xmm2,%xmm2 + movq %rax,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm1,%xmm9 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r11,%r12 + xorq %r11,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 40(%rsp),%rcx + movq %rdx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r8,%rdi + addq %r12,%rcx + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm2,%xmm2 + xorq %rdx,%r14 + addq %r13,%rcx + vpaddq -64(%rsi),%xmm2,%xmm10 + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,32(%rsp) + vpalignr $8,%xmm3,%xmm4,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rcx + vpalignr $8,%xmm7,%xmm0,%xmm11 + movq %r11,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r10,%r13 + xorq %rax,%r12 + vpaddq %xmm11,%xmm3,%xmm3 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r10,%r12 + xorq %r10,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 48(%rsp),%rbx + movq %rcx,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rdx,%r15 + addq %r12,%rbx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm2,%xmm11 + addq %rbx,%r9 + addq %rdi,%rbx + vpxor %xmm9,%xmm8,%xmm8 + movq %r9,%r13 + addq %rbx,%r14 + vpsllq $3,%xmm2,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rbx + vpaddq %xmm8,%xmm3,%xmm3 + movq %r10,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm2,%xmm9 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r9,%r12 + xorq %r9,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 56(%rsp),%rax + movq %rbx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r11,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rcx,%rdi + addq %r12,%rax + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm3,%xmm3 + xorq %rbx,%r14 + addq %r13,%rax + vpaddq -32(%rsi),%xmm3,%xmm10 + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,48(%rsp) + vpalignr $8,%xmm4,%xmm5,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rax + vpalignr $8,%xmm0,%xmm1,%xmm11 + movq %r9,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r8,%r13 + xorq %r10,%r12 + vpaddq %xmm11,%xmm4,%xmm4 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r8,%r12 + xorq %r8,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 64(%rsp),%r11 + movq %rax,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rbx,%r15 + addq %r12,%r11 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm3,%xmm11 + addq %r11,%rdx + addq %rdi,%r11 + vpxor %xmm9,%xmm8,%xmm8 + movq %rdx,%r13 + addq %r11,%r14 + vpsllq $3,%xmm3,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r11 + vpaddq %xmm8,%xmm4,%xmm4 + movq %r8,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm3,%xmm9 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rdx,%r12 + xorq %rdx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 72(%rsp),%r10 + movq %r11,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r9,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rax,%rdi + addq %r12,%r10 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm4,%xmm4 + xorq %r11,%r14 + addq %r13,%r10 + vpaddq 0(%rsi),%xmm4,%xmm10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,64(%rsp) + vpalignr $8,%xmm5,%xmm6,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r10 + vpalignr $8,%xmm1,%xmm2,%xmm11 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rcx,%r13 + xorq %r8,%r12 + vpaddq %xmm11,%xmm5,%xmm5 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rcx,%r12 + xorq %rcx,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 80(%rsp),%r9 + movq %r10,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r11,%r15 + addq %r12,%r9 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm4,%xmm11 + addq %r9,%rbx + addq %rdi,%r9 + vpxor %xmm9,%xmm8,%xmm8 + movq %rbx,%r13 + addq %r9,%r14 + vpsllq $3,%xmm4,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r9 + vpaddq %xmm8,%xmm5,%xmm5 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm4,%xmm9 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rbx,%r12 + xorq %rbx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 88(%rsp),%r8 + movq %r9,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r10,%rdi + addq %r12,%r8 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm5,%xmm5 + xorq %r9,%r14 + addq %r13,%r8 + vpaddq 32(%rsi),%xmm5,%xmm10 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,80(%rsp) + vpalignr $8,%xmm6,%xmm7,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r8 + vpalignr $8,%xmm2,%xmm3,%xmm11 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rax,%r13 + xorq %rcx,%r12 + vpaddq %xmm11,%xmm6,%xmm6 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rax,%r12 + xorq %rax,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 96(%rsp),%rdx + movq %r8,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r9,%r15 + addq %r12,%rdx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm5,%xmm11 + addq %rdx,%r11 + addq %rdi,%rdx + vpxor %xmm9,%xmm8,%xmm8 + movq %r11,%r13 + addq %rdx,%r14 + vpsllq $3,%xmm5,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rdx + vpaddq %xmm8,%xmm6,%xmm6 + movq %rax,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm5,%xmm9 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r11,%r12 + xorq %r11,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 104(%rsp),%rcx + movq %rdx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r8,%rdi + addq %r12,%rcx + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm6,%xmm6 + xorq %rdx,%r14 + addq %r13,%rcx + vpaddq 64(%rsi),%xmm6,%xmm10 + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,96(%rsp) + vpalignr $8,%xmm7,%xmm0,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rcx + vpalignr $8,%xmm3,%xmm4,%xmm11 + movq %r11,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r10,%r13 + xorq %rax,%r12 + vpaddq %xmm11,%xmm7,%xmm7 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r10,%r12 + xorq %r10,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 112(%rsp),%rbx + movq %rcx,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rdx,%r15 + addq %r12,%rbx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm6,%xmm11 + addq %rbx,%r9 + addq %rdi,%rbx + vpxor %xmm9,%xmm8,%xmm8 + movq %r9,%r13 + addq %rbx,%r14 + vpsllq $3,%xmm6,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rbx + vpaddq %xmm8,%xmm7,%xmm7 + movq %r10,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm6,%xmm9 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r9,%r12 + xorq %r9,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 120(%rsp),%rax + movq %rbx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r11,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rcx,%rdi + addq %r12,%rax + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm7,%xmm7 + xorq %rbx,%r14 + addq %r13,%rax + vpaddq 96(%rsi),%xmm7,%xmm10 + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,112(%rsp) + cmpb $0,135(%rsi) + jne .Lavx_00_47 + shrdq $23,%r13,%r13 + movq %r14,%rax + movq %r9,%r12 + shrdq $5,%r14,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r11 + movq %r8,%r12 + shrdq $5,%r14,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + shrdq $6,%r14,%r14 + xorq %rax,%rdi + addq %r12,%r10 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r10 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + xorq %r11,%r15 + addq %r12,%r9 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r9 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + xorq %r10,%rdi + addq %r12,%r8 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r8 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + xorq %r9,%r15 + addq %r12,%rdx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + shrdq $28,%r14,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rdx + movq %rax,%r12 + shrdq $5,%r14,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + xorq %r8,%rdi + addq %r12,%rcx + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rcx + movq %r11,%r12 + shrdq $5,%r14,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rbx + movq %r10,%r12 + shrdq $5,%r14,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + shrdq $6,%r14,%r14 + xorq %rcx,%rdi + addq %r12,%rax + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rax + movq %r9,%r12 + shrdq $5,%r14,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r11 + movq %r8,%r12 + shrdq $5,%r14,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + shrdq $6,%r14,%r14 + xorq %rax,%rdi + addq %r12,%r10 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r10 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + xorq %r11,%r15 + addq %r12,%r9 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r9 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + xorq %r10,%rdi + addq %r12,%r8 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r8 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + xorq %r9,%r15 + addq %r12,%rdx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + shrdq $28,%r14,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rdx + movq %rax,%r12 + shrdq $5,%r14,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + xorq %r8,%rdi + addq %r12,%rcx + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rcx + movq %r11,%r12 + shrdq $5,%r14,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rbx + movq %r10,%r12 + shrdq $5,%r14,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + shrdq $6,%r14,%r14 + xorq %rcx,%rdi + addq %r12,%rax + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + movq -64(%rbp),%rdi + movq %r14,%rax + movq -56(%rbp),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + leaq 128(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb .Lloop_avx + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha512_asm_block_data_order_avx,.-crypton_sha512_asm_block_data_order_avx +.type crypton_sha512_asm_block_data_order_avx2,@function +.align 64 +crypton_sha512_asm_block_data_order_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lavx2_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-128,%rsp + subq $-128,%rsi + movq 0(%rdi),%rax + movq %rsi,%r12 + movq 8(%rdi),%rbx + cmpq %rdx,%rsi + movq 16(%rdi),%rcx + cmoveq %rsp,%r12 + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Loop_avx2 +.align 16 +.Loop_avx2: + vmovdqa K512+1280(%rip),%ymm10 + movq %rsi,-56(%rbp) + vmovdqu -128(%rsi),%xmm0 + vmovdqu -128+16(%rsi),%xmm1 + vmovdqu -128+32(%rsi),%xmm2 + vmovdqu -128+48(%rsi),%xmm3 + vmovdqu -128+64(%rsi),%xmm4 + vmovdqu -128+80(%rsi),%xmm5 + vmovdqu -128+96(%rsi),%xmm6 + vmovdqu -128+112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vinserti128 $1,(%r12),%ymm0,%ymm0 + vinserti128 $1,16(%r12),%ymm1,%ymm1 + vpshufb %ymm10,%ymm0,%ymm0 + vinserti128 $1,32(%r12),%ymm2,%ymm2 + vpshufb %ymm10,%ymm1,%ymm1 + vinserti128 $1,48(%r12),%ymm3,%ymm3 + vpshufb %ymm10,%ymm2,%ymm2 + vinserti128 $1,64(%r12),%ymm4,%ymm4 + vpshufb %ymm10,%ymm3,%ymm3 + vinserti128 $1,80(%r12),%ymm5,%ymm5 + vpshufb %ymm10,%ymm4,%ymm4 + vinserti128 $1,96(%r12),%ymm6,%ymm6 + vpshufb %ymm10,%ymm5,%ymm5 + vinserti128 $1,112(%r12),%ymm7,%ymm7 + + vpaddq -128(%rsi),%ymm0,%ymm8 + vpshufb %ymm10,%ymm6,%ymm6 + vpaddq -96(%rsi),%ymm1,%ymm9 + vpshufb %ymm10,%ymm7,%ymm7 + vpaddq -64(%rsi),%ymm2,%ymm10 + vpaddq -32(%rsi),%ymm3,%ymm11 + vmovdqa %ymm8,0(%rsp) + vpaddq 0(%rsi),%ymm4,%ymm8 + vmovdqa %ymm9,32(%rsp) + vpaddq 32(%rsi),%ymm5,%ymm9 + vmovdqa %ymm10,64(%rsp) + vpaddq 64(%rsi),%ymm6,%ymm10 + vmovdqa %ymm11,96(%rsp) + leaq -128(%rsp),%rsp + vpaddq 96(%rsi),%ymm7,%ymm11 + vmovdqa %ymm8,0(%rsp) + xorq %r14,%r14 + vmovdqa %ymm9,32(%rsp) + movq %rbx,%rdi + vmovdqa %ymm10,64(%rsp) + xorq %rcx,%rdi + vmovdqa %ymm11,96(%rsp) + movq %r9,%r12 + addq $32*8,%rsi + jmp .Lavx2_00_47 + +.align 16 +.Lavx2_00_47: + leaq -128(%rsp),%rsp + vpalignr $8,%ymm0,%ymm1,%ymm8 + addq 0+256(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + vpalignr $8,%ymm4,%ymm5,%ymm11 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + vpsrlq $1,%ymm8,%ymm10 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + vpaddq %ymm11,%ymm0,%ymm0 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + vpsrlq $6,%ymm7,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + vpsllq $3,%ymm7,%ymm10 + vpaddq %ymm8,%ymm0,%ymm0 + addq 8+256(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + vpsrlq $19,%ymm7,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + vpaddq %ymm11,%ymm0,%ymm0 + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + vpaddq -128(%rsi),%ymm0,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + vmovdqa %ymm10,0(%rsp) + vpalignr $8,%ymm1,%ymm2,%ymm8 + addq 32+256(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + vpalignr $8,%ymm5,%ymm6,%ymm11 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + vpsrlq $1,%ymm8,%ymm10 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + vpaddq %ymm11,%ymm1,%ymm1 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + vpsrlq $6,%ymm0,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + vpsllq $3,%ymm0,%ymm10 + vpaddq %ymm8,%ymm1,%ymm1 + addq 40+256(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + vpsrlq $19,%ymm0,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + vpaddq %ymm11,%ymm1,%ymm1 + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + vpaddq -96(%rsi),%ymm1,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + vmovdqa %ymm10,32(%rsp) + vpalignr $8,%ymm2,%ymm3,%ymm8 + addq 64+256(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + vpalignr $8,%ymm6,%ymm7,%ymm11 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + vpsrlq $1,%ymm8,%ymm10 + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + vpaddq %ymm11,%ymm2,%ymm2 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + vpsrlq $6,%ymm1,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + vpsllq $3,%ymm1,%ymm10 + vpaddq %ymm8,%ymm2,%ymm2 + addq 72+256(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + vpsrlq $19,%ymm1,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + vpaddq %ymm11,%ymm2,%ymm2 + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + vpaddq -64(%rsi),%ymm2,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + vmovdqa %ymm10,64(%rsp) + vpalignr $8,%ymm3,%ymm4,%ymm8 + addq 96+256(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + vpalignr $8,%ymm7,%ymm0,%ymm11 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + vpsrlq $1,%ymm8,%ymm10 + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + vpaddq %ymm11,%ymm3,%ymm3 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + vpsrlq $6,%ymm2,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + vpsllq $3,%ymm2,%ymm10 + vpaddq %ymm8,%ymm3,%ymm3 + addq 104+256(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + vpsrlq $19,%ymm2,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + vpaddq %ymm11,%ymm3,%ymm3 + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + vpaddq -32(%rsi),%ymm3,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + vmovdqa %ymm10,96(%rsp) + leaq -128(%rsp),%rsp + vpalignr $8,%ymm4,%ymm5,%ymm8 + addq 0+256(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + vpalignr $8,%ymm0,%ymm1,%ymm11 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + vpsrlq $1,%ymm8,%ymm10 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + vpaddq %ymm11,%ymm4,%ymm4 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + vpsrlq $6,%ymm3,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + vpsllq $3,%ymm3,%ymm10 + vpaddq %ymm8,%ymm4,%ymm4 + addq 8+256(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + vpsrlq $19,%ymm3,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + vpaddq %ymm11,%ymm4,%ymm4 + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + vpaddq 0(%rsi),%ymm4,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + vmovdqa %ymm10,0(%rsp) + vpalignr $8,%ymm5,%ymm6,%ymm8 + addq 32+256(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + vpalignr $8,%ymm1,%ymm2,%ymm11 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + vpsrlq $1,%ymm8,%ymm10 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + vpaddq %ymm11,%ymm5,%ymm5 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + vpsrlq $6,%ymm4,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + vpsllq $3,%ymm4,%ymm10 + vpaddq %ymm8,%ymm5,%ymm5 + addq 40+256(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + vpsrlq $19,%ymm4,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + vpaddq %ymm11,%ymm5,%ymm5 + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + vpaddq 32(%rsi),%ymm5,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + vmovdqa %ymm10,32(%rsp) + vpalignr $8,%ymm6,%ymm7,%ymm8 + addq 64+256(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + vpalignr $8,%ymm2,%ymm3,%ymm11 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + vpsrlq $1,%ymm8,%ymm10 + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + vpaddq %ymm11,%ymm6,%ymm6 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + vpsrlq $6,%ymm5,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + vpsllq $3,%ymm5,%ymm10 + vpaddq %ymm8,%ymm6,%ymm6 + addq 72+256(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + vpsrlq $19,%ymm5,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + vpaddq %ymm11,%ymm6,%ymm6 + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + vpaddq 64(%rsi),%ymm6,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + vmovdqa %ymm10,64(%rsp) + vpalignr $8,%ymm7,%ymm0,%ymm8 + addq 96+256(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + vpalignr $8,%ymm3,%ymm4,%ymm11 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + vpsrlq $1,%ymm8,%ymm10 + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + vpaddq %ymm11,%ymm7,%ymm7 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + vpsrlq $6,%ymm6,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + vpsllq $3,%ymm6,%ymm10 + vpaddq %ymm8,%ymm7,%ymm7 + addq 104+256(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + vpsrlq $19,%ymm6,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + vpaddq %ymm11,%ymm7,%ymm7 + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + vpaddq 96(%rsi),%ymm7,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + vmovdqa %ymm10,96(%rsp) + leaq 256(%rsi),%rsi + cmpb $0,-121(%rsi) + jne .Lavx2_00_47 + addq 0+128(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8+128(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32+128(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40+128(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64+128(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72+128(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96+128(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104+128(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + addq 0(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + movq -64(%rbp),%rdi + addq %r14,%rax + movq -56(%rbp),%r12 + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + + cmpq -48(%rbp),%r12 + je .Ldone_avx2 + + leaq 1152(%rsp),%rsi + xorq %r14,%r14 + movq %rbx,%rdi + xorq %rcx,%rdi + movq %r9,%r12 + jmp .Lower_avx2 +.align 16 +.Lower_avx2: + addq 0+16(%rsi),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8+16(%rsi),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32+16(%rsi),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40+16(%rsi),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64+16(%rsi),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72+16(%rsi),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96+16(%rsi),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104+16(%rsi),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + leaq -128(%rsi),%rsi + cmpq %rsp,%rsi + jae .Lower_avx2 + + movq -64(%rbp),%rdi + addq %r14,%rax + movq -56(%rbp),%rsi + leaq 1152(%rsp),%rsp + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + leaq 256(%rsi),%rsi + addq 48(%rdi),%r10 + movq %rsi,%r12 + addq 56(%rdi),%r11 + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + cmoveq %rsp,%r12 + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + + jbe .Loop_avx2 + +.Ldone_avx2: + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc +.size crypton_sha512_asm_block_data_order_avx2,.-crypton_sha512_asm_block_data_order_avx2 + +.section .note.gnu.property,"a",@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align 8 +2: + +.section .note.GNU-stack,"",@progbits diff --git a/cbits/asm/sha512-x86_64-macosx.S b/cbits/asm/sha512-x86_64-macosx.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha512-x86_64-macosx.S @@ -0,0 +1,5718 @@ +.text + + +.globl _crypton_sha512_asm_block_data_order + +.p2align 4 +_crypton_sha512_asm_block_data_order: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp + leaq _crypton_ia32cap_P(%rip),%rax + movl 0(%rax),%r9d + movl 4(%rax),%r10d + movl 8(%rax),%eax + testl $2048,%r10d + jnz L$xop_shortcut + andl $296,%eax + cmpl $296,%eax + je L$avx2_shortcut + andl $1073741824,%r9d + andl $268435968,%r10d + orl %r9d,%r10d + cmpl $1342177792,%r10d + je L$avx_shortcut + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $128+24,%rsp + +.cfi_def_cfa %rsp,208 + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,128+0(%rsp) + movq %rsi,128+8(%rsp) + movq %rdx,128+16(%rsp) + + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp L$loop + +.p2align 4 +L$loop: + movq %rbx,%rdi + leaq K512(%rip),%rbp + xorq %rcx,%rdi + movq 0(%rsi),%r12 + movq %r8,%r13 + movq %rax,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,0(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + addq %r14,%r11 + movq 8(%rsi),%r12 + movq %rdx,%r13 + movq %r11,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,8(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + addq %r14,%r10 + movq 16(%rsi),%r12 + movq %rcx,%r13 + movq %r10,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,16(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + addq %r14,%r9 + movq 24(%rsi),%r12 + movq %rbx,%r13 + movq %r9,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,24(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + addq %r14,%r8 + movq 32(%rsi),%r12 + movq %rax,%r13 + movq %r8,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,32(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + addq %r14,%rdx + movq 40(%rsi),%r12 + movq %r11,%r13 + movq %rdx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,40(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + addq %r14,%rcx + movq 48(%rsi),%r12 + movq %r10,%r13 + movq %rcx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,48(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + addq %r14,%rbx + movq 56(%rsi),%r12 + movq %r9,%r13 + movq %rbx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,56(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + addq %r14,%rax + movq 64(%rsi),%r12 + movq %r8,%r13 + movq %rax,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,64(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + addq %r14,%r11 + movq 72(%rsi),%r12 + movq %rdx,%r13 + movq %r11,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,72(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + addq %r14,%r10 + movq 80(%rsi),%r12 + movq %rcx,%r13 + movq %r10,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,80(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + addq %r14,%r9 + movq 88(%rsi),%r12 + movq %rbx,%r13 + movq %r9,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,88(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + addq %r14,%r8 + movq 96(%rsi),%r12 + movq %rax,%r13 + movq %r8,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,96(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + addq %r14,%rdx + movq 104(%rsi),%r12 + movq %r11,%r13 + movq %rdx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,104(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + addq %r14,%rcx + movq 112(%rsi),%r12 + movq %r10,%r13 + movq %rcx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,112(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + addq %r14,%rbx + movq 120(%rsi),%r12 + movq %r9,%r13 + movq %rbx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,120(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + jmp L$rounds_16_xx +.p2align 4 +L$rounds_16_xx: + movq 8(%rsp),%r13 + movq 112(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rax + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 72(%rsp),%r12 + + addq 0(%rsp),%r12 + movq %r8,%r13 + addq %r15,%r12 + movq %rax,%r14 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,0(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + movq 16(%rsp),%r13 + movq 120(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r11 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 80(%rsp),%r12 + + addq 8(%rsp),%r12 + movq %rdx,%r13 + addq %rdi,%r12 + movq %r11,%r14 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,8(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + movq 24(%rsp),%r13 + movq 0(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r10 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 88(%rsp),%r12 + + addq 16(%rsp),%r12 + movq %rcx,%r13 + addq %r15,%r12 + movq %r10,%r14 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,16(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + movq 32(%rsp),%r13 + movq 8(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r9 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 96(%rsp),%r12 + + addq 24(%rsp),%r12 + movq %rbx,%r13 + addq %rdi,%r12 + movq %r9,%r14 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,24(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + movq 40(%rsp),%r13 + movq 16(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r8 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 104(%rsp),%r12 + + addq 32(%rsp),%r12 + movq %rax,%r13 + addq %r15,%r12 + movq %r8,%r14 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,32(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + movq 48(%rsp),%r13 + movq 24(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rdx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 112(%rsp),%r12 + + addq 40(%rsp),%r12 + movq %r11,%r13 + addq %rdi,%r12 + movq %rdx,%r14 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,40(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + movq 56(%rsp),%r13 + movq 32(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rcx + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 120(%rsp),%r12 + + addq 48(%rsp),%r12 + movq %r10,%r13 + addq %r15,%r12 + movq %rcx,%r14 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,48(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + movq 64(%rsp),%r13 + movq 40(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rbx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 0(%rsp),%r12 + + addq 56(%rsp),%r12 + movq %r9,%r13 + addq %rdi,%r12 + movq %rbx,%r14 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,56(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + movq 72(%rsp),%r13 + movq 48(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rax + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 8(%rsp),%r12 + + addq 64(%rsp),%r12 + movq %r8,%r13 + addq %r15,%r12 + movq %rax,%r14 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,64(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + movq 80(%rsp),%r13 + movq 56(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r11 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 16(%rsp),%r12 + + addq 72(%rsp),%r12 + movq %rdx,%r13 + addq %rdi,%r12 + movq %r11,%r14 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,72(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + movq 88(%rsp),%r13 + movq 64(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r10 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 24(%rsp),%r12 + + addq 80(%rsp),%r12 + movq %rcx,%r13 + addq %r15,%r12 + movq %r10,%r14 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,80(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + movq 96(%rsp),%r13 + movq 72(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r9 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 32(%rsp),%r12 + + addq 88(%rsp),%r12 + movq %rbx,%r13 + addq %rdi,%r12 + movq %r9,%r14 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,88(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + movq 104(%rsp),%r13 + movq 80(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r8 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 40(%rsp),%r12 + + addq 96(%rsp),%r12 + movq %rax,%r13 + addq %r15,%r12 + movq %r8,%r14 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,96(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + movq 112(%rsp),%r13 + movq 88(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rdx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 48(%rsp),%r12 + + addq 104(%rsp),%r12 + movq %r11,%r13 + addq %rdi,%r12 + movq %rdx,%r14 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,104(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + movq 120(%rsp),%r13 + movq 96(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rcx + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 56(%rsp),%r12 + + addq 112(%rsp),%r12 + movq %r10,%r13 + addq %r15,%r12 + movq %rcx,%r14 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,112(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + movq 0(%rsp),%r13 + movq 104(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rbx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 64(%rsp),%r12 + + addq 120(%rsp),%r12 + movq %r9,%r13 + addq %rdi,%r12 + movq %rbx,%r14 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,120(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + cmpb $0,7(%rbp) + jnz L$rounds_16_xx + + movq 128+0(%rsp),%rdi + addq %r14,%rax + leaq 128(%rsi),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + cmpq 128+16(%rsp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb L$loop + + leaq 128+24+48(%rsp),%r11 +.cfi_def_cfa %r11,8 + movq 128+24(%rsp),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbx + movq -8(%r11),%rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbp +.cfi_restore %rbx + leaq (%r11),%rsp + .byte 0xf3,0xc3 +.cfi_endproc + +.p2align 6 + +K512: +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + +.quad 0x0001020304050607,0x08090a0b0c0d0e0f +.quad 0x0001020304050607,0x08090a0b0c0d0e0f + +K512_nodup: +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + +.byte 83,72,65,53,49,50,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.globl _crypton_sha512_asm_block_data_order_shaext + +.p2align 6 +_crypton_sha512_asm_block_data_order_shaext: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$shaext_shortcut: + + leaq K512_nodup+128(%rip),%rcx + vmovdqu (%rdi),%ymm0 + vmovdqu 32(%rdi),%ymm1 + vmovdqa -160(%rcx),%ymm8 + + vpermq $27,%ymm0,%ymm0 + vpblendd $15,%ymm1,%ymm0,%ymm5 + vpblendd $15,%ymm0,%ymm1,%ymm6 + vpermq $225,%ymm5,%ymm5 + vpermq $75,%ymm6,%ymm6 + jmp L$oop_shaext + +.p2align 4 +L$oop_shaext: + vmovdqu (%rsi),%ymm0 + vmovdqu 32(%rsi),%ymm1 + vmovdqu 64(%rsi),%ymm2 + vpshufb %ymm8,%ymm0,%ymm0 + vmovdqu 96(%rsi),%ymm3 + + vpaddq 0-128(%rcx),%ymm0,%ymm4 + vpshufb %ymm8,%ymm1,%ymm1 + vmovdqa %ymm6,%ymm10 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vmovdqa %ymm5,%ymm9 +.byte 196,226,79,203,236 + + vpaddq 32-128(%rcx),%ymm1,%ymm4 + vpshufb %ymm8,%ymm2,%ymm2 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + leaq 128(%rsi),%rsi +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + + vpaddq 64-128(%rcx),%ymm2,%ymm4 + vpshufb %ymm8,%ymm3,%ymm3 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + + vpaddq 96-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 128-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 160-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 192-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 224-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 256-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 288-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 320-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 352-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 384-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 416-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 448-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 480-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 512-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 544-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 +.byte 196,226,79,203,236 + vpaddq %ymm7,%ymm3,%ymm3 + + vpaddq 576-128(%rcx),%ymm2,%ymm4 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 +.byte 196,226,127,205,218 +.byte 196,226,79,203,236 + + vpaddq 608-128(%rcx),%ymm3,%ymm4 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + decq %rdx +.byte 196,226,79,203,236 + + vpaddq %ymm10,%ymm6,%ymm6 + vpaddq %ymm9,%ymm5,%ymm5 + jnz L$oop_shaext + + vpermq $75,%ymm5,%ymm5 + vpblendd $240,%ymm6,%ymm5,%ymm1 + vpblendd $240,%ymm5,%ymm6,%ymm2 + vpermq $180,%ymm1,%ymm1 + vpermq $27,%ymm2,%ymm2 + + vmovdqu %ymm1,(%rdi) + vmovdqu %ymm2,32(%rdi) + + vzeroupper +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp + + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 6 +crypton_sha512_asm_block_data_order_xop: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$xop_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp L$loop_xop +.p2align 4 +L$loop_xop: + vmovdqa K512+1280(%rip),%xmm11 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vpshufb %xmm11,%xmm0,%xmm0 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm11,%xmm1,%xmm1 + vmovdqu 64(%rsi),%xmm4 + vpshufb %xmm11,%xmm2,%xmm2 + vmovdqu 80(%rsi),%xmm5 + vpshufb %xmm11,%xmm3,%xmm3 + vmovdqu 96(%rsi),%xmm6 + vpshufb %xmm11,%xmm4,%xmm4 + vmovdqu 112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vpshufb %xmm11,%xmm5,%xmm5 + vpaddq -128(%rsi),%xmm0,%xmm8 + vpshufb %xmm11,%xmm6,%xmm6 + vpaddq -96(%rsi),%xmm1,%xmm9 + vpshufb %xmm11,%xmm7,%xmm7 + vpaddq -64(%rsi),%xmm2,%xmm10 + vpaddq -32(%rsi),%xmm3,%xmm11 + vmovdqa %xmm8,0(%rsp) + vpaddq 0(%rsi),%xmm4,%xmm8 + vmovdqa %xmm9,16(%rsp) + vpaddq 32(%rsi),%xmm5,%xmm9 + vmovdqa %xmm10,32(%rsp) + vpaddq 64(%rsi),%xmm6,%xmm10 + vmovdqa %xmm11,48(%rsp) + vpaddq 96(%rsi),%xmm7,%xmm11 + vmovdqa %xmm8,64(%rsp) + movq %rax,%r14 + vmovdqa %xmm9,80(%rsp) + movq %rbx,%rdi + vmovdqa %xmm10,96(%rsp) + xorq %rcx,%rdi + vmovdqa %xmm11,112(%rsp) + movq %r8,%r13 + jmp L$xop_00_47 + +.p2align 4 +L$xop_00_47: + addq $256,%rsi + vpalignr $8,%xmm0,%xmm1,%xmm8 + rorq $23,%r13 + movq %r14,%rax + vpalignr $8,%xmm4,%xmm5,%xmm11 + movq %r9,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r8,%r13 + xorq %r10,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rax,%r14 + vpaddq %xmm11,%xmm0,%xmm0 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 +.byte 143,72,120,195,209,7 + xorq %r10,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,223,3 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm7,%xmm10 + addq %r11,%rdx + addq %rdi,%r11 + vpaddq %xmm8,%xmm0,%xmm0 + movq %rdx,%r13 + addq %r11,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r11 + vpxor %xmm10,%xmm11,%xmm11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + vpaddq %xmm11,%xmm0,%xmm0 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + vpaddq -128(%rsi),%xmm0,%xmm10 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,0(%rsp) + vpalignr $8,%xmm1,%xmm2,%xmm8 + rorq $23,%r13 + movq %r14,%r10 + vpalignr $8,%xmm5,%xmm6,%xmm11 + movq %rdx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rcx,%r13 + xorq %r8,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r10,%r14 + vpaddq %xmm11,%xmm1,%xmm1 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 +.byte 143,72,120,195,209,7 + xorq %r8,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,216,3 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm0,%xmm10 + addq %r9,%rbx + addq %rdi,%r9 + vpaddq %xmm8,%xmm1,%xmm1 + movq %rbx,%r13 + addq %r9,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r9 + vpxor %xmm10,%xmm11,%xmm11 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + vpaddq %xmm11,%xmm1,%xmm1 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + vpaddq -96(%rsi),%xmm1,%xmm10 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,16(%rsp) + vpalignr $8,%xmm2,%xmm3,%xmm8 + rorq $23,%r13 + movq %r14,%r8 + vpalignr $8,%xmm6,%xmm7,%xmm11 + movq %rbx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rax,%r13 + xorq %rcx,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r8,%r14 + vpaddq %xmm11,%xmm2,%xmm2 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 +.byte 143,72,120,195,209,7 + xorq %rcx,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,217,3 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm1,%xmm10 + addq %rdx,%r11 + addq %rdi,%rdx + vpaddq %xmm8,%xmm2,%xmm2 + movq %r11,%r13 + addq %rdx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rdx + vpxor %xmm10,%xmm11,%xmm11 + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + vpaddq %xmm11,%xmm2,%xmm2 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + vpaddq -64(%rsi),%xmm2,%xmm10 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,32(%rsp) + vpalignr $8,%xmm3,%xmm4,%xmm8 + rorq $23,%r13 + movq %r14,%rcx + vpalignr $8,%xmm7,%xmm0,%xmm11 + movq %r11,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r10,%r13 + xorq %rax,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rcx,%r14 + vpaddq %xmm11,%xmm3,%xmm3 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 +.byte 143,72,120,195,209,7 + xorq %rax,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,218,3 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm2,%xmm10 + addq %rbx,%r9 + addq %rdi,%rbx + vpaddq %xmm8,%xmm3,%xmm3 + movq %r9,%r13 + addq %rbx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rbx + vpxor %xmm10,%xmm11,%xmm11 + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + vpaddq %xmm11,%xmm3,%xmm3 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + vpaddq -32(%rsi),%xmm3,%xmm10 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,48(%rsp) + vpalignr $8,%xmm4,%xmm5,%xmm8 + rorq $23,%r13 + movq %r14,%rax + vpalignr $8,%xmm0,%xmm1,%xmm11 + movq %r9,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r8,%r13 + xorq %r10,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rax,%r14 + vpaddq %xmm11,%xmm4,%xmm4 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 +.byte 143,72,120,195,209,7 + xorq %r10,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,219,3 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm3,%xmm10 + addq %r11,%rdx + addq %rdi,%r11 + vpaddq %xmm8,%xmm4,%xmm4 + movq %rdx,%r13 + addq %r11,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r11 + vpxor %xmm10,%xmm11,%xmm11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + vpaddq %xmm11,%xmm4,%xmm4 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + vpaddq 0(%rsi),%xmm4,%xmm10 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,64(%rsp) + vpalignr $8,%xmm5,%xmm6,%xmm8 + rorq $23,%r13 + movq %r14,%r10 + vpalignr $8,%xmm1,%xmm2,%xmm11 + movq %rdx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rcx,%r13 + xorq %r8,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r10,%r14 + vpaddq %xmm11,%xmm5,%xmm5 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 +.byte 143,72,120,195,209,7 + xorq %r8,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,220,3 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm4,%xmm10 + addq %r9,%rbx + addq %rdi,%r9 + vpaddq %xmm8,%xmm5,%xmm5 + movq %rbx,%r13 + addq %r9,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r9 + vpxor %xmm10,%xmm11,%xmm11 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + vpaddq %xmm11,%xmm5,%xmm5 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + vpaddq 32(%rsi),%xmm5,%xmm10 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,80(%rsp) + vpalignr $8,%xmm6,%xmm7,%xmm8 + rorq $23,%r13 + movq %r14,%r8 + vpalignr $8,%xmm2,%xmm3,%xmm11 + movq %rbx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rax,%r13 + xorq %rcx,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r8,%r14 + vpaddq %xmm11,%xmm6,%xmm6 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 +.byte 143,72,120,195,209,7 + xorq %rcx,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,221,3 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm5,%xmm10 + addq %rdx,%r11 + addq %rdi,%rdx + vpaddq %xmm8,%xmm6,%xmm6 + movq %r11,%r13 + addq %rdx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rdx + vpxor %xmm10,%xmm11,%xmm11 + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + vpaddq %xmm11,%xmm6,%xmm6 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + vpaddq 64(%rsi),%xmm6,%xmm10 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,96(%rsp) + vpalignr $8,%xmm7,%xmm0,%xmm8 + rorq $23,%r13 + movq %r14,%rcx + vpalignr $8,%xmm3,%xmm4,%xmm11 + movq %r11,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r10,%r13 + xorq %rax,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rcx,%r14 + vpaddq %xmm11,%xmm7,%xmm7 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 +.byte 143,72,120,195,209,7 + xorq %rax,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,222,3 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm6,%xmm10 + addq %rbx,%r9 + addq %rdi,%rbx + vpaddq %xmm8,%xmm7,%xmm7 + movq %r9,%r13 + addq %rbx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rbx + vpxor %xmm10,%xmm11,%xmm11 + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + vpaddq %xmm11,%xmm7,%xmm7 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + vpaddq 96(%rsi),%xmm7,%xmm10 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,112(%rsp) + cmpb $0,135(%rsi) + jne L$xop_00_47 + rorq $23,%r13 + movq %r14,%rax + movq %r9,%r12 + rorq $5,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + rorq $4,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + rorq $6,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + rorq $28,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + rorq $23,%r13 + movq %r14,%r11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + rorq $23,%r13 + movq %r14,%r10 + movq %rdx,%r12 + rorq $5,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + rorq $4,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + rorq $6,%r14 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + rorq $28,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + rorq $23,%r13 + movq %r14,%r9 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + rorq $23,%r13 + movq %r14,%r8 + movq %rbx,%r12 + rorq $5,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + rorq $4,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + rorq $6,%r14 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + rorq $28,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + rorq $23,%r13 + movq %r14,%rdx + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + rorq $23,%r13 + movq %r14,%rcx + movq %r11,%r12 + rorq $5,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + rorq $4,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + rorq $6,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + rorq $28,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + rorq $23,%r13 + movq %r14,%rbx + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + rorq $23,%r13 + movq %r14,%rax + movq %r9,%r12 + rorq $5,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + rorq $4,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + rorq $6,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + rorq $28,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + rorq $23,%r13 + movq %r14,%r11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + rorq $23,%r13 + movq %r14,%r10 + movq %rdx,%r12 + rorq $5,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + rorq $4,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + rorq $6,%r14 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + rorq $28,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + rorq $23,%r13 + movq %r14,%r9 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + rorq $23,%r13 + movq %r14,%r8 + movq %rbx,%r12 + rorq $5,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + rorq $4,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + rorq $6,%r14 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + rorq $28,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + rorq $23,%r13 + movq %r14,%rdx + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + rorq $23,%r13 + movq %r14,%rcx + movq %r11,%r12 + rorq $5,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + rorq $4,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + rorq $6,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + rorq $28,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + rorq $23,%r13 + movq %r14,%rbx + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + movq -64(%rbp),%rdi + movq %r14,%rax + movq -56(%rbp),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + leaq 128(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb L$loop_xop + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 6 +crypton_sha512_asm_block_data_order_avx: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$avx_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp L$loop_avx +.p2align 4 +L$loop_avx: + vmovdqa K512+1280(%rip),%xmm11 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vpshufb %xmm11,%xmm0,%xmm0 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm11,%xmm1,%xmm1 + vmovdqu 64(%rsi),%xmm4 + vpshufb %xmm11,%xmm2,%xmm2 + vmovdqu 80(%rsi),%xmm5 + vpshufb %xmm11,%xmm3,%xmm3 + vmovdqu 96(%rsi),%xmm6 + vpshufb %xmm11,%xmm4,%xmm4 + vmovdqu 112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vpshufb %xmm11,%xmm5,%xmm5 + vpaddq -128(%rsi),%xmm0,%xmm8 + vpshufb %xmm11,%xmm6,%xmm6 + vpaddq -96(%rsi),%xmm1,%xmm9 + vpshufb %xmm11,%xmm7,%xmm7 + vpaddq -64(%rsi),%xmm2,%xmm10 + vpaddq -32(%rsi),%xmm3,%xmm11 + vmovdqa %xmm8,0(%rsp) + vpaddq 0(%rsi),%xmm4,%xmm8 + vmovdqa %xmm9,16(%rsp) + vpaddq 32(%rsi),%xmm5,%xmm9 + vmovdqa %xmm10,32(%rsp) + vpaddq 64(%rsi),%xmm6,%xmm10 + vmovdqa %xmm11,48(%rsp) + vpaddq 96(%rsi),%xmm7,%xmm11 + vmovdqa %xmm8,64(%rsp) + movq %rax,%r14 + vmovdqa %xmm9,80(%rsp) + movq %rbx,%rdi + vmovdqa %xmm10,96(%rsp) + xorq %rcx,%rdi + vmovdqa %xmm11,112(%rsp) + movq %r8,%r13 + jmp L$avx_00_47 + +.p2align 4 +L$avx_00_47: + addq $256,%rsi + vpalignr $8,%xmm0,%xmm1,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rax + vpalignr $8,%xmm4,%xmm5,%xmm11 + movq %r9,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r8,%r13 + xorq %r10,%r12 + vpaddq %xmm11,%xmm0,%xmm0 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r8,%r12 + xorq %r8,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 0(%rsp),%r11 + movq %rax,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rbx,%r15 + addq %r12,%r11 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm7,%xmm11 + addq %r11,%rdx + addq %rdi,%r11 + vpxor %xmm9,%xmm8,%xmm8 + movq %rdx,%r13 + addq %r11,%r14 + vpsllq $3,%xmm7,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r11 + vpaddq %xmm8,%xmm0,%xmm0 + movq %r8,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm7,%xmm9 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rdx,%r12 + xorq %rdx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 8(%rsp),%r10 + movq %r11,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r9,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rax,%rdi + addq %r12,%r10 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm0,%xmm0 + xorq %r11,%r14 + addq %r13,%r10 + vpaddq -128(%rsi),%xmm0,%xmm10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,0(%rsp) + vpalignr $8,%xmm1,%xmm2,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r10 + vpalignr $8,%xmm5,%xmm6,%xmm11 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rcx,%r13 + xorq %r8,%r12 + vpaddq %xmm11,%xmm1,%xmm1 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rcx,%r12 + xorq %rcx,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 16(%rsp),%r9 + movq %r10,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r11,%r15 + addq %r12,%r9 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm0,%xmm11 + addq %r9,%rbx + addq %rdi,%r9 + vpxor %xmm9,%xmm8,%xmm8 + movq %rbx,%r13 + addq %r9,%r14 + vpsllq $3,%xmm0,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r9 + vpaddq %xmm8,%xmm1,%xmm1 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm0,%xmm9 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rbx,%r12 + xorq %rbx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 24(%rsp),%r8 + movq %r9,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r10,%rdi + addq %r12,%r8 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm1,%xmm1 + xorq %r9,%r14 + addq %r13,%r8 + vpaddq -96(%rsi),%xmm1,%xmm10 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,16(%rsp) + vpalignr $8,%xmm2,%xmm3,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r8 + vpalignr $8,%xmm6,%xmm7,%xmm11 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rax,%r13 + xorq %rcx,%r12 + vpaddq %xmm11,%xmm2,%xmm2 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rax,%r12 + xorq %rax,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 32(%rsp),%rdx + movq %r8,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r9,%r15 + addq %r12,%rdx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm1,%xmm11 + addq %rdx,%r11 + addq %rdi,%rdx + vpxor %xmm9,%xmm8,%xmm8 + movq %r11,%r13 + addq %rdx,%r14 + vpsllq $3,%xmm1,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rdx + vpaddq %xmm8,%xmm2,%xmm2 + movq %rax,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm1,%xmm9 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r11,%r12 + xorq %r11,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 40(%rsp),%rcx + movq %rdx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r8,%rdi + addq %r12,%rcx + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm2,%xmm2 + xorq %rdx,%r14 + addq %r13,%rcx + vpaddq -64(%rsi),%xmm2,%xmm10 + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,32(%rsp) + vpalignr $8,%xmm3,%xmm4,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rcx + vpalignr $8,%xmm7,%xmm0,%xmm11 + movq %r11,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r10,%r13 + xorq %rax,%r12 + vpaddq %xmm11,%xmm3,%xmm3 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r10,%r12 + xorq %r10,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 48(%rsp),%rbx + movq %rcx,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rdx,%r15 + addq %r12,%rbx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm2,%xmm11 + addq %rbx,%r9 + addq %rdi,%rbx + vpxor %xmm9,%xmm8,%xmm8 + movq %r9,%r13 + addq %rbx,%r14 + vpsllq $3,%xmm2,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rbx + vpaddq %xmm8,%xmm3,%xmm3 + movq %r10,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm2,%xmm9 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r9,%r12 + xorq %r9,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 56(%rsp),%rax + movq %rbx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r11,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rcx,%rdi + addq %r12,%rax + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm3,%xmm3 + xorq %rbx,%r14 + addq %r13,%rax + vpaddq -32(%rsi),%xmm3,%xmm10 + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,48(%rsp) + vpalignr $8,%xmm4,%xmm5,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rax + vpalignr $8,%xmm0,%xmm1,%xmm11 + movq %r9,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r8,%r13 + xorq %r10,%r12 + vpaddq %xmm11,%xmm4,%xmm4 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r8,%r12 + xorq %r8,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 64(%rsp),%r11 + movq %rax,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rbx,%r15 + addq %r12,%r11 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm3,%xmm11 + addq %r11,%rdx + addq %rdi,%r11 + vpxor %xmm9,%xmm8,%xmm8 + movq %rdx,%r13 + addq %r11,%r14 + vpsllq $3,%xmm3,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r11 + vpaddq %xmm8,%xmm4,%xmm4 + movq %r8,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm3,%xmm9 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rdx,%r12 + xorq %rdx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 72(%rsp),%r10 + movq %r11,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r9,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rax,%rdi + addq %r12,%r10 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm4,%xmm4 + xorq %r11,%r14 + addq %r13,%r10 + vpaddq 0(%rsi),%xmm4,%xmm10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,64(%rsp) + vpalignr $8,%xmm5,%xmm6,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r10 + vpalignr $8,%xmm1,%xmm2,%xmm11 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rcx,%r13 + xorq %r8,%r12 + vpaddq %xmm11,%xmm5,%xmm5 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rcx,%r12 + xorq %rcx,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 80(%rsp),%r9 + movq %r10,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r11,%r15 + addq %r12,%r9 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm4,%xmm11 + addq %r9,%rbx + addq %rdi,%r9 + vpxor %xmm9,%xmm8,%xmm8 + movq %rbx,%r13 + addq %r9,%r14 + vpsllq $3,%xmm4,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r9 + vpaddq %xmm8,%xmm5,%xmm5 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm4,%xmm9 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rbx,%r12 + xorq %rbx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 88(%rsp),%r8 + movq %r9,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r10,%rdi + addq %r12,%r8 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm5,%xmm5 + xorq %r9,%r14 + addq %r13,%r8 + vpaddq 32(%rsi),%xmm5,%xmm10 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,80(%rsp) + vpalignr $8,%xmm6,%xmm7,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r8 + vpalignr $8,%xmm2,%xmm3,%xmm11 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rax,%r13 + xorq %rcx,%r12 + vpaddq %xmm11,%xmm6,%xmm6 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rax,%r12 + xorq %rax,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 96(%rsp),%rdx + movq %r8,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r9,%r15 + addq %r12,%rdx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm5,%xmm11 + addq %rdx,%r11 + addq %rdi,%rdx + vpxor %xmm9,%xmm8,%xmm8 + movq %r11,%r13 + addq %rdx,%r14 + vpsllq $3,%xmm5,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rdx + vpaddq %xmm8,%xmm6,%xmm6 + movq %rax,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm5,%xmm9 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r11,%r12 + xorq %r11,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 104(%rsp),%rcx + movq %rdx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r8,%rdi + addq %r12,%rcx + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm6,%xmm6 + xorq %rdx,%r14 + addq %r13,%rcx + vpaddq 64(%rsi),%xmm6,%xmm10 + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,96(%rsp) + vpalignr $8,%xmm7,%xmm0,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rcx + vpalignr $8,%xmm3,%xmm4,%xmm11 + movq %r11,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r10,%r13 + xorq %rax,%r12 + vpaddq %xmm11,%xmm7,%xmm7 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r10,%r12 + xorq %r10,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 112(%rsp),%rbx + movq %rcx,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rdx,%r15 + addq %r12,%rbx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm6,%xmm11 + addq %rbx,%r9 + addq %rdi,%rbx + vpxor %xmm9,%xmm8,%xmm8 + movq %r9,%r13 + addq %rbx,%r14 + vpsllq $3,%xmm6,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rbx + vpaddq %xmm8,%xmm7,%xmm7 + movq %r10,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm6,%xmm9 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r9,%r12 + xorq %r9,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 120(%rsp),%rax + movq %rbx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r11,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rcx,%rdi + addq %r12,%rax + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm7,%xmm7 + xorq %rbx,%r14 + addq %r13,%rax + vpaddq 96(%rsi),%xmm7,%xmm10 + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,112(%rsp) + cmpb $0,135(%rsi) + jne L$avx_00_47 + shrdq $23,%r13,%r13 + movq %r14,%rax + movq %r9,%r12 + shrdq $5,%r14,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r11 + movq %r8,%r12 + shrdq $5,%r14,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + shrdq $6,%r14,%r14 + xorq %rax,%rdi + addq %r12,%r10 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r10 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + xorq %r11,%r15 + addq %r12,%r9 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r9 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + xorq %r10,%rdi + addq %r12,%r8 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r8 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + xorq %r9,%r15 + addq %r12,%rdx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + shrdq $28,%r14,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rdx + movq %rax,%r12 + shrdq $5,%r14,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + xorq %r8,%rdi + addq %r12,%rcx + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rcx + movq %r11,%r12 + shrdq $5,%r14,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rbx + movq %r10,%r12 + shrdq $5,%r14,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + shrdq $6,%r14,%r14 + xorq %rcx,%rdi + addq %r12,%rax + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rax + movq %r9,%r12 + shrdq $5,%r14,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r11 + movq %r8,%r12 + shrdq $5,%r14,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + shrdq $6,%r14,%r14 + xorq %rax,%rdi + addq %r12,%r10 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r10 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + xorq %r11,%r15 + addq %r12,%r9 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r9 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + xorq %r10,%rdi + addq %r12,%r8 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r8 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + xorq %r9,%r15 + addq %r12,%rdx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + shrdq $28,%r14,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rdx + movq %rax,%r12 + shrdq $5,%r14,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + xorq %r8,%rdi + addq %r12,%rcx + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rcx + movq %r11,%r12 + shrdq $5,%r14,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rbx + movq %r10,%r12 + shrdq $5,%r14,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + shrdq $6,%r14,%r14 + xorq %rcx,%rdi + addq %r12,%rax + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + movq -64(%rbp),%rdi + movq %r14,%rax + movq -56(%rbp),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + leaq 128(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb L$loop_avx + + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + + +.p2align 6 +crypton_sha512_asm_block_data_order_avx2: +.cfi_startproc + .byte 0xf3,0x0f,0x1e,0xfa + + + pushq %rbp +.cfi_adjust_cfa_offset 8 +.cfi_offset %rbp,-16 + movq %rsp,%rbp +.cfi_def_cfa_register %rbp +L$avx2_shortcut: + pushq %rbx +.cfi_offset %rbx,-24 + pushq %r12 +.cfi_offset %r12,-32 + pushq %r13 +.cfi_offset %r13,-40 + pushq %r14 +.cfi_offset %r14,-48 + pushq %r15 +.cfi_offset %r15,-56 + shlq $4,%rdx + subq $24,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-128,%rsp + subq $-128,%rsi + movq 0(%rdi),%rax + movq %rsi,%r12 + movq 8(%rdi),%rbx + cmpq %rdx,%rsi + movq 16(%rdi),%rcx + cmoveq %rsp,%r12 + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp L$oop_avx2 +.p2align 4 +L$oop_avx2: + vmovdqa K512+1280(%rip),%ymm10 + movq %rsi,-56(%rbp) + vmovdqu -128(%rsi),%xmm0 + vmovdqu -128+16(%rsi),%xmm1 + vmovdqu -128+32(%rsi),%xmm2 + vmovdqu -128+48(%rsi),%xmm3 + vmovdqu -128+64(%rsi),%xmm4 + vmovdqu -128+80(%rsi),%xmm5 + vmovdqu -128+96(%rsi),%xmm6 + vmovdqu -128+112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vinserti128 $1,(%r12),%ymm0,%ymm0 + vinserti128 $1,16(%r12),%ymm1,%ymm1 + vpshufb %ymm10,%ymm0,%ymm0 + vinserti128 $1,32(%r12),%ymm2,%ymm2 + vpshufb %ymm10,%ymm1,%ymm1 + vinserti128 $1,48(%r12),%ymm3,%ymm3 + vpshufb %ymm10,%ymm2,%ymm2 + vinserti128 $1,64(%r12),%ymm4,%ymm4 + vpshufb %ymm10,%ymm3,%ymm3 + vinserti128 $1,80(%r12),%ymm5,%ymm5 + vpshufb %ymm10,%ymm4,%ymm4 + vinserti128 $1,96(%r12),%ymm6,%ymm6 + vpshufb %ymm10,%ymm5,%ymm5 + vinserti128 $1,112(%r12),%ymm7,%ymm7 + + vpaddq -128(%rsi),%ymm0,%ymm8 + vpshufb %ymm10,%ymm6,%ymm6 + vpaddq -96(%rsi),%ymm1,%ymm9 + vpshufb %ymm10,%ymm7,%ymm7 + vpaddq -64(%rsi),%ymm2,%ymm10 + vpaddq -32(%rsi),%ymm3,%ymm11 + vmovdqa %ymm8,0(%rsp) + vpaddq 0(%rsi),%ymm4,%ymm8 + vmovdqa %ymm9,32(%rsp) + vpaddq 32(%rsi),%ymm5,%ymm9 + vmovdqa %ymm10,64(%rsp) + vpaddq 64(%rsi),%ymm6,%ymm10 + vmovdqa %ymm11,96(%rsp) + leaq -128(%rsp),%rsp + vpaddq 96(%rsi),%ymm7,%ymm11 + vmovdqa %ymm8,0(%rsp) + xorq %r14,%r14 + vmovdqa %ymm9,32(%rsp) + movq %rbx,%rdi + vmovdqa %ymm10,64(%rsp) + xorq %rcx,%rdi + vmovdqa %ymm11,96(%rsp) + movq %r9,%r12 + addq $32*8,%rsi + jmp L$avx2_00_47 + +.p2align 4 +L$avx2_00_47: + leaq -128(%rsp),%rsp + vpalignr $8,%ymm0,%ymm1,%ymm8 + addq 0+256(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + vpalignr $8,%ymm4,%ymm5,%ymm11 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + vpsrlq $1,%ymm8,%ymm10 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + vpaddq %ymm11,%ymm0,%ymm0 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + vpsrlq $6,%ymm7,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + vpsllq $3,%ymm7,%ymm10 + vpaddq %ymm8,%ymm0,%ymm0 + addq 8+256(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + vpsrlq $19,%ymm7,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + vpaddq %ymm11,%ymm0,%ymm0 + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + vpaddq -128(%rsi),%ymm0,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + vmovdqa %ymm10,0(%rsp) + vpalignr $8,%ymm1,%ymm2,%ymm8 + addq 32+256(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + vpalignr $8,%ymm5,%ymm6,%ymm11 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + vpsrlq $1,%ymm8,%ymm10 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + vpaddq %ymm11,%ymm1,%ymm1 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + vpsrlq $6,%ymm0,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + vpsllq $3,%ymm0,%ymm10 + vpaddq %ymm8,%ymm1,%ymm1 + addq 40+256(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + vpsrlq $19,%ymm0,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + vpaddq %ymm11,%ymm1,%ymm1 + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + vpaddq -96(%rsi),%ymm1,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + vmovdqa %ymm10,32(%rsp) + vpalignr $8,%ymm2,%ymm3,%ymm8 + addq 64+256(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + vpalignr $8,%ymm6,%ymm7,%ymm11 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + vpsrlq $1,%ymm8,%ymm10 + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + vpaddq %ymm11,%ymm2,%ymm2 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + vpsrlq $6,%ymm1,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + vpsllq $3,%ymm1,%ymm10 + vpaddq %ymm8,%ymm2,%ymm2 + addq 72+256(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + vpsrlq $19,%ymm1,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + vpaddq %ymm11,%ymm2,%ymm2 + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + vpaddq -64(%rsi),%ymm2,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + vmovdqa %ymm10,64(%rsp) + vpalignr $8,%ymm3,%ymm4,%ymm8 + addq 96+256(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + vpalignr $8,%ymm7,%ymm0,%ymm11 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + vpsrlq $1,%ymm8,%ymm10 + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + vpaddq %ymm11,%ymm3,%ymm3 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + vpsrlq $6,%ymm2,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + vpsllq $3,%ymm2,%ymm10 + vpaddq %ymm8,%ymm3,%ymm3 + addq 104+256(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + vpsrlq $19,%ymm2,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + vpaddq %ymm11,%ymm3,%ymm3 + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + vpaddq -32(%rsi),%ymm3,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + vmovdqa %ymm10,96(%rsp) + leaq -128(%rsp),%rsp + vpalignr $8,%ymm4,%ymm5,%ymm8 + addq 0+256(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + vpalignr $8,%ymm0,%ymm1,%ymm11 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + vpsrlq $1,%ymm8,%ymm10 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + vpaddq %ymm11,%ymm4,%ymm4 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + vpsrlq $6,%ymm3,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + vpsllq $3,%ymm3,%ymm10 + vpaddq %ymm8,%ymm4,%ymm4 + addq 8+256(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + vpsrlq $19,%ymm3,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + vpaddq %ymm11,%ymm4,%ymm4 + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + vpaddq 0(%rsi),%ymm4,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + vmovdqa %ymm10,0(%rsp) + vpalignr $8,%ymm5,%ymm6,%ymm8 + addq 32+256(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + vpalignr $8,%ymm1,%ymm2,%ymm11 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + vpsrlq $1,%ymm8,%ymm10 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + vpaddq %ymm11,%ymm5,%ymm5 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + vpsrlq $6,%ymm4,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + vpsllq $3,%ymm4,%ymm10 + vpaddq %ymm8,%ymm5,%ymm5 + addq 40+256(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + vpsrlq $19,%ymm4,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + vpaddq %ymm11,%ymm5,%ymm5 + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + vpaddq 32(%rsi),%ymm5,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + vmovdqa %ymm10,32(%rsp) + vpalignr $8,%ymm6,%ymm7,%ymm8 + addq 64+256(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + vpalignr $8,%ymm2,%ymm3,%ymm11 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + vpsrlq $1,%ymm8,%ymm10 + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + vpaddq %ymm11,%ymm6,%ymm6 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + vpsrlq $6,%ymm5,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + vpsllq $3,%ymm5,%ymm10 + vpaddq %ymm8,%ymm6,%ymm6 + addq 72+256(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + vpsrlq $19,%ymm5,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + vpaddq %ymm11,%ymm6,%ymm6 + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + vpaddq 64(%rsi),%ymm6,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + vmovdqa %ymm10,64(%rsp) + vpalignr $8,%ymm7,%ymm0,%ymm8 + addq 96+256(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + vpalignr $8,%ymm3,%ymm4,%ymm11 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + vpsrlq $1,%ymm8,%ymm10 + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + vpaddq %ymm11,%ymm7,%ymm7 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + vpsrlq $6,%ymm6,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + vpsllq $3,%ymm6,%ymm10 + vpaddq %ymm8,%ymm7,%ymm7 + addq 104+256(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + vpsrlq $19,%ymm6,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + vpaddq %ymm11,%ymm7,%ymm7 + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + vpaddq 96(%rsi),%ymm7,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + vmovdqa %ymm10,96(%rsp) + leaq 256(%rsi),%rsi + cmpb $0,-121(%rsi) + jne L$avx2_00_47 + addq 0+128(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8+128(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32+128(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40+128(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64+128(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72+128(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96+128(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104+128(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + addq 0(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + movq -64(%rbp),%rdi + addq %r14,%rax + movq -56(%rbp),%r12 + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + + cmpq -48(%rbp),%r12 + je L$done_avx2 + + leaq 1152(%rsp),%rsi + xorq %r14,%r14 + movq %rbx,%rdi + xorq %rcx,%rdi + movq %r9,%r12 + jmp L$ower_avx2 +.p2align 4 +L$ower_avx2: + addq 0+16(%rsi),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8+16(%rsi),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32+16(%rsi),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40+16(%rsi),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64+16(%rsi),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72+16(%rsi),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96+16(%rsi),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104+16(%rsi),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + leaq -128(%rsi),%rsi + cmpq %rsp,%rsi + jae L$ower_avx2 + + movq -64(%rbp),%rdi + addq %r14,%rax + movq -56(%rbp),%rsi + leaq 1152(%rsp),%rsp + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + leaq 256(%rsi),%rsi + addq 48(%rdi),%r10 + movq %rsi,%r12 + addq 56(%rdi),%r11 + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + cmoveq %rsp,%r12 + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + + jbe L$oop_avx2 + +L$done_avx2: + vzeroupper + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp +.cfi_def_cfa_register %rsp + popq %rbp +.cfi_adjust_cfa_offset -8 +.cfi_restore %rbp +.cfi_restore %r12 +.cfi_restore %r13 +.cfi_restore %r14 +.cfi_restore %r15 +.cfi_restore %rbx + .byte 0xf3,0xc3 +.cfi_endproc + diff --git a/cbits/asm/sha512-x86_64-mingw64.S b/cbits/asm/sha512-x86_64-mingw64.S new file mode 100644 --- /dev/null +++ b/cbits/asm/sha512-x86_64-mingw64.S @@ -0,0 +1,6016 @@ +.text + + +.globl crypton_sha512_asm_block_data_order +.def crypton_sha512_asm_block_data_order; .scl 2; .type 32; .endef +.p2align 4 +crypton_sha512_asm_block_data_order: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha512_asm_block_data_order: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx + leaq crypton_ia32cap_P(%rip),%rax + movl 0(%rax),%r9d + movl 4(%rax),%r10d + movl 8(%rax),%eax + testl $2048,%r10d + jnz .Lxop_shortcut + andl $296,%eax + cmpl $296,%eax + je .Lavx2_shortcut + andl $1073741824,%r9d + andl $268435968,%r10d + orl %r9d,%r10d + cmpl $1342177792,%r10d + je .Lavx_shortcut + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $128+24,%rsp + + +.LSEH_body_crypton_sha512_asm_block_data_order: + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,128+0(%rsp) + movq %rsi,128+8(%rsp) + movq %rdx,128+16(%rsp) + + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Lloop + +.p2align 4 +.Lloop: + movq %rbx,%rdi + leaq K512(%rip),%rbp + xorq %rcx,%rdi + movq 0(%rsi),%r12 + movq %r8,%r13 + movq %rax,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,0(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + addq %r14,%r11 + movq 8(%rsi),%r12 + movq %rdx,%r13 + movq %r11,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,8(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + addq %r14,%r10 + movq 16(%rsi),%r12 + movq %rcx,%r13 + movq %r10,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,16(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + addq %r14,%r9 + movq 24(%rsi),%r12 + movq %rbx,%r13 + movq %r9,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,24(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + addq %r14,%r8 + movq 32(%rsi),%r12 + movq %rax,%r13 + movq %r8,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,32(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + addq %r14,%rdx + movq 40(%rsi),%r12 + movq %r11,%r13 + movq %rdx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,40(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + addq %r14,%rcx + movq 48(%rsi),%r12 + movq %r10,%r13 + movq %rcx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,48(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + addq %r14,%rbx + movq 56(%rsi),%r12 + movq %r9,%r13 + movq %rbx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,56(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + addq %r14,%rax + movq 64(%rsi),%r12 + movq %r8,%r13 + movq %rax,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,64(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + addq %r14,%r11 + movq 72(%rsi),%r12 + movq %rdx,%r13 + movq %r11,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,72(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + addq %r14,%r10 + movq 80(%rsi),%r12 + movq %rcx,%r13 + movq %r10,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,80(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + addq %r14,%r9 + movq 88(%rsi),%r12 + movq %rbx,%r13 + movq %r9,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,88(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + addq %r14,%r8 + movq 96(%rsi),%r12 + movq %rax,%r13 + movq %r8,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,96(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + addq %r14,%rdx + movq 104(%rsi),%r12 + movq %r11,%r13 + movq %rdx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,104(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + addq %r14,%rcx + movq 112(%rsi),%r12 + movq %r10,%r13 + movq %rcx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,112(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + addq %r14,%rbx + movq 120(%rsi),%r12 + movq %r9,%r13 + movq %rbx,%r14 + bswapq %r12 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,120(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + jmp .Lrounds_16_xx +.p2align 4 +.Lrounds_16_xx: + movq 8(%rsp),%r13 + movq 112(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rax + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 72(%rsp),%r12 + + addq 0(%rsp),%r12 + movq %r8,%r13 + addq %r15,%r12 + movq %rax,%r14 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,0(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + movq 16(%rsp),%r13 + movq 120(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r11 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 80(%rsp),%r12 + + addq 8(%rsp),%r12 + movq %rdx,%r13 + addq %rdi,%r12 + movq %r11,%r14 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,8(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + movq 24(%rsp),%r13 + movq 0(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r10 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 88(%rsp),%r12 + + addq 16(%rsp),%r12 + movq %rcx,%r13 + addq %r15,%r12 + movq %r10,%r14 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,16(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + movq 32(%rsp),%r13 + movq 8(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r9 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 96(%rsp),%r12 + + addq 24(%rsp),%r12 + movq %rbx,%r13 + addq %rdi,%r12 + movq %r9,%r14 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,24(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + movq 40(%rsp),%r13 + movq 16(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r8 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 104(%rsp),%r12 + + addq 32(%rsp),%r12 + movq %rax,%r13 + addq %r15,%r12 + movq %r8,%r14 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,32(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + movq 48(%rsp),%r13 + movq 24(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rdx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 112(%rsp),%r12 + + addq 40(%rsp),%r12 + movq %r11,%r13 + addq %rdi,%r12 + movq %rdx,%r14 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,40(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + movq 56(%rsp),%r13 + movq 32(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rcx + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 120(%rsp),%r12 + + addq 48(%rsp),%r12 + movq %r10,%r13 + addq %r15,%r12 + movq %rcx,%r14 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,48(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + movq 64(%rsp),%r13 + movq 40(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rbx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 0(%rsp),%r12 + + addq 56(%rsp),%r12 + movq %r9,%r13 + addq %rdi,%r12 + movq %rbx,%r14 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,56(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + movq 72(%rsp),%r13 + movq 48(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rax + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 8(%rsp),%r12 + + addq 64(%rsp),%r12 + movq %r8,%r13 + addq %r15,%r12 + movq %rax,%r14 + rorq $23,%r13 + movq %r9,%r15 + + xorq %r8,%r13 + rorq $5,%r14 + xorq %r10,%r15 + + movq %r12,64(%rsp) + xorq %rax,%r14 + andq %r8,%r15 + + rorq $4,%r13 + addq %r11,%r12 + xorq %r10,%r15 + + rorq $6,%r14 + xorq %r8,%r13 + addq %r15,%r12 + + movq %rax,%r15 + addq (%rbp),%r12 + xorq %rax,%r14 + + xorq %rbx,%r15 + rorq $14,%r13 + movq %rbx,%r11 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r11 + addq %r12,%rdx + addq %r12,%r11 + + leaq 8(%rbp),%rbp + movq 80(%rsp),%r13 + movq 56(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r11 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 16(%rsp),%r12 + + addq 72(%rsp),%r12 + movq %rdx,%r13 + addq %rdi,%r12 + movq %r11,%r14 + rorq $23,%r13 + movq %r8,%rdi + + xorq %rdx,%r13 + rorq $5,%r14 + xorq %r9,%rdi + + movq %r12,72(%rsp) + xorq %r11,%r14 + andq %rdx,%rdi + + rorq $4,%r13 + addq %r10,%r12 + xorq %r9,%rdi + + rorq $6,%r14 + xorq %rdx,%r13 + addq %rdi,%r12 + + movq %r11,%rdi + addq (%rbp),%r12 + xorq %r11,%r14 + + xorq %rax,%rdi + rorq $14,%r13 + movq %rax,%r10 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r10 + addq %r12,%rcx + addq %r12,%r10 + + leaq 24(%rbp),%rbp + movq 88(%rsp),%r13 + movq 64(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r10 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 24(%rsp),%r12 + + addq 80(%rsp),%r12 + movq %rcx,%r13 + addq %r15,%r12 + movq %r10,%r14 + rorq $23,%r13 + movq %rdx,%r15 + + xorq %rcx,%r13 + rorq $5,%r14 + xorq %r8,%r15 + + movq %r12,80(%rsp) + xorq %r10,%r14 + andq %rcx,%r15 + + rorq $4,%r13 + addq %r9,%r12 + xorq %r8,%r15 + + rorq $6,%r14 + xorq %rcx,%r13 + addq %r15,%r12 + + movq %r10,%r15 + addq (%rbp),%r12 + xorq %r10,%r14 + + xorq %r11,%r15 + rorq $14,%r13 + movq %r11,%r9 + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%r9 + addq %r12,%rbx + addq %r12,%r9 + + leaq 8(%rbp),%rbp + movq 96(%rsp),%r13 + movq 72(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r9 + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 32(%rsp),%r12 + + addq 88(%rsp),%r12 + movq %rbx,%r13 + addq %rdi,%r12 + movq %r9,%r14 + rorq $23,%r13 + movq %rcx,%rdi + + xorq %rbx,%r13 + rorq $5,%r14 + xorq %rdx,%rdi + + movq %r12,88(%rsp) + xorq %r9,%r14 + andq %rbx,%rdi + + rorq $4,%r13 + addq %r8,%r12 + xorq %rdx,%rdi + + rorq $6,%r14 + xorq %rbx,%r13 + addq %rdi,%r12 + + movq %r9,%rdi + addq (%rbp),%r12 + xorq %r9,%r14 + + xorq %r10,%rdi + rorq $14,%r13 + movq %r10,%r8 + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%r8 + addq %r12,%rax + addq %r12,%r8 + + leaq 24(%rbp),%rbp + movq 104(%rsp),%r13 + movq 80(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%r8 + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 40(%rsp),%r12 + + addq 96(%rsp),%r12 + movq %rax,%r13 + addq %r15,%r12 + movq %r8,%r14 + rorq $23,%r13 + movq %rbx,%r15 + + xorq %rax,%r13 + rorq $5,%r14 + xorq %rcx,%r15 + + movq %r12,96(%rsp) + xorq %r8,%r14 + andq %rax,%r15 + + rorq $4,%r13 + addq %rdx,%r12 + xorq %rcx,%r15 + + rorq $6,%r14 + xorq %rax,%r13 + addq %r15,%r12 + + movq %r8,%r15 + addq (%rbp),%r12 + xorq %r8,%r14 + + xorq %r9,%r15 + rorq $14,%r13 + movq %r9,%rdx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rdx + addq %r12,%r11 + addq %r12,%rdx + + leaq 8(%rbp),%rbp + movq 112(%rsp),%r13 + movq 88(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rdx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 48(%rsp),%r12 + + addq 104(%rsp),%r12 + movq %r11,%r13 + addq %rdi,%r12 + movq %rdx,%r14 + rorq $23,%r13 + movq %rax,%rdi + + xorq %r11,%r13 + rorq $5,%r14 + xorq %rbx,%rdi + + movq %r12,104(%rsp) + xorq %rdx,%r14 + andq %r11,%rdi + + rorq $4,%r13 + addq %rcx,%r12 + xorq %rbx,%rdi + + rorq $6,%r14 + xorq %r11,%r13 + addq %rdi,%r12 + + movq %rdx,%rdi + addq (%rbp),%r12 + xorq %rdx,%r14 + + xorq %r8,%rdi + rorq $14,%r13 + movq %r8,%rcx + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rcx + addq %r12,%r10 + addq %r12,%rcx + + leaq 24(%rbp),%rbp + movq 120(%rsp),%r13 + movq 96(%rsp),%r15 + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rcx + movq %r15,%r14 + rorq $42,%r15 + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%r15 + shrq $6,%r14 + + rorq $19,%r15 + xorq %r13,%r12 + xorq %r14,%r15 + addq 56(%rsp),%r12 + + addq 112(%rsp),%r12 + movq %r10,%r13 + addq %r15,%r12 + movq %rcx,%r14 + rorq $23,%r13 + movq %r11,%r15 + + xorq %r10,%r13 + rorq $5,%r14 + xorq %rax,%r15 + + movq %r12,112(%rsp) + xorq %rcx,%r14 + andq %r10,%r15 + + rorq $4,%r13 + addq %rbx,%r12 + xorq %rax,%r15 + + rorq $6,%r14 + xorq %r10,%r13 + addq %r15,%r12 + + movq %rcx,%r15 + addq (%rbp),%r12 + xorq %rcx,%r14 + + xorq %rdx,%r15 + rorq $14,%r13 + movq %rdx,%rbx + + andq %r15,%rdi + rorq $28,%r14 + addq %r13,%r12 + + xorq %rdi,%rbx + addq %r12,%r9 + addq %r12,%rbx + + leaq 8(%rbp),%rbp + movq 0(%rsp),%r13 + movq 104(%rsp),%rdi + + movq %r13,%r12 + rorq $7,%r13 + addq %r14,%rbx + movq %rdi,%r14 + rorq $42,%rdi + + xorq %r12,%r13 + shrq $7,%r12 + rorq $1,%r13 + xorq %r14,%rdi + shrq $6,%r14 + + rorq $19,%rdi + xorq %r13,%r12 + xorq %r14,%rdi + addq 64(%rsp),%r12 + + addq 120(%rsp),%r12 + movq %r9,%r13 + addq %rdi,%r12 + movq %rbx,%r14 + rorq $23,%r13 + movq %r10,%rdi + + xorq %r9,%r13 + rorq $5,%r14 + xorq %r11,%rdi + + movq %r12,120(%rsp) + xorq %rbx,%r14 + andq %r9,%rdi + + rorq $4,%r13 + addq %rax,%r12 + xorq %r11,%rdi + + rorq $6,%r14 + xorq %r9,%r13 + addq %rdi,%r12 + + movq %rbx,%rdi + addq (%rbp),%r12 + xorq %rbx,%r14 + + xorq %rcx,%rdi + rorq $14,%r13 + movq %rcx,%rax + + andq %rdi,%r15 + rorq $28,%r14 + addq %r13,%r12 + + xorq %r15,%rax + addq %r12,%r8 + addq %r12,%rax + + leaq 24(%rbp),%rbp + cmpb $0,7(%rbp) + jnz .Lrounds_16_xx + + movq 128+0(%rsp),%rdi + addq %r14,%rax + leaq 128(%rsi),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + cmpq 128+16(%rsp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb .Lloop + + leaq 128+24+48(%rsp),%r11 + + movq 128+24(%rsp),%r15 + movq -40(%r11),%r14 + movq -32(%r11),%r13 + movq -24(%r11),%r12 + movq -16(%r11),%rbx + movq -8(%r11),%rbp +.LSEH_epilogue_crypton_sha512_asm_block_data_order: + mov 8(%r11),%rdi + mov 16(%r11),%rsi + + leaq (%r11),%rsp + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha512_asm_block_data_order: +.p2align 6 + +K512: +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + +.quad 0x0001020304050607,0x08090a0b0c0d0e0f +.quad 0x0001020304050607,0x08090a0b0c0d0e0f + +K512_nodup: +.quad 0x428a2f98d728ae22,0x7137449123ef65cd +.quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc +.quad 0x3956c25bf348b538,0x59f111f1b605d019 +.quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 +.quad 0xd807aa98a3030242,0x12835b0145706fbe +.quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 +.quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 +.quad 0x9bdc06a725c71235,0xc19bf174cf692694 +.quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 +.quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 +.quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 +.quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 +.quad 0x983e5152ee66dfab,0xa831c66d2db43210 +.quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 +.quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 +.quad 0x06ca6351e003826f,0x142929670a0e6e70 +.quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 +.quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df +.quad 0x650a73548baf63de,0x766a0abb3c77b2a8 +.quad 0x81c2c92e47edaee6,0x92722c851482353b +.quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 +.quad 0xc24b8b70d0f89791,0xc76c51a30654be30 +.quad 0xd192e819d6ef5218,0xd69906245565a910 +.quad 0xf40e35855771202a,0x106aa07032bbd1b8 +.quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 +.quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 +.quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb +.quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 +.quad 0x748f82ee5defb2fc,0x78a5636f43172f60 +.quad 0x84c87814a1f0ab72,0x8cc702081a6439ec +.quad 0x90befffa23631e28,0xa4506cebde82bde9 +.quad 0xbef9a3f7b2c67915,0xc67178f2e372532b +.quad 0xca273eceea26619c,0xd186b8c721c0c207 +.quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 +.quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 +.quad 0x113f9804bef90dae,0x1b710b35131c471b +.quad 0x28db77f523047d84,0x32caab7b40c72493 +.quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c +.quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a +.quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + +.byte 83,72,65,53,49,50,32,98,108,111,99,107,32,116,114,97,110,115,102,111,114,109,32,102,111,114,32,120,56,54,95,54,52,44,32,67,82,89,80,84,79,71,65,77,83,32,98,121,32,64,100,111,116,45,97,115,109,0 +.globl crypton_sha512_asm_block_data_order_shaext +.def crypton_sha512_asm_block_data_order_shaext; .scl 2; .type 32; .endef +.p2align 6 +crypton_sha512_asm_block_data_order_shaext: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha512_asm_block_data_order_shaext: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lshaext_shortcut: + subq $0x50,%rsp + + movaps %xmm6,-80(%rbp) + movaps %xmm7,-64(%rbp) + movaps %xmm8,-48(%rbp) + movaps %xmm9,-32(%rbp) + movaps %xmm10,-16(%rbp) + +.LSEH_body_crypton_sha512_asm_block_data_order_shaext: + + leaq K512_nodup+128(%rip),%rcx + vmovdqu (%rdi),%ymm0 + vmovdqu 32(%rdi),%ymm1 + vmovdqa -160(%rcx),%ymm8 + + vpermq $27,%ymm0,%ymm0 + vpblendd $15,%ymm1,%ymm0,%ymm5 + vpblendd $15,%ymm0,%ymm1,%ymm6 + vpermq $225,%ymm5,%ymm5 + vpermq $75,%ymm6,%ymm6 + jmp .Loop_shaext + +.p2align 4 +.Loop_shaext: + vmovdqu (%rsi),%ymm0 + vmovdqu 32(%rsi),%ymm1 + vmovdqu 64(%rsi),%ymm2 + vpshufb %ymm8,%ymm0,%ymm0 + vmovdqu 96(%rsi),%ymm3 + + vpaddq 0-128(%rcx),%ymm0,%ymm4 + vpshufb %ymm8,%ymm1,%ymm1 + vmovdqa %ymm6,%ymm10 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vmovdqa %ymm5,%ymm9 +.byte 196,226,79,203,236 + + vpaddq 32-128(%rcx),%ymm1,%ymm4 + vpshufb %ymm8,%ymm2,%ymm2 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + leaq 128(%rsi),%rsi +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + + vpaddq 64-128(%rcx),%ymm2,%ymm4 + vpshufb %ymm8,%ymm3,%ymm3 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + + vpaddq 96-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 128-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 160-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 192-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 224-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 256-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 288-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 320-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 352-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 384-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 416-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm3,%ymm3 +.byte 196,226,127,204,193 +.byte 196,226,79,203,236 + vpaddq 448-128(%rcx),%ymm2,%ymm4 +.byte 196,226,127,205,218 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm3,%ymm2,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm0,%ymm0 +.byte 196,226,127,204,202 +.byte 196,226,79,203,236 + vpaddq 480-128(%rcx),%ymm3,%ymm4 +.byte 196,226,127,205,195 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm0,%ymm3,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm1,%ymm1 +.byte 196,226,127,204,211 +.byte 196,226,79,203,236 + vpaddq 512-128(%rcx),%ymm0,%ymm4 +.byte 196,226,127,205,200 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm1,%ymm0,%ymm7 + vpermq $0x39,%ymm7,%ymm7 + vpaddq %ymm7,%ymm2,%ymm2 +.byte 196,226,127,204,216 +.byte 196,226,79,203,236 + vpaddq 544-128(%rcx),%ymm1,%ymm4 +.byte 196,226,127,205,209 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + vpblendd $0x03,%ymm2,%ymm1,%ymm7 + vpermq $0x39,%ymm7,%ymm7 +.byte 196,226,79,203,236 + vpaddq %ymm7,%ymm3,%ymm3 + + vpaddq 576-128(%rcx),%ymm2,%ymm4 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 +.byte 196,226,127,205,218 +.byte 196,226,79,203,236 + + vpaddq 608-128(%rcx),%ymm3,%ymm4 +.byte 196,226,87,203,244 + vextracti128 $1,%ymm4,%xmm4 + decq %rdx +.byte 196,226,79,203,236 + + vpaddq %ymm10,%ymm6,%ymm6 + vpaddq %ymm9,%ymm5,%ymm5 + jnz .Loop_shaext + + vpermq $75,%ymm5,%ymm5 + vpblendd $240,%ymm6,%ymm5,%ymm1 + vpblendd $240,%ymm5,%ymm6,%ymm2 + vpermq $180,%ymm1,%ymm1 + vpermq $27,%ymm2,%ymm2 + + vmovdqu %ymm1,(%rdi) + vmovdqu %ymm2,32(%rdi) + + vzeroupper + movaps -80(%rbp),%xmm6 + movaps -64(%rbp),%xmm7 + movaps -48(%rbp),%xmm8 + movaps -32(%rbp),%xmm9 + movaps -16(%rbp),%xmm10 + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha512_asm_block_data_order_shaext: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha512_asm_block_data_order_shaext: +.def crypton_sha512_asm_block_data_order_xop; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha512_asm_block_data_order_xop: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha512_asm_block_data_order_xop: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lxop_shortcut: + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $120,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + movaps %xmm6,-160(%rbp) + movaps %xmm7,-144(%rbp) + movaps %xmm8,-128(%rbp) + movaps %xmm9,-112(%rbp) + + movaps %xmm10,-96(%rbp) + movaps %xmm11,-80(%rbp) + +.LSEH_body_crypton_sha512_asm_block_data_order_xop: + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Lloop_xop +.p2align 4 +.Lloop_xop: + vmovdqa K512+1280(%rip),%xmm11 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vpshufb %xmm11,%xmm0,%xmm0 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm11,%xmm1,%xmm1 + vmovdqu 64(%rsi),%xmm4 + vpshufb %xmm11,%xmm2,%xmm2 + vmovdqu 80(%rsi),%xmm5 + vpshufb %xmm11,%xmm3,%xmm3 + vmovdqu 96(%rsi),%xmm6 + vpshufb %xmm11,%xmm4,%xmm4 + vmovdqu 112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vpshufb %xmm11,%xmm5,%xmm5 + vpaddq -128(%rsi),%xmm0,%xmm8 + vpshufb %xmm11,%xmm6,%xmm6 + vpaddq -96(%rsi),%xmm1,%xmm9 + vpshufb %xmm11,%xmm7,%xmm7 + vpaddq -64(%rsi),%xmm2,%xmm10 + vpaddq -32(%rsi),%xmm3,%xmm11 + vmovdqa %xmm8,0(%rsp) + vpaddq 0(%rsi),%xmm4,%xmm8 + vmovdqa %xmm9,16(%rsp) + vpaddq 32(%rsi),%xmm5,%xmm9 + vmovdqa %xmm10,32(%rsp) + vpaddq 64(%rsi),%xmm6,%xmm10 + vmovdqa %xmm11,48(%rsp) + vpaddq 96(%rsi),%xmm7,%xmm11 + vmovdqa %xmm8,64(%rsp) + movq %rax,%r14 + vmovdqa %xmm9,80(%rsp) + movq %rbx,%rdi + vmovdqa %xmm10,96(%rsp) + xorq %rcx,%rdi + vmovdqa %xmm11,112(%rsp) + movq %r8,%r13 + jmp .Lxop_00_47 + +.p2align 4 +.Lxop_00_47: + addq $256,%rsi + vpalignr $8,%xmm0,%xmm1,%xmm8 + rorq $23,%r13 + movq %r14,%rax + vpalignr $8,%xmm4,%xmm5,%xmm11 + movq %r9,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r8,%r13 + xorq %r10,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rax,%r14 + vpaddq %xmm11,%xmm0,%xmm0 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 +.byte 143,72,120,195,209,7 + xorq %r10,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,223,3 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm7,%xmm10 + addq %r11,%rdx + addq %rdi,%r11 + vpaddq %xmm8,%xmm0,%xmm0 + movq %rdx,%r13 + addq %r11,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r11 + vpxor %xmm10,%xmm11,%xmm11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + vpaddq %xmm11,%xmm0,%xmm0 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + vpaddq -128(%rsi),%xmm0,%xmm10 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,0(%rsp) + vpalignr $8,%xmm1,%xmm2,%xmm8 + rorq $23,%r13 + movq %r14,%r10 + vpalignr $8,%xmm5,%xmm6,%xmm11 + movq %rdx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rcx,%r13 + xorq %r8,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r10,%r14 + vpaddq %xmm11,%xmm1,%xmm1 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 +.byte 143,72,120,195,209,7 + xorq %r8,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,216,3 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm0,%xmm10 + addq %r9,%rbx + addq %rdi,%r9 + vpaddq %xmm8,%xmm1,%xmm1 + movq %rbx,%r13 + addq %r9,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r9 + vpxor %xmm10,%xmm11,%xmm11 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + vpaddq %xmm11,%xmm1,%xmm1 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + vpaddq -96(%rsi),%xmm1,%xmm10 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,16(%rsp) + vpalignr $8,%xmm2,%xmm3,%xmm8 + rorq $23,%r13 + movq %r14,%r8 + vpalignr $8,%xmm6,%xmm7,%xmm11 + movq %rbx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rax,%r13 + xorq %rcx,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r8,%r14 + vpaddq %xmm11,%xmm2,%xmm2 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 +.byte 143,72,120,195,209,7 + xorq %rcx,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,217,3 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm1,%xmm10 + addq %rdx,%r11 + addq %rdi,%rdx + vpaddq %xmm8,%xmm2,%xmm2 + movq %r11,%r13 + addq %rdx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rdx + vpxor %xmm10,%xmm11,%xmm11 + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + vpaddq %xmm11,%xmm2,%xmm2 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + vpaddq -64(%rsi),%xmm2,%xmm10 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,32(%rsp) + vpalignr $8,%xmm3,%xmm4,%xmm8 + rorq $23,%r13 + movq %r14,%rcx + vpalignr $8,%xmm7,%xmm0,%xmm11 + movq %r11,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r10,%r13 + xorq %rax,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rcx,%r14 + vpaddq %xmm11,%xmm3,%xmm3 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 +.byte 143,72,120,195,209,7 + xorq %rax,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,218,3 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm2,%xmm10 + addq %rbx,%r9 + addq %rdi,%rbx + vpaddq %xmm8,%xmm3,%xmm3 + movq %r9,%r13 + addq %rbx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rbx + vpxor %xmm10,%xmm11,%xmm11 + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + vpaddq %xmm11,%xmm3,%xmm3 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + vpaddq -32(%rsi),%xmm3,%xmm10 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,48(%rsp) + vpalignr $8,%xmm4,%xmm5,%xmm8 + rorq $23,%r13 + movq %r14,%rax + vpalignr $8,%xmm0,%xmm1,%xmm11 + movq %r9,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r8,%r13 + xorq %r10,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rax,%r14 + vpaddq %xmm11,%xmm4,%xmm4 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 +.byte 143,72,120,195,209,7 + xorq %r10,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,219,3 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm3,%xmm10 + addq %r11,%rdx + addq %rdi,%r11 + vpaddq %xmm8,%xmm4,%xmm4 + movq %rdx,%r13 + addq %r11,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r11 + vpxor %xmm10,%xmm11,%xmm11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + vpaddq %xmm11,%xmm4,%xmm4 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + vpaddq 0(%rsi),%xmm4,%xmm10 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,64(%rsp) + vpalignr $8,%xmm5,%xmm6,%xmm8 + rorq $23,%r13 + movq %r14,%r10 + vpalignr $8,%xmm1,%xmm2,%xmm11 + movq %rdx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rcx,%r13 + xorq %r8,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r10,%r14 + vpaddq %xmm11,%xmm5,%xmm5 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 +.byte 143,72,120,195,209,7 + xorq %r8,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,220,3 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm4,%xmm10 + addq %r9,%rbx + addq %rdi,%r9 + vpaddq %xmm8,%xmm5,%xmm5 + movq %rbx,%r13 + addq %r9,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%r9 + vpxor %xmm10,%xmm11,%xmm11 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + vpaddq %xmm11,%xmm5,%xmm5 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + vpaddq 32(%rsi),%xmm5,%xmm10 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,80(%rsp) + vpalignr $8,%xmm6,%xmm7,%xmm8 + rorq $23,%r13 + movq %r14,%r8 + vpalignr $8,%xmm2,%xmm3,%xmm11 + movq %rbx,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %rax,%r13 + xorq %rcx,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %r8,%r14 + vpaddq %xmm11,%xmm6,%xmm6 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 +.byte 143,72,120,195,209,7 + xorq %rcx,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,221,3 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm5,%xmm10 + addq %rdx,%r11 + addq %rdi,%rdx + vpaddq %xmm8,%xmm6,%xmm6 + movq %r11,%r13 + addq %rdx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rdx + vpxor %xmm10,%xmm11,%xmm11 + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + vpaddq %xmm11,%xmm6,%xmm6 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + vpaddq 64(%rsi),%xmm6,%xmm10 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,96(%rsp) + vpalignr $8,%xmm7,%xmm0,%xmm8 + rorq $23,%r13 + movq %r14,%rcx + vpalignr $8,%xmm3,%xmm4,%xmm11 + movq %r11,%r12 + rorq $5,%r14 +.byte 143,72,120,195,200,56 + xorq %r10,%r13 + xorq %rax,%r12 + vpsrlq $7,%xmm8,%xmm8 + rorq $4,%r13 + xorq %rcx,%r14 + vpaddq %xmm11,%xmm7,%xmm7 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 +.byte 143,72,120,195,209,7 + xorq %rax,%r12 + rorq $6,%r14 + vpxor %xmm9,%xmm8,%xmm8 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi +.byte 143,104,120,195,222,3 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + rorq $28,%r14 + vpsrlq $6,%xmm6,%xmm10 + addq %rbx,%r9 + addq %rdi,%rbx + vpaddq %xmm8,%xmm7,%xmm7 + movq %r9,%r13 + addq %rbx,%r14 +.byte 143,72,120,195,203,42 + rorq $23,%r13 + movq %r14,%rbx + vpxor %xmm10,%xmm11,%xmm11 + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm9,%xmm11,%xmm11 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + vpaddq %xmm11,%xmm7,%xmm7 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + vpaddq 96(%rsi),%xmm7,%xmm10 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,112(%rsp) + cmpb $0,135(%rsi) + jne .Lxop_00_47 + rorq $23,%r13 + movq %r14,%rax + movq %r9,%r12 + rorq $5,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + rorq $4,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + rorq $6,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + rorq $28,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + rorq $23,%r13 + movq %r14,%r11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + rorq $23,%r13 + movq %r14,%r10 + movq %rdx,%r12 + rorq $5,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + rorq $4,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + rorq $6,%r14 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + rorq $28,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + rorq $23,%r13 + movq %r14,%r9 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + rorq $23,%r13 + movq %r14,%r8 + movq %rbx,%r12 + rorq $5,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + rorq $4,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + rorq $6,%r14 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + rorq $28,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + rorq $23,%r13 + movq %r14,%rdx + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + rorq $23,%r13 + movq %r14,%rcx + movq %r11,%r12 + rorq $5,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + rorq $4,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + rorq $6,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + rorq $28,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + rorq $23,%r13 + movq %r14,%rbx + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + rorq $23,%r13 + movq %r14,%rax + movq %r9,%r12 + rorq $5,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + rorq $4,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + rorq $6,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + rorq $14,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + rorq $28,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + rorq $23,%r13 + movq %r14,%r11 + movq %r8,%r12 + rorq $5,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + rorq $4,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + rorq $6,%r14 + xorq %rax,%rdi + addq %r12,%r10 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + rorq $28,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + rorq $23,%r13 + movq %r14,%r10 + movq %rdx,%r12 + rorq $5,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + rorq $4,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + rorq $6,%r14 + xorq %r11,%r15 + addq %r12,%r9 + rorq $14,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + rorq $28,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + rorq $23,%r13 + movq %r14,%r9 + movq %rcx,%r12 + rorq $5,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + rorq $4,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + rorq $6,%r14 + xorq %r10,%rdi + addq %r12,%r8 + rorq $14,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + rorq $28,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + rorq $23,%r13 + movq %r14,%r8 + movq %rbx,%r12 + rorq $5,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + rorq $4,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + rorq $6,%r14 + xorq %r9,%r15 + addq %r12,%rdx + rorq $14,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + rorq $28,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + rorq $23,%r13 + movq %r14,%rdx + movq %rax,%r12 + rorq $5,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + rorq $4,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + rorq $6,%r14 + xorq %r8,%rdi + addq %r12,%rcx + rorq $14,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + rorq $28,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + rorq $23,%r13 + movq %r14,%rcx + movq %r11,%r12 + rorq $5,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + rorq $4,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + rorq $6,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + rorq $14,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + rorq $28,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + rorq $23,%r13 + movq %r14,%rbx + movq %r10,%r12 + rorq $5,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + rorq $4,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + rorq $6,%r14 + xorq %rcx,%rdi + addq %r12,%rax + rorq $14,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + rorq $28,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + movq -64(%rbp),%rdi + movq %r14,%rax + movq -56(%rbp),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + leaq 128(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb .Lloop_xop + + vzeroupper + movaps -160(%rbp),%xmm6 + movaps -144(%rbp),%xmm7 + movaps -128(%rbp),%xmm8 + movaps -112(%rbp),%xmm9 + movaps -96(%rbp),%xmm10 + movaps -80(%rbp),%xmm11 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha512_asm_block_data_order_xop: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha512_asm_block_data_order_xop: +.def crypton_sha512_asm_block_data_order_avx; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha512_asm_block_data_order_avx: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha512_asm_block_data_order_avx: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lavx_shortcut: + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $120,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + movaps %xmm6,-160(%rbp) + movaps %xmm7,-144(%rbp) + movaps %xmm8,-128(%rbp) + movaps %xmm9,-112(%rbp) + + movaps %xmm10,-96(%rbp) + movaps %xmm11,-80(%rbp) + +.LSEH_body_crypton_sha512_asm_block_data_order_avx: + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-64,%rsp + movq 0(%rdi),%rax + movq 8(%rdi),%rbx + movq 16(%rdi),%rcx + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Lloop_avx +.p2align 4 +.Lloop_avx: + vmovdqa K512+1280(%rip),%xmm11 + movq %rsi,-56(%rbp) + vmovdqu 0(%rsi),%xmm0 + vmovdqu 16(%rsi),%xmm1 + vmovdqu 32(%rsi),%xmm2 + vpshufb %xmm11,%xmm0,%xmm0 + vmovdqu 48(%rsi),%xmm3 + vpshufb %xmm11,%xmm1,%xmm1 + vmovdqu 64(%rsi),%xmm4 + vpshufb %xmm11,%xmm2,%xmm2 + vmovdqu 80(%rsi),%xmm5 + vpshufb %xmm11,%xmm3,%xmm3 + vmovdqu 96(%rsi),%xmm6 + vpshufb %xmm11,%xmm4,%xmm4 + vmovdqu 112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vpshufb %xmm11,%xmm5,%xmm5 + vpaddq -128(%rsi),%xmm0,%xmm8 + vpshufb %xmm11,%xmm6,%xmm6 + vpaddq -96(%rsi),%xmm1,%xmm9 + vpshufb %xmm11,%xmm7,%xmm7 + vpaddq -64(%rsi),%xmm2,%xmm10 + vpaddq -32(%rsi),%xmm3,%xmm11 + vmovdqa %xmm8,0(%rsp) + vpaddq 0(%rsi),%xmm4,%xmm8 + vmovdqa %xmm9,16(%rsp) + vpaddq 32(%rsi),%xmm5,%xmm9 + vmovdqa %xmm10,32(%rsp) + vpaddq 64(%rsi),%xmm6,%xmm10 + vmovdqa %xmm11,48(%rsp) + vpaddq 96(%rsi),%xmm7,%xmm11 + vmovdqa %xmm8,64(%rsp) + movq %rax,%r14 + vmovdqa %xmm9,80(%rsp) + movq %rbx,%rdi + vmovdqa %xmm10,96(%rsp) + xorq %rcx,%rdi + vmovdqa %xmm11,112(%rsp) + movq %r8,%r13 + jmp .Lavx_00_47 + +.p2align 4 +.Lavx_00_47: + addq $256,%rsi + vpalignr $8,%xmm0,%xmm1,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rax + vpalignr $8,%xmm4,%xmm5,%xmm11 + movq %r9,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r8,%r13 + xorq %r10,%r12 + vpaddq %xmm11,%xmm0,%xmm0 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r8,%r12 + xorq %r8,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 0(%rsp),%r11 + movq %rax,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rbx,%r15 + addq %r12,%r11 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm7,%xmm11 + addq %r11,%rdx + addq %rdi,%r11 + vpxor %xmm9,%xmm8,%xmm8 + movq %rdx,%r13 + addq %r11,%r14 + vpsllq $3,%xmm7,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r11 + vpaddq %xmm8,%xmm0,%xmm0 + movq %r8,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm7,%xmm9 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rdx,%r12 + xorq %rdx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 8(%rsp),%r10 + movq %r11,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r9,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rax,%rdi + addq %r12,%r10 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm0,%xmm0 + xorq %r11,%r14 + addq %r13,%r10 + vpaddq -128(%rsi),%xmm0,%xmm10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,0(%rsp) + vpalignr $8,%xmm1,%xmm2,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r10 + vpalignr $8,%xmm5,%xmm6,%xmm11 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rcx,%r13 + xorq %r8,%r12 + vpaddq %xmm11,%xmm1,%xmm1 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rcx,%r12 + xorq %rcx,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 16(%rsp),%r9 + movq %r10,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r11,%r15 + addq %r12,%r9 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm0,%xmm11 + addq %r9,%rbx + addq %rdi,%r9 + vpxor %xmm9,%xmm8,%xmm8 + movq %rbx,%r13 + addq %r9,%r14 + vpsllq $3,%xmm0,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r9 + vpaddq %xmm8,%xmm1,%xmm1 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm0,%xmm9 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rbx,%r12 + xorq %rbx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 24(%rsp),%r8 + movq %r9,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r10,%rdi + addq %r12,%r8 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm1,%xmm1 + xorq %r9,%r14 + addq %r13,%r8 + vpaddq -96(%rsi),%xmm1,%xmm10 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,16(%rsp) + vpalignr $8,%xmm2,%xmm3,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r8 + vpalignr $8,%xmm6,%xmm7,%xmm11 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rax,%r13 + xorq %rcx,%r12 + vpaddq %xmm11,%xmm2,%xmm2 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rax,%r12 + xorq %rax,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 32(%rsp),%rdx + movq %r8,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r9,%r15 + addq %r12,%rdx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm1,%xmm11 + addq %rdx,%r11 + addq %rdi,%rdx + vpxor %xmm9,%xmm8,%xmm8 + movq %r11,%r13 + addq %rdx,%r14 + vpsllq $3,%xmm1,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rdx + vpaddq %xmm8,%xmm2,%xmm2 + movq %rax,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm1,%xmm9 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r11,%r12 + xorq %r11,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 40(%rsp),%rcx + movq %rdx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r8,%rdi + addq %r12,%rcx + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm2,%xmm2 + xorq %rdx,%r14 + addq %r13,%rcx + vpaddq -64(%rsi),%xmm2,%xmm10 + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,32(%rsp) + vpalignr $8,%xmm3,%xmm4,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rcx + vpalignr $8,%xmm7,%xmm0,%xmm11 + movq %r11,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r10,%r13 + xorq %rax,%r12 + vpaddq %xmm11,%xmm3,%xmm3 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r10,%r12 + xorq %r10,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 48(%rsp),%rbx + movq %rcx,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rdx,%r15 + addq %r12,%rbx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm2,%xmm11 + addq %rbx,%r9 + addq %rdi,%rbx + vpxor %xmm9,%xmm8,%xmm8 + movq %r9,%r13 + addq %rbx,%r14 + vpsllq $3,%xmm2,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rbx + vpaddq %xmm8,%xmm3,%xmm3 + movq %r10,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm2,%xmm9 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r9,%r12 + xorq %r9,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 56(%rsp),%rax + movq %rbx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r11,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rcx,%rdi + addq %r12,%rax + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm3,%xmm3 + xorq %rbx,%r14 + addq %r13,%rax + vpaddq -32(%rsi),%xmm3,%xmm10 + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,48(%rsp) + vpalignr $8,%xmm4,%xmm5,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rax + vpalignr $8,%xmm0,%xmm1,%xmm11 + movq %r9,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r8,%r13 + xorq %r10,%r12 + vpaddq %xmm11,%xmm4,%xmm4 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r8,%r12 + xorq %r8,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 64(%rsp),%r11 + movq %rax,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rbx,%r15 + addq %r12,%r11 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rax,%r14 + addq %r13,%r11 + vpxor %xmm10,%xmm8,%xmm8 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm3,%xmm11 + addq %r11,%rdx + addq %rdi,%r11 + vpxor %xmm9,%xmm8,%xmm8 + movq %rdx,%r13 + addq %r11,%r14 + vpsllq $3,%xmm3,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r11 + vpaddq %xmm8,%xmm4,%xmm4 + movq %r8,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm3,%xmm9 + xorq %rdx,%r13 + xorq %r9,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rdx,%r12 + xorq %rdx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 72(%rsp),%r10 + movq %r11,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r9,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rax,%rdi + addq %r12,%r10 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm4,%xmm4 + xorq %r11,%r14 + addq %r13,%r10 + vpaddq 0(%rsi),%xmm4,%xmm10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + vmovdqa %xmm10,64(%rsp) + vpalignr $8,%xmm5,%xmm6,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r10 + vpalignr $8,%xmm1,%xmm2,%xmm11 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rcx,%r13 + xorq %r8,%r12 + vpaddq %xmm11,%xmm5,%xmm5 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rcx,%r12 + xorq %rcx,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 80(%rsp),%r9 + movq %r10,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r11,%r15 + addq %r12,%r9 + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r10,%r14 + addq %r13,%r9 + vpxor %xmm10,%xmm8,%xmm8 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm4,%xmm11 + addq %r9,%rbx + addq %rdi,%r9 + vpxor %xmm9,%xmm8,%xmm8 + movq %rbx,%r13 + addq %r9,%r14 + vpsllq $3,%xmm4,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%r9 + vpaddq %xmm8,%xmm5,%xmm5 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm4,%xmm9 + xorq %rbx,%r13 + xorq %rdx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %rbx,%r12 + xorq %rbx,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 88(%rsp),%r8 + movq %r9,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r10,%rdi + addq %r12,%r8 + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm5,%xmm5 + xorq %r9,%r14 + addq %r13,%r8 + vpaddq 32(%rsi),%xmm5,%xmm10 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + vmovdqa %xmm10,80(%rsp) + vpalignr $8,%xmm6,%xmm7,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%r8 + vpalignr $8,%xmm2,%xmm3,%xmm11 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %rax,%r13 + xorq %rcx,%r12 + vpaddq %xmm11,%xmm6,%xmm6 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %rax,%r12 + xorq %rax,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 96(%rsp),%rdx + movq %r8,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %r9,%r15 + addq %r12,%rdx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %r8,%r14 + addq %r13,%rdx + vpxor %xmm10,%xmm8,%xmm8 + xorq %r9,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm5,%xmm11 + addq %rdx,%r11 + addq %rdi,%rdx + vpxor %xmm9,%xmm8,%xmm8 + movq %r11,%r13 + addq %rdx,%r14 + vpsllq $3,%xmm5,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rdx + vpaddq %xmm8,%xmm6,%xmm6 + movq %rax,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm5,%xmm9 + xorq %r11,%r13 + xorq %rbx,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r11,%r12 + xorq %r11,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 104(%rsp),%rcx + movq %rdx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %r8,%rdi + addq %r12,%rcx + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm6,%xmm6 + xorq %rdx,%r14 + addq %r13,%rcx + vpaddq 64(%rsi),%xmm6,%xmm10 + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + vmovdqa %xmm10,96(%rsp) + vpalignr $8,%xmm7,%xmm0,%xmm8 + shrdq $23,%r13,%r13 + movq %r14,%rcx + vpalignr $8,%xmm3,%xmm4,%xmm11 + movq %r11,%r12 + shrdq $5,%r14,%r14 + vpsrlq $1,%xmm8,%xmm10 + xorq %r10,%r13 + xorq %rax,%r12 + vpaddq %xmm11,%xmm7,%xmm7 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + vpsrlq $7,%xmm8,%xmm11 + andq %r10,%r12 + xorq %r10,%r13 + vpsllq $56,%xmm8,%xmm9 + addq 112(%rsp),%rbx + movq %rcx,%r15 + vpxor %xmm10,%xmm11,%xmm8 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + vpsrlq $7,%xmm10,%xmm10 + xorq %rdx,%r15 + addq %r12,%rbx + vpxor %xmm9,%xmm8,%xmm8 + shrdq $14,%r13,%r13 + andq %r15,%rdi + vpsllq $7,%xmm9,%xmm9 + xorq %rcx,%r14 + addq %r13,%rbx + vpxor %xmm10,%xmm8,%xmm8 + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + vpsrlq $6,%xmm6,%xmm11 + addq %rbx,%r9 + addq %rdi,%rbx + vpxor %xmm9,%xmm8,%xmm8 + movq %r9,%r13 + addq %rbx,%r14 + vpsllq $3,%xmm6,%xmm10 + shrdq $23,%r13,%r13 + movq %r14,%rbx + vpaddq %xmm8,%xmm7,%xmm7 + movq %r10,%r12 + shrdq $5,%r14,%r14 + vpsrlq $19,%xmm6,%xmm9 + xorq %r9,%r13 + xorq %r11,%r12 + vpxor %xmm10,%xmm11,%xmm11 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + vpsllq $42,%xmm10,%xmm10 + andq %r9,%r12 + xorq %r9,%r13 + vpxor %xmm9,%xmm11,%xmm11 + addq 120(%rsp),%rax + movq %rbx,%rdi + vpsrlq $42,%xmm9,%xmm9 + xorq %r11,%r12 + shrdq $6,%r14,%r14 + vpxor %xmm10,%xmm11,%xmm11 + xorq %rcx,%rdi + addq %r12,%rax + vpxor %xmm9,%xmm11,%xmm11 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + vpaddq %xmm11,%xmm7,%xmm7 + xorq %rbx,%r14 + addq %r13,%rax + vpaddq 96(%rsi),%xmm7,%xmm10 + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + vmovdqa %xmm10,112(%rsp) + cmpb $0,135(%rsi) + jne .Lavx_00_47 + shrdq $23,%r13,%r13 + movq %r14,%rax + movq %r9,%r12 + shrdq $5,%r14,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 0(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r11 + movq %r8,%r12 + shrdq $5,%r14,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 8(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + shrdq $6,%r14,%r14 + xorq %rax,%rdi + addq %r12,%r10 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r10 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 16(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + xorq %r11,%r15 + addq %r12,%r9 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r9 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 24(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + xorq %r10,%rdi + addq %r12,%r8 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r8 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 32(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + xorq %r9,%r15 + addq %r12,%rdx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + shrdq $28,%r14,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rdx + movq %rax,%r12 + shrdq $5,%r14,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 40(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + xorq %r8,%rdi + addq %r12,%rcx + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rcx + movq %r11,%r12 + shrdq $5,%r14,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 48(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rbx + movq %r10,%r12 + shrdq $5,%r14,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 56(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + shrdq $6,%r14,%r14 + xorq %rcx,%rdi + addq %r12,%rax + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rax + movq %r9,%r12 + shrdq $5,%r14,%r14 + xorq %r8,%r13 + xorq %r10,%r12 + shrdq $4,%r13,%r13 + xorq %rax,%r14 + andq %r8,%r12 + xorq %r8,%r13 + addq 64(%rsp),%r11 + movq %rax,%r15 + xorq %r10,%r12 + shrdq $6,%r14,%r14 + xorq %rbx,%r15 + addq %r12,%r11 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rax,%r14 + addq %r13,%r11 + xorq %rbx,%rdi + shrdq $28,%r14,%r14 + addq %r11,%rdx + addq %rdi,%r11 + movq %rdx,%r13 + addq %r11,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r11 + movq %r8,%r12 + shrdq $5,%r14,%r14 + xorq %rdx,%r13 + xorq %r9,%r12 + shrdq $4,%r13,%r13 + xorq %r11,%r14 + andq %rdx,%r12 + xorq %rdx,%r13 + addq 72(%rsp),%r10 + movq %r11,%rdi + xorq %r9,%r12 + shrdq $6,%r14,%r14 + xorq %rax,%rdi + addq %r12,%r10 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r11,%r14 + addq %r13,%r10 + xorq %rax,%r15 + shrdq $28,%r14,%r14 + addq %r10,%rcx + addq %r15,%r10 + movq %rcx,%r13 + addq %r10,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r10 + movq %rdx,%r12 + shrdq $5,%r14,%r14 + xorq %rcx,%r13 + xorq %r8,%r12 + shrdq $4,%r13,%r13 + xorq %r10,%r14 + andq %rcx,%r12 + xorq %rcx,%r13 + addq 80(%rsp),%r9 + movq %r10,%r15 + xorq %r8,%r12 + shrdq $6,%r14,%r14 + xorq %r11,%r15 + addq %r12,%r9 + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r10,%r14 + addq %r13,%r9 + xorq %r11,%rdi + shrdq $28,%r14,%r14 + addq %r9,%rbx + addq %rdi,%r9 + movq %rbx,%r13 + addq %r9,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r9 + movq %rcx,%r12 + shrdq $5,%r14,%r14 + xorq %rbx,%r13 + xorq %rdx,%r12 + shrdq $4,%r13,%r13 + xorq %r9,%r14 + andq %rbx,%r12 + xorq %rbx,%r13 + addq 88(%rsp),%r8 + movq %r9,%rdi + xorq %rdx,%r12 + shrdq $6,%r14,%r14 + xorq %r10,%rdi + addq %r12,%r8 + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %r9,%r14 + addq %r13,%r8 + xorq %r10,%r15 + shrdq $28,%r14,%r14 + addq %r8,%rax + addq %r15,%r8 + movq %rax,%r13 + addq %r8,%r14 + shrdq $23,%r13,%r13 + movq %r14,%r8 + movq %rbx,%r12 + shrdq $5,%r14,%r14 + xorq %rax,%r13 + xorq %rcx,%r12 + shrdq $4,%r13,%r13 + xorq %r8,%r14 + andq %rax,%r12 + xorq %rax,%r13 + addq 96(%rsp),%rdx + movq %r8,%r15 + xorq %rcx,%r12 + shrdq $6,%r14,%r14 + xorq %r9,%r15 + addq %r12,%rdx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %r8,%r14 + addq %r13,%rdx + xorq %r9,%rdi + shrdq $28,%r14,%r14 + addq %rdx,%r11 + addq %rdi,%rdx + movq %r11,%r13 + addq %rdx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rdx + movq %rax,%r12 + shrdq $5,%r14,%r14 + xorq %r11,%r13 + xorq %rbx,%r12 + shrdq $4,%r13,%r13 + xorq %rdx,%r14 + andq %r11,%r12 + xorq %r11,%r13 + addq 104(%rsp),%rcx + movq %rdx,%rdi + xorq %rbx,%r12 + shrdq $6,%r14,%r14 + xorq %r8,%rdi + addq %r12,%rcx + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rdx,%r14 + addq %r13,%rcx + xorq %r8,%r15 + shrdq $28,%r14,%r14 + addq %rcx,%r10 + addq %r15,%rcx + movq %r10,%r13 + addq %rcx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rcx + movq %r11,%r12 + shrdq $5,%r14,%r14 + xorq %r10,%r13 + xorq %rax,%r12 + shrdq $4,%r13,%r13 + xorq %rcx,%r14 + andq %r10,%r12 + xorq %r10,%r13 + addq 112(%rsp),%rbx + movq %rcx,%r15 + xorq %rax,%r12 + shrdq $6,%r14,%r14 + xorq %rdx,%r15 + addq %r12,%rbx + shrdq $14,%r13,%r13 + andq %r15,%rdi + xorq %rcx,%r14 + addq %r13,%rbx + xorq %rdx,%rdi + shrdq $28,%r14,%r14 + addq %rbx,%r9 + addq %rdi,%rbx + movq %r9,%r13 + addq %rbx,%r14 + shrdq $23,%r13,%r13 + movq %r14,%rbx + movq %r10,%r12 + shrdq $5,%r14,%r14 + xorq %r9,%r13 + xorq %r11,%r12 + shrdq $4,%r13,%r13 + xorq %rbx,%r14 + andq %r9,%r12 + xorq %r9,%r13 + addq 120(%rsp),%rax + movq %rbx,%rdi + xorq %r11,%r12 + shrdq $6,%r14,%r14 + xorq %rcx,%rdi + addq %r12,%rax + shrdq $14,%r13,%r13 + andq %rdi,%r15 + xorq %rbx,%r14 + addq %r13,%rax + xorq %rcx,%r15 + shrdq $28,%r14,%r14 + addq %rax,%r8 + addq %r15,%rax + movq %r8,%r13 + addq %rax,%r14 + movq -64(%rbp),%rdi + movq %r14,%rax + movq -56(%rbp),%rsi + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + leaq 128(%rsi),%rsi + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + jb .Lloop_avx + + vzeroupper + movaps -160(%rbp),%xmm6 + movaps -144(%rbp),%xmm7 + movaps -128(%rbp),%xmm8 + movaps -112(%rbp),%xmm9 + movaps -96(%rbp),%xmm10 + movaps -80(%rbp),%xmm11 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha512_asm_block_data_order_avx: +.def crypton_sha512_asm_block_data_order_avx2; .scl 3; .type 32; .endef +.p2align 6 +crypton_sha512_asm_block_data_order_avx2: + .byte 0xf3,0x0f,0x1e,0xfa + movq %rdi,8(%rsp) + movq %rsi,16(%rsp) + movq %rsp,%r11 +.LSEH_begin_crypton_sha512_asm_block_data_order_avx2: + + + pushq %rbp + + movq %rsp,%rbp + + movq %rcx,%rdi + movq %rdx,%rsi + movq %r8,%rdx +.Lavx2_shortcut: + pushq %rbx + + pushq %r12 + + pushq %r13 + + pushq %r14 + + pushq %r15 + + shlq $4,%rdx + subq $120,%rsp + + leaq (%rsi,%rdx,8),%rdx + movq %rdi,-64(%rbp) + + movq %rdx,-48(%rbp) + movaps %xmm6,-160(%rbp) + movaps %xmm7,-144(%rbp) + movaps %xmm8,-128(%rbp) + movaps %xmm9,-112(%rbp) + + movaps %xmm10,-96(%rbp) + movaps %xmm11,-80(%rbp) + +.LSEH_body_crypton_sha512_asm_block_data_order_avx2: + + + leaq -128(%rsp),%rsp + vzeroupper + andq $-128,%rsp + subq $-128,%rsi + movq 0(%rdi),%rax + movq %rsi,%r12 + movq 8(%rdi),%rbx + cmpq %rdx,%rsi + movq 16(%rdi),%rcx + cmoveq %rsp,%r12 + movq 24(%rdi),%rdx + movq 32(%rdi),%r8 + movq 40(%rdi),%r9 + movq 48(%rdi),%r10 + movq 56(%rdi),%r11 + jmp .Loop_avx2 +.p2align 4 +.Loop_avx2: + vmovdqa K512+1280(%rip),%ymm10 + movq %rsi,-56(%rbp) + vmovdqu -128(%rsi),%xmm0 + vmovdqu -128+16(%rsi),%xmm1 + vmovdqu -128+32(%rsi),%xmm2 + vmovdqu -128+48(%rsi),%xmm3 + vmovdqu -128+64(%rsi),%xmm4 + vmovdqu -128+80(%rsi),%xmm5 + vmovdqu -128+96(%rsi),%xmm6 + vmovdqu -128+112(%rsi),%xmm7 + leaq K512+128(%rip),%rsi + vinserti128 $1,(%r12),%ymm0,%ymm0 + vinserti128 $1,16(%r12),%ymm1,%ymm1 + vpshufb %ymm10,%ymm0,%ymm0 + vinserti128 $1,32(%r12),%ymm2,%ymm2 + vpshufb %ymm10,%ymm1,%ymm1 + vinserti128 $1,48(%r12),%ymm3,%ymm3 + vpshufb %ymm10,%ymm2,%ymm2 + vinserti128 $1,64(%r12),%ymm4,%ymm4 + vpshufb %ymm10,%ymm3,%ymm3 + vinserti128 $1,80(%r12),%ymm5,%ymm5 + vpshufb %ymm10,%ymm4,%ymm4 + vinserti128 $1,96(%r12),%ymm6,%ymm6 + vpshufb %ymm10,%ymm5,%ymm5 + vinserti128 $1,112(%r12),%ymm7,%ymm7 + + vpaddq -128(%rsi),%ymm0,%ymm8 + vpshufb %ymm10,%ymm6,%ymm6 + vpaddq -96(%rsi),%ymm1,%ymm9 + vpshufb %ymm10,%ymm7,%ymm7 + vpaddq -64(%rsi),%ymm2,%ymm10 + vpaddq -32(%rsi),%ymm3,%ymm11 + vmovdqa %ymm8,0(%rsp) + vpaddq 0(%rsi),%ymm4,%ymm8 + vmovdqa %ymm9,32(%rsp) + vpaddq 32(%rsi),%ymm5,%ymm9 + vmovdqa %ymm10,64(%rsp) + vpaddq 64(%rsi),%ymm6,%ymm10 + vmovdqa %ymm11,96(%rsp) + leaq -128(%rsp),%rsp + vpaddq 96(%rsi),%ymm7,%ymm11 + vmovdqa %ymm8,0(%rsp) + xorq %r14,%r14 + vmovdqa %ymm9,32(%rsp) + movq %rbx,%rdi + vmovdqa %ymm10,64(%rsp) + xorq %rcx,%rdi + vmovdqa %ymm11,96(%rsp) + movq %r9,%r12 + addq $32*8,%rsi + jmp .Lavx2_00_47 + +.p2align 4 +.Lavx2_00_47: + leaq -128(%rsp),%rsp + vpalignr $8,%ymm0,%ymm1,%ymm8 + addq 0+256(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + vpalignr $8,%ymm4,%ymm5,%ymm11 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + vpsrlq $1,%ymm8,%ymm10 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + vpaddq %ymm11,%ymm0,%ymm0 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + vpsrlq $6,%ymm7,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + vpsllq $3,%ymm7,%ymm10 + vpaddq %ymm8,%ymm0,%ymm0 + addq 8+256(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + vpsrlq $19,%ymm7,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + vpaddq %ymm11,%ymm0,%ymm0 + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + vpaddq -128(%rsi),%ymm0,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + vmovdqa %ymm10,0(%rsp) + vpalignr $8,%ymm1,%ymm2,%ymm8 + addq 32+256(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + vpalignr $8,%ymm5,%ymm6,%ymm11 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + vpsrlq $1,%ymm8,%ymm10 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + vpaddq %ymm11,%ymm1,%ymm1 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + vpsrlq $6,%ymm0,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + vpsllq $3,%ymm0,%ymm10 + vpaddq %ymm8,%ymm1,%ymm1 + addq 40+256(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + vpsrlq $19,%ymm0,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + vpaddq %ymm11,%ymm1,%ymm1 + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + vpaddq -96(%rsi),%ymm1,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + vmovdqa %ymm10,32(%rsp) + vpalignr $8,%ymm2,%ymm3,%ymm8 + addq 64+256(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + vpalignr $8,%ymm6,%ymm7,%ymm11 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + vpsrlq $1,%ymm8,%ymm10 + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + vpaddq %ymm11,%ymm2,%ymm2 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + vpsrlq $6,%ymm1,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + vpsllq $3,%ymm1,%ymm10 + vpaddq %ymm8,%ymm2,%ymm2 + addq 72+256(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + vpsrlq $19,%ymm1,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + vpaddq %ymm11,%ymm2,%ymm2 + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + vpaddq -64(%rsi),%ymm2,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + vmovdqa %ymm10,64(%rsp) + vpalignr $8,%ymm3,%ymm4,%ymm8 + addq 96+256(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + vpalignr $8,%ymm7,%ymm0,%ymm11 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + vpsrlq $1,%ymm8,%ymm10 + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + vpaddq %ymm11,%ymm3,%ymm3 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + vpsrlq $6,%ymm2,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + vpsllq $3,%ymm2,%ymm10 + vpaddq %ymm8,%ymm3,%ymm3 + addq 104+256(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + vpsrlq $19,%ymm2,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + vpaddq %ymm11,%ymm3,%ymm3 + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + vpaddq -32(%rsi),%ymm3,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + vmovdqa %ymm10,96(%rsp) + leaq -128(%rsp),%rsp + vpalignr $8,%ymm4,%ymm5,%ymm8 + addq 0+256(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + vpalignr $8,%ymm0,%ymm1,%ymm11 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + vpsrlq $1,%ymm8,%ymm10 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + vpaddq %ymm11,%ymm4,%ymm4 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + vpsrlq $6,%ymm3,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + vpsllq $3,%ymm3,%ymm10 + vpaddq %ymm8,%ymm4,%ymm4 + addq 8+256(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + vpsrlq $19,%ymm3,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + vpaddq %ymm11,%ymm4,%ymm4 + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + vpaddq 0(%rsi),%ymm4,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + vmovdqa %ymm10,0(%rsp) + vpalignr $8,%ymm5,%ymm6,%ymm8 + addq 32+256(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + vpalignr $8,%ymm1,%ymm2,%ymm11 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + vpsrlq $1,%ymm8,%ymm10 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + vpaddq %ymm11,%ymm5,%ymm5 + vpsrlq $7,%ymm8,%ymm11 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + vpsrlq $6,%ymm4,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + vpsllq $3,%ymm4,%ymm10 + vpaddq %ymm8,%ymm5,%ymm5 + addq 40+256(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + vpsrlq $19,%ymm4,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + vpaddq %ymm11,%ymm5,%ymm5 + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + vpaddq 32(%rsi),%ymm5,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + vmovdqa %ymm10,32(%rsp) + vpalignr $8,%ymm6,%ymm7,%ymm8 + addq 64+256(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + vpalignr $8,%ymm2,%ymm3,%ymm11 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + vpsrlq $1,%ymm8,%ymm10 + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + vpaddq %ymm11,%ymm6,%ymm6 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + vpsrlq $6,%ymm5,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + vpsllq $3,%ymm5,%ymm10 + vpaddq %ymm8,%ymm6,%ymm6 + addq 72+256(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + vpsrlq $19,%ymm5,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + vpaddq %ymm11,%ymm6,%ymm6 + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + vpaddq 64(%rsi),%ymm6,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + vmovdqa %ymm10,64(%rsp) + vpalignr $8,%ymm7,%ymm0,%ymm8 + addq 96+256(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + vpalignr $8,%ymm3,%ymm4,%ymm11 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + vpsrlq $1,%ymm8,%ymm10 + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + vpaddq %ymm11,%ymm7,%ymm7 + vpsrlq $7,%ymm8,%ymm11 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + vpsllq $56,%ymm8,%ymm9 + vpxor %ymm10,%ymm11,%ymm8 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + vpsrlq $7,%ymm10,%ymm10 + vpxor %ymm9,%ymm8,%ymm8 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + vpsllq $7,%ymm9,%ymm9 + vpxor %ymm10,%ymm8,%ymm8 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + vpsrlq $6,%ymm6,%ymm11 + vpxor %ymm9,%ymm8,%ymm8 + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + vpsllq $3,%ymm6,%ymm10 + vpaddq %ymm8,%ymm7,%ymm7 + addq 104+256(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + vpsrlq $19,%ymm6,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + vpsllq $42,%ymm10,%ymm10 + vpxor %ymm9,%ymm11,%ymm11 + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + vpsrlq $42,%ymm9,%ymm9 + vpxor %ymm10,%ymm11,%ymm11 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + vpxor %ymm9,%ymm11,%ymm11 + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + vpaddq %ymm11,%ymm7,%ymm7 + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + vpaddq 96(%rsi),%ymm7,%ymm10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + vmovdqa %ymm10,96(%rsp) + leaq 256(%rsi),%rsi + cmpb $0,-121(%rsi) + jne .Lavx2_00_47 + addq 0+128(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8+128(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32+128(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40+128(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64+128(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72+128(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96+128(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104+128(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + addq 0(%rsp),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8(%rsp),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32(%rsp),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40(%rsp),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64(%rsp),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72(%rsp),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96(%rsp),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104(%rsp),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + movq -64(%rbp),%rdi + addq %r14,%rax + movq -56(%rbp),%r12 + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + addq 48(%rdi),%r10 + addq 56(%rdi),%r11 + + movq %rax,0(%rdi) + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + + cmpq -48(%rbp),%r12 + je .Ldone_avx2 + + leaq 1152(%rsp),%rsi + xorq %r14,%r14 + movq %rbx,%rdi + xorq %rcx,%rdi + movq %r9,%r12 + jmp .Lower_avx2 +.p2align 4 +.Lower_avx2: + addq 0+16(%rsi),%r11 + andq %r8,%r12 + rorxq $41,%r8,%r13 + rorxq $18,%r8,%r15 + leaq (%rax,%r14,1),%rax + leaq (%r11,%r12,1),%r11 + andnq %r10,%r8,%r12 + xorq %r15,%r13 + rorxq $14,%r8,%r14 + leaq (%r11,%r12,1),%r11 + xorq %r14,%r13 + movq %rax,%r15 + rorxq $39,%rax,%r12 + leaq (%r11,%r13,1),%r11 + xorq %rbx,%r15 + rorxq $34,%rax,%r14 + rorxq $28,%rax,%r13 + leaq (%rdx,%r11,1),%rdx + andq %r15,%rdi + xorq %r12,%r14 + xorq %rbx,%rdi + xorq %r13,%r14 + leaq (%r11,%rdi,1),%r11 + movq %r8,%r12 + addq 8+16(%rsi),%r10 + andq %rdx,%r12 + rorxq $41,%rdx,%r13 + rorxq $18,%rdx,%rdi + leaq (%r11,%r14,1),%r11 + leaq (%r10,%r12,1),%r10 + andnq %r9,%rdx,%r12 + xorq %rdi,%r13 + rorxq $14,%rdx,%r14 + leaq (%r10,%r12,1),%r10 + xorq %r14,%r13 + movq %r11,%rdi + rorxq $39,%r11,%r12 + leaq (%r10,%r13,1),%r10 + xorq %rax,%rdi + rorxq $34,%r11,%r14 + rorxq $28,%r11,%r13 + leaq (%rcx,%r10,1),%rcx + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rax,%r15 + xorq %r13,%r14 + leaq (%r10,%r15,1),%r10 + movq %rdx,%r12 + addq 32+16(%rsi),%r9 + andq %rcx,%r12 + rorxq $41,%rcx,%r13 + rorxq $18,%rcx,%r15 + leaq (%r10,%r14,1),%r10 + leaq (%r9,%r12,1),%r9 + andnq %r8,%rcx,%r12 + xorq %r15,%r13 + rorxq $14,%rcx,%r14 + leaq (%r9,%r12,1),%r9 + xorq %r14,%r13 + movq %r10,%r15 + rorxq $39,%r10,%r12 + leaq (%r9,%r13,1),%r9 + xorq %r11,%r15 + rorxq $34,%r10,%r14 + rorxq $28,%r10,%r13 + leaq (%rbx,%r9,1),%rbx + andq %r15,%rdi + xorq %r12,%r14 + xorq %r11,%rdi + xorq %r13,%r14 + leaq (%r9,%rdi,1),%r9 + movq %rcx,%r12 + addq 40+16(%rsi),%r8 + andq %rbx,%r12 + rorxq $41,%rbx,%r13 + rorxq $18,%rbx,%rdi + leaq (%r9,%r14,1),%r9 + leaq (%r8,%r12,1),%r8 + andnq %rdx,%rbx,%r12 + xorq %rdi,%r13 + rorxq $14,%rbx,%r14 + leaq (%r8,%r12,1),%r8 + xorq %r14,%r13 + movq %r9,%rdi + rorxq $39,%r9,%r12 + leaq (%r8,%r13,1),%r8 + xorq %r10,%rdi + rorxq $34,%r9,%r14 + rorxq $28,%r9,%r13 + leaq (%rax,%r8,1),%rax + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r10,%r15 + xorq %r13,%r14 + leaq (%r8,%r15,1),%r8 + movq %rbx,%r12 + addq 64+16(%rsi),%rdx + andq %rax,%r12 + rorxq $41,%rax,%r13 + rorxq $18,%rax,%r15 + leaq (%r8,%r14,1),%r8 + leaq (%rdx,%r12,1),%rdx + andnq %rcx,%rax,%r12 + xorq %r15,%r13 + rorxq $14,%rax,%r14 + leaq (%rdx,%r12,1),%rdx + xorq %r14,%r13 + movq %r8,%r15 + rorxq $39,%r8,%r12 + leaq (%rdx,%r13,1),%rdx + xorq %r9,%r15 + rorxq $34,%r8,%r14 + rorxq $28,%r8,%r13 + leaq (%r11,%rdx,1),%r11 + andq %r15,%rdi + xorq %r12,%r14 + xorq %r9,%rdi + xorq %r13,%r14 + leaq (%rdx,%rdi,1),%rdx + movq %rax,%r12 + addq 72+16(%rsi),%rcx + andq %r11,%r12 + rorxq $41,%r11,%r13 + rorxq $18,%r11,%rdi + leaq (%rdx,%r14,1),%rdx + leaq (%rcx,%r12,1),%rcx + andnq %rbx,%r11,%r12 + xorq %rdi,%r13 + rorxq $14,%r11,%r14 + leaq (%rcx,%r12,1),%rcx + xorq %r14,%r13 + movq %rdx,%rdi + rorxq $39,%rdx,%r12 + leaq (%rcx,%r13,1),%rcx + xorq %r8,%rdi + rorxq $34,%rdx,%r14 + rorxq $28,%rdx,%r13 + leaq (%r10,%rcx,1),%r10 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %r8,%r15 + xorq %r13,%r14 + leaq (%rcx,%r15,1),%rcx + movq %r11,%r12 + addq 96+16(%rsi),%rbx + andq %r10,%r12 + rorxq $41,%r10,%r13 + rorxq $18,%r10,%r15 + leaq (%rcx,%r14,1),%rcx + leaq (%rbx,%r12,1),%rbx + andnq %rax,%r10,%r12 + xorq %r15,%r13 + rorxq $14,%r10,%r14 + leaq (%rbx,%r12,1),%rbx + xorq %r14,%r13 + movq %rcx,%r15 + rorxq $39,%rcx,%r12 + leaq (%rbx,%r13,1),%rbx + xorq %rdx,%r15 + rorxq $34,%rcx,%r14 + rorxq $28,%rcx,%r13 + leaq (%r9,%rbx,1),%r9 + andq %r15,%rdi + xorq %r12,%r14 + xorq %rdx,%rdi + xorq %r13,%r14 + leaq (%rbx,%rdi,1),%rbx + movq %r10,%r12 + addq 104+16(%rsi),%rax + andq %r9,%r12 + rorxq $41,%r9,%r13 + rorxq $18,%r9,%rdi + leaq (%rbx,%r14,1),%rbx + leaq (%rax,%r12,1),%rax + andnq %r11,%r9,%r12 + xorq %rdi,%r13 + rorxq $14,%r9,%r14 + leaq (%rax,%r12,1),%rax + xorq %r14,%r13 + movq %rbx,%rdi + rorxq $39,%rbx,%r12 + leaq (%rax,%r13,1),%rax + xorq %rcx,%rdi + rorxq $34,%rbx,%r14 + rorxq $28,%rbx,%r13 + leaq (%r8,%rax,1),%r8 + andq %rdi,%r15 + xorq %r12,%r14 + xorq %rcx,%r15 + xorq %r13,%r14 + leaq (%rax,%r15,1),%rax + movq %r9,%r12 + leaq -128(%rsi),%rsi + cmpq %rsp,%rsi + jae .Lower_avx2 + + movq -64(%rbp),%rdi + addq %r14,%rax + movq -56(%rbp),%rsi + leaq 1152(%rsp),%rsp + + addq 0(%rdi),%rax + addq 8(%rdi),%rbx + addq 16(%rdi),%rcx + addq 24(%rdi),%rdx + addq 32(%rdi),%r8 + addq 40(%rdi),%r9 + leaq 256(%rsi),%rsi + addq 48(%rdi),%r10 + movq %rsi,%r12 + addq 56(%rdi),%r11 + cmpq -48(%rbp),%rsi + + movq %rax,0(%rdi) + cmoveq %rsp,%r12 + movq %rbx,8(%rdi) + movq %rcx,16(%rdi) + movq %rdx,24(%rdi) + movq %r8,32(%rdi) + movq %r9,40(%rdi) + movq %r10,48(%rdi) + movq %r11,56(%rdi) + + jbe .Loop_avx2 + +.Ldone_avx2: + vzeroupper + movaps -160(%rbp),%xmm6 + movaps -144(%rbp),%xmm7 + movaps -128(%rbp),%xmm8 + movaps -112(%rbp),%xmm9 + movaps -96(%rbp),%xmm10 + movaps -80(%rbp),%xmm11 + movq -40(%rbp),%r15 + movq -32(%rbp),%r14 + movq -24(%rbp),%r13 + movq -16(%rbp),%r12 + movq -8(%rbp),%rbx + movq %rbp,%rsp + + popq %rbp + +.LSEH_epilogue_crypton_sha512_asm_block_data_order_avx2: + mov 8(%rsp),%rdi + mov 16(%rsp),%rsi + + .byte 0xf3,0xc3 + +.LSEH_end_crypton_sha512_asm_block_data_order_avx2: +.section .pdata +.p2align 2 +.rva .LSEH_begin_crypton_sha512_asm_block_data_order +.rva .LSEH_body_crypton_sha512_asm_block_data_order +.rva .LSEH_info_crypton_sha512_asm_block_data_order_prologue + +.rva .LSEH_body_crypton_sha512_asm_block_data_order +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order +.rva .LSEH_info_crypton_sha512_asm_block_data_order_body + +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order +.rva .LSEH_end_crypton_sha512_asm_block_data_order +.rva .LSEH_info_crypton_sha512_asm_block_data_order_epilogue + +.rva .LSEH_begin_crypton_sha512_asm_block_data_order_shaext +.rva .LSEH_body_crypton_sha512_asm_block_data_order_shaext +.rva .LSEH_info_crypton_sha512_asm_block_data_order_shaext_prologue + +.rva .LSEH_body_crypton_sha512_asm_block_data_order_shaext +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_shaext +.rva .LSEH_info_crypton_sha512_asm_block_data_order_shaext_body + +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_shaext +.rva .LSEH_end_crypton_sha512_asm_block_data_order_shaext +.rva .LSEH_info_crypton_sha512_asm_block_data_order_shaext_epilogue + +.rva .LSEH_begin_crypton_sha512_asm_block_data_order_xop +.rva .LSEH_body_crypton_sha512_asm_block_data_order_xop +.rva .LSEH_info_crypton_sha512_asm_block_data_order_xop_prologue + +.rva .LSEH_body_crypton_sha512_asm_block_data_order_xop +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_xop +.rva .LSEH_info_crypton_sha512_asm_block_data_order_xop_body + +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_xop +.rva .LSEH_end_crypton_sha512_asm_block_data_order_xop +.rva .LSEH_info_crypton_sha512_asm_block_data_order_xop_epilogue + +.rva .LSEH_begin_crypton_sha512_asm_block_data_order_avx +.rva .LSEH_body_crypton_sha512_asm_block_data_order_avx +.rva .LSEH_info_crypton_sha512_asm_block_data_order_avx_prologue + +.rva .LSEH_body_crypton_sha512_asm_block_data_order_avx +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_avx +.rva .LSEH_info_crypton_sha512_asm_block_data_order_avx_body + +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_avx +.rva .LSEH_end_crypton_sha512_asm_block_data_order_avx +.rva .LSEH_info_crypton_sha512_asm_block_data_order_avx_epilogue + +.rva .LSEH_begin_crypton_sha512_asm_block_data_order_avx2 +.rva .LSEH_body_crypton_sha512_asm_block_data_order_avx2 +.rva .LSEH_info_crypton_sha512_asm_block_data_order_avx2_prologue + +.rva .LSEH_body_crypton_sha512_asm_block_data_order_avx2 +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_avx2 +.rva .LSEH_info_crypton_sha512_asm_block_data_order_avx2_body + +.rva .LSEH_epilogue_crypton_sha512_asm_block_data_order_avx2 +.rva .LSEH_end_crypton_sha512_asm_block_data_order_avx2 +.rva .LSEH_info_crypton_sha512_asm_block_data_order_avx2_epilogue + +.section .xdata +.p2align 3 +.LSEH_info_crypton_sha512_asm_block_data_order_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha512_asm_block_data_order_body: +.byte 1,0,18,0 +.byte 0x00,0xf4,0x13,0x00 +.byte 0x00,0xe4,0x14,0x00 +.byte 0x00,0xd4,0x15,0x00 +.byte 0x00,0xc4,0x16,0x00 +.byte 0x00,0x34,0x17,0x00 +.byte 0x00,0x54,0x18,0x00 +.byte 0x00,0x74,0x1a,0x00 +.byte 0x00,0x64,0x1b,0x00 +.byte 0x00,0x01,0x19,0x00 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha512_asm_block_data_order_epilogue: +.byte 1,0,5,11 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0xb3 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha512_asm_block_data_order_shaext_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha512_asm_block_data_order_shaext_body: +.byte 1,0,17,85 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0x74,0x0c,0x00 +.byte 0x00,0x64,0x0d,0x00 +.byte 0x00,0x53 +.byte 0x00,0x92 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha512_asm_block_data_order_shaext_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha512_asm_block_data_order_xop_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha512_asm_block_data_order_xop_body: +.byte 1,0,30,165 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0xb8,0x05,0x00 +.byte 0x00,0xf4,0x0f,0x00 +.byte 0x00,0xe4,0x10,0x00 +.byte 0x00,0xd4,0x11,0x00 +.byte 0x00,0xc4,0x12,0x00 +.byte 0x00,0x34,0x13,0x00 +.byte 0x00,0x74,0x16,0x00 +.byte 0x00,0x64,0x17,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x14,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha512_asm_block_data_order_xop_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha512_asm_block_data_order_avx_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha512_asm_block_data_order_avx_body: +.byte 1,0,30,165 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0xb8,0x05,0x00 +.byte 0x00,0xf4,0x0f,0x00 +.byte 0x00,0xe4,0x10,0x00 +.byte 0x00,0xd4,0x11,0x00 +.byte 0x00,0xc4,0x12,0x00 +.byte 0x00,0x34,0x13,0x00 +.byte 0x00,0x74,0x16,0x00 +.byte 0x00,0x64,0x17,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x14,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha512_asm_block_data_order_avx_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + +.LSEH_info_crypton_sha512_asm_block_data_order_avx2_prologue: +.byte 1,4,6,0x05 +.byte 4,0x74,2,0 +.byte 4,0x64,3,0 +.byte 4,0x53 +.byte 1,0x50 +.long 0,0 +.LSEH_info_crypton_sha512_asm_block_data_order_avx2_body: +.byte 1,0,30,165 +.byte 0x00,0x68,0x00,0x00 +.byte 0x00,0x78,0x01,0x00 +.byte 0x00,0x88,0x02,0x00 +.byte 0x00,0x98,0x03,0x00 +.byte 0x00,0xa8,0x04,0x00 +.byte 0x00,0xb8,0x05,0x00 +.byte 0x00,0xf4,0x0f,0x00 +.byte 0x00,0xe4,0x10,0x00 +.byte 0x00,0xd4,0x11,0x00 +.byte 0x00,0xc4,0x12,0x00 +.byte 0x00,0x34,0x13,0x00 +.byte 0x00,0x74,0x16,0x00 +.byte 0x00,0x64,0x17,0x00 +.byte 0x00,0x53 +.byte 0x00,0x01,0x14,0x00 +.byte 0x00,0x50 +.byte 0x00,0x00,0x00,0x00 +.byte 0x00,0x00,0x00,0x00 +.LSEH_info_crypton_sha512_asm_block_data_order_avx2_epilogue: +.byte 1,0,4,0 +.byte 0x00,0x74,0x01,0x00 +.byte 0x00,0x64,0x02,0x00 +.byte 0x00,0x00,0x00,0x00 + diff --git a/cbits/asm/sha512-x86_64.pl b/cbits/asm/sha512-x86_64.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/sha512-x86_64.pl @@ -0,0 +1,2519 @@ +#!/usr/bin/env perl +# +# ==================================================================== +# Written by Andy Polyakov, @dot-asm, initially for the OpenSSL +# project. +# ==================================================================== +# +# sha256/512_block procedure for x86_64. +# +# 40% improvement over compiler-generated code on Opteron. On EM64T +# sha256 was observed to run >80% faster and sha512 - >40%. No magical +# tricks, just straight implementation... I really wonder why gcc +# [being armed with inline assembler] fails to generate as fast code. +# The only thing which is cool about this module is that it's very +# same instruction sequence used for both SHA-256 and SHA-512. In +# former case the instructions operate on 32-bit operands, while in +# latter - on 64-bit ones. All I had to do is to get one flavor right, +# the other one passed the test right away:-) +# +# sha256_block runs in ~1005 cycles on Opteron, which gives you +# asymptotic performance of 64*1000/1005=63.7MBps times CPU clock +# frequency in GHz. sha512_block runs in ~1275 cycles, which results +# in 128*1000/1275=100MBps per GHz. Is there room for improvement? +# Well, if you compare it to IA-64 implementation, which maintains +# X[16] in register bank[!], tends to 4 instructions per CPU clock +# cycle and runs in 1003 cycles, 1275 is very good result for 3-way +# issue Opteron pipeline and X[16] maintained in memory. So that *if* +# there is a way to improve it, *then* the only way would be to try to +# offload X[16] updates to SSE unit, but that would require "deeper" +# loop unroll, which in turn would naturally cause size blow-up, not +# to mention increased complexity! And once again, only *if* it's +# actually possible to noticeably improve overall ILP, instruction +# level parallelism, on a given CPU implementation in this case. +# +# Special note on Intel EM64T. While Opteron CPU exhibits perfect +# performance ratio of 1.5 between 64- and 32-bit flavors [see above], +# [currently available] EM64T CPUs apparently are far from it. On the +# contrary, 64-bit version, sha512_block, is ~30% *slower* than 32-bit +# sha256_block:-( This is presumably because 64-bit shifts/rotates +# apparently are not atomic instructions, but implemented in microcode. +# +# May 2012. +# +# Optimization including one of Pavel Semjanov's ideas, alternative +# Maj, resulted in >=5% improvement on most CPUs, +20% SHA256 and +# unfortunately -2% SHA512 on P4 [which nobody should care about +# that much]. +# +# June 2012. +# +# Add SIMD code paths, see below for improvement coefficients. SSSE3 +# code path was not attempted for SHA512, because improvement is not +# estimated to be high enough, noticeably less than 9%, to justify +# the effort, not on pre-AVX processors. [Obviously with exclusion +# for VIA Nano, but it has SHA512 instruction that is faster and +# should be used instead.] For reference, corresponding estimated +# upper limit for improvement for SSSE3 SHA256 is 28%. The fact that +# higher coefficients are observed on VIA Nano and Bulldozer has more +# to do with specifics of their architecture [which is topic for +# separate discussion]. +# +# November 2012. +# +# Add AVX2 code path. Two consecutive input blocks are loaded to +# 256-bit %ymm registers, with data from first block to least +# significant 128-bit halves and data from second to most significant. +# The data is then processed with same SIMD instruction sequence as +# for AVX, but with %ymm as operands. Side effect is increased stack +# frame, 448 additional bytes in SHA256 and 1152 in SHA512, and 1.2KB +# code size increase. +# +# March 2014. +# +# Add support for Intel SHA Extensions. +# +# October 2023. +# +# Add support for Intel SHA512 Extension. + +###################################################################### +# Current performance in cycles per processed byte (less is better): +# +# SHA256 SSSE3 AVX/XOP(*) SHA512 AVX/XOP(*) +# +# AMD K8 14.9 - - 9.57 - +# P4 17.3 - - 30.8 - +# Core 2 15.6 13.8(+13%) - 9.97 - +# Westmere 14.8 12.3(+19%) - 9.58 - +# Sandy Bridge 17.4 14.2(+23%) 11.6(+50%(**)) 11.2 8.10(+38%(**)) +# Ivy Bridge 12.6 10.5(+20%) 10.3(+22%) 8.17 7.22(+13%) +# Haswell 12.2 9.28(+31%) 7.80(+56%) 7.66 5.40(+42%) +# Skylake 11.4 9.03(+26%) 7.70(+48%) 7.25 5.20(+40%) +# Cannon Lake 11.4 9.00(+27%) 3.55(+220%) 7.20 5.12(+41%) +# Rocket Lake 10.4 9.13(+14%) 2.43(+330%) 6.66 5.34(+25%) +# Bulldozer 21.1 13.6(+54%) 13.6(+54%(***)) 13.5 8.58(+57%) +# Ryzen 11.0 9.02(+22%) 2.05(+440%) 7.05 5.67(+20%) +# VIA Nano 23.0 16.5(+39%) - 14.7 - +# Atom 23.0 18.9(+22%) - 14.7 - +# Silvermont 27.4 20.6(+33%) - 17.5 - +# Knights L 27.4 21.0(+30%) 19.6(+40%) 17.5 12.8(+37%) +# Goldmont 18.9 14.3(+32%) 4.16(+350%) 12.0 - +# +# (*) whichever best applicable, including SHAEXT; +# (**) switch from ror to shrd stands for fair share of improvement; +# (***) execution time is fully determined by remaining integer-only +# part, body_00_15; reducing the amount of SIMD instructions +# below certain limit makes no difference/sense; to conserve +# space SHA256 XOP code path is therefore omitted; + +$flavour = shift; +$output = pop; +if ($flavour =~ /\./) { $output = $flavour; undef $flavour; } + +$win64=0; $win64=1 if ($flavour =~ /[nm]asm|mingw64/ || $output =~ /\.asm$/); + +$0 =~ m/(.*[\/\\])[^\/\\]+$/; $dir=$1; +( $xlate="${dir}x86_64-xlate.pl" and -f $xlate ) or +( $xlate="${dir}../../perlasm/x86_64-xlate.pl" and -f $xlate) or +die "can't locate x86_64-xlate.pl"; + +$avx=undef; +$shaext=1; ### set to zero if compiling for 1.0.1 + +if (!defined($avx) && $win64 && ($flavour =~ /nasm/ || $ENV{ASM} =~ /nasm/) && + ($ENV{ASM} //= "nasm") && + `"$ENV{ASM}" -v 2>&1` =~ /NASM version ([0-9]+\.[0-9]+)(?:\.([0-9]+))?/) { + $avx = ($1>=2.09) + ($1>=2.10) + 2 * ($1>=2.12); + $avx += 2 if ($1==2.11 && $2>=8); +} + +if (!defined($avx) && $win64 && ($flavour =~ /masm/ || $ENV{ASM} =~ /ml64/) && + ($ENV{ASM} //= "ml64") && + `"$ENV{ASM}" 2>&1` =~ /Version ([0-9]+)\./) { + $avx = ($1>=10) + ($1>=12) + 2 * ($1>=14); +} + +$ENV{CC} //= "cc"; +if (!defined($avx) && `$ENV{CC} -Wa,-v -c -o /dev/zero -x assembler /dev/null 2>&1` + =~ /GNU assembler version ([0-9]+)\.([0-9]+)/) { + my $ver = $1 + $2/100.0; # 3.1->3.01, 3.10->3.10 + $avx = ($ver>=2.19) + ($ver>=2.22) + ($ver>=2.25) + ($ver>=2.26); +} + +if (!defined($avx) && `$ENV{CC} -v 2>&1` + =~ /((?:^clang|LLVM) version|.*based on LLVM) ([0-9]+)\.([0-9]+)/) { + my $ver = $2 + $3/100.0; # 3.1->3.01, 3.10->3.10 + $avx = ($ver>=3.0) + ($ver>3.0); + $avx += 2*($ver>=7.0) if ($1 =~ /^clang/); +} + +open STDOUT,"| \"$^X\" \"$xlate\" $flavour \"$output\""; + +if ($output =~ /512/) { + $func="sha512_block_data_order"; + $TABLE="K512"; + $SZ=8; + @ROT=($A,$B,$C,$D,$E,$F,$G,$H)=("%rax","%rbx","%rcx","%rdx", + "%r8", "%r9", "%r10","%r11"); + ($T1,$a0,$a1,$a2,$a3)=("%r12","%r13","%r14","%r15","%rdi"); + @Sigma0=(28,34,39); + @Sigma1=(14,18,41); + @sigma0=(1, 8, 7); + @sigma1=(19,61, 6); + $rounds=80; +} else { + $func="sha256_block_data_order"; + $TABLE="K256"; + $SZ=4; + @ROT=($A,$B,$C,$D,$E,$F,$G,$H)=("%eax","%ebx","%ecx","%edx", + "%r8d","%r9d","%r10d","%r11d"); + ($T1,$a0,$a1,$a2,$a3)=("%r12d","%r13d","%r14d","%r15d","%edi"); + @Sigma0=( 2,13,22); + @Sigma1=( 6,11,25); + @sigma0=( 7,18, 3); + @sigma1=(17,19,10); + $rounds=64; +} + +$ctx="%rdi"; # 1st arg, zapped by $a3 +$inp="%rsi"; # 2nd arg +$Tbl="%rbp"; + +$_ctx="16*$SZ+0*8(%rsp)"; +$_inp="16*$SZ+1*8(%rsp)"; +$_end="16*$SZ+2*8(%rsp)"; +$framesz="16*$SZ+3*8"; + + +sub ROUND_00_15() +{ my ($i,$a,$b,$c,$d,$e,$f,$g,$h) = @_; + my $STRIDE=$SZ; + $STRIDE += 16 if ($i%(16/$SZ)==(16/$SZ-1)); + +$code.=<<___; + ror \$`$Sigma1[2]-$Sigma1[1]`,$a0 + mov $f,$a2 + + xor $e,$a0 + ror \$`$Sigma0[2]-$Sigma0[1]`,$a1 + xor $g,$a2 # f^g + + mov $T1,`$SZ*($i&0xf)`(%rsp) + xor $a,$a1 + and $e,$a2 # (f^g)&e + + ror \$`$Sigma1[1]-$Sigma1[0]`,$a0 + add $h,$T1 # T1+=h + xor $g,$a2 # Ch(e,f,g)=((f^g)&e)^g + + ror \$`$Sigma0[1]-$Sigma0[0]`,$a1 + xor $e,$a0 + add $a2,$T1 # T1+=Ch(e,f,g) + + mov $a,$a2 + add ($Tbl),$T1 # T1+=K[round] + xor $a,$a1 + + xor $b,$a2 # a^b, b^c in next round + ror \$$Sigma1[0],$a0 # Sigma1(e) + mov $b,$h + + and $a2,$a3 + ror \$$Sigma0[0],$a1 # Sigma0(a) + add $a0,$T1 # T1+=Sigma1(e) + + xor $a3,$h # h=Maj(a,b,c)=Ch(a^b,c,b) + add $T1,$d # d+=T1 + add $T1,$h # h+=T1 + + lea $STRIDE($Tbl),$Tbl # round++ +___ +$code.=<<___ if ($i<15); + add $a1,$h # h+=Sigma0(a) +___ + ($a2,$a3) = ($a3,$a2); +} + +sub ROUND_16_XX() +{ my ($i,$a,$b,$c,$d,$e,$f,$g,$h) = @_; + +$code.=<<___; + mov `$SZ*(($i+1)&0xf)`(%rsp),$a0 + mov `$SZ*(($i+14)&0xf)`(%rsp),$a2 + + mov $a0,$T1 + ror \$`$sigma0[1]-$sigma0[0]`,$a0 + add $a1,$a # modulo-scheduled h+=Sigma0(a) + mov $a2,$a1 + ror \$`$sigma1[1]-$sigma1[0]`,$a2 + + xor $T1,$a0 + shr \$$sigma0[2],$T1 + ror \$$sigma0[0],$a0 + xor $a1,$a2 + shr \$$sigma1[2],$a1 + + ror \$$sigma1[0],$a2 + xor $a0,$T1 # sigma0(X[(i+1)&0xf]) + xor $a1,$a2 # sigma1(X[(i+14)&0xf]) + add `$SZ*(($i+9)&0xf)`(%rsp),$T1 + + add `$SZ*($i&0xf)`(%rsp),$T1 + mov $e,$a0 + add $a2,$T1 + mov $a,$a1 +___ + &ROUND_00_15(@_); +} + +$code=<<___; +.text + +.extern OPENSSL_ia32cap_P +.globl $func +.type $func,\@function,3,"unwind" +.align 16 +$func: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +___ +$code.=<<___ if ($SZ==4 || $avx); + lea OPENSSL_ia32cap_P(%rip),%rax + mov 0(%rax),%r9d + mov 4(%rax),%r10d + mov 8(%rax),%eax +___ +$code.=<<___ if ($SZ==4 && $shaext); + test \$`1<<29`,%eax # check for SHA + jnz .Lshaext_shortcut +___ +$code.=<<___ if ($avx && $SZ==8); + test \$`1<<11`,%r10d # check for XOP + jnz .Lxop_shortcut +___ +$code.=<<___ if ($avx>1); + and \$`1<<8|1<<5|1<<3`,%eax # check for BMI2+AVX2+BMI1 + cmp \$`1<<8|1<<5|1<<3`,%eax + je .Lavx2_shortcut +___ +$code.=<<___ if ($avx); + and \$`1<<30`,%r9d # mask "Intel CPU" bit + and \$`1<<28|1<<9`,%r10d # mask AVX and SSSE3 bits + or %r9d,%r10d + cmp \$`1<<28|1<<9|1<<30`,%r10d + je .Lavx_shortcut +___ +$code.=<<___ if ($SZ==4); + test \$`1<<9`,%r10d + jnz .Lssse3_shortcut +___ +$code.=<<___; + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + shl \$4,%rdx # num*16 + sub \$$framesz,%rsp +.cfi_alloca $framesz +.cfi_def_cfa %rsp +.cfi_end_prologue + lea ($inp,%rdx,$SZ),%rdx # inp+num*16*$SZ + mov $ctx,$_ctx # save ctx, 1st arg + mov $inp,$_inp # save inp, 2nd arh + mov %rdx,$_end # save end pointer, "3rd" arg + + mov $SZ*0($ctx),$A + mov $SZ*1($ctx),$B + mov $SZ*2($ctx),$C + mov $SZ*3($ctx),$D + mov $SZ*4($ctx),$E + mov $SZ*5($ctx),$F + mov $SZ*6($ctx),$G + mov $SZ*7($ctx),$H + jmp .Lloop + +.align 16 +.Lloop: + mov $B,$a3 + lea $TABLE(%rip),$Tbl + xor $C,$a3 # magic +___ + for($i=0;$i<16;$i++) { + $code.=" mov $SZ*$i($inp),$T1\n"; + $code.=" mov @ROT[4],$a0\n"; + $code.=" mov @ROT[0],$a1\n"; + $code.=" bswap $T1\n"; + &ROUND_00_15($i,@ROT); + unshift(@ROT,pop(@ROT)); + } +$code.=<<___; + jmp .Lrounds_16_xx +.align 16 +.Lrounds_16_xx: +___ + for(;$i<32;$i++) { + &ROUND_16_XX($i,@ROT); + unshift(@ROT,pop(@ROT)); + } + +$code.=<<___; + cmpb \$0,`$SZ-1`($Tbl) + jnz .Lrounds_16_xx + + mov $_ctx,$ctx + add $a1,$A # modulo-scheduled h+=Sigma0(a) + lea 16*$SZ($inp),$inp + + add $SZ*0($ctx),$A + add $SZ*1($ctx),$B + add $SZ*2($ctx),$C + add $SZ*3($ctx),$D + add $SZ*4($ctx),$E + add $SZ*5($ctx),$F + add $SZ*6($ctx),$G + add $SZ*7($ctx),$H + + cmp $_end,$inp + + mov $A,$SZ*0($ctx) + mov $B,$SZ*1($ctx) + mov $C,$SZ*2($ctx) + mov $D,$SZ*3($ctx) + mov $E,$SZ*4($ctx) + mov $F,$SZ*5($ctx) + mov $G,$SZ*6($ctx) + mov $H,$SZ*7($ctx) + jb .Lloop + + lea $framesz+6*8(%rsp),%r11 +.cfi_def_cfa %r11,8 + mov $framesz(%rsp),%r15 + mov -40(%r11),%r14 + mov -32(%r11),%r13 + mov -24(%r11),%r12 + mov -16(%r11),%rbx + mov -8(%r11),%rbp +.cfi_epilogue + lea (%r11),%rsp + ret +.cfi_endproc +.size $func,.-$func +___ + +if ($SZ==4) { +$code.=<<___; +.align 64 +.type $TABLE,\@object +$TABLE: + .long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 + .long 0x428a2f98,0x71374491,0xb5c0fbcf,0xe9b5dba5 + .long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 + .long 0x3956c25b,0x59f111f1,0x923f82a4,0xab1c5ed5 + .long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 + .long 0xd807aa98,0x12835b01,0x243185be,0x550c7dc3 + .long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 + .long 0x72be5d74,0x80deb1fe,0x9bdc06a7,0xc19bf174 + .long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc + .long 0xe49b69c1,0xefbe4786,0x0fc19dc6,0x240ca1cc + .long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da + .long 0x2de92c6f,0x4a7484aa,0x5cb0a9dc,0x76f988da + .long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 + .long 0x983e5152,0xa831c66d,0xb00327c8,0xbf597fc7 + .long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 + .long 0xc6e00bf3,0xd5a79147,0x06ca6351,0x14292967 + .long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 + .long 0x27b70a85,0x2e1b2138,0x4d2c6dfc,0x53380d13 + .long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 + .long 0x650a7354,0x766a0abb,0x81c2c92e,0x92722c85 + .long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 + .long 0xa2bfe8a1,0xa81a664b,0xc24b8b70,0xc76c51a3 + .long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 + .long 0xd192e819,0xd6990624,0xf40e3585,0x106aa070 + .long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 + .long 0x19a4c116,0x1e376c08,0x2748774c,0x34b0bcb5 + .long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 + .long 0x391c0cb3,0x4ed8aa4a,0x5b9cca4f,0x682e6ff3 + .long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 + .long 0x748f82ee,0x78a5636f,0x84c87814,0x8cc70208 + .long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 + .long 0x90befffa,0xa4506ceb,0xbef9a3f7,0xc67178f2 + + .long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f + .long 0x00010203,0x04050607,0x08090a0b,0x0c0d0e0f + .long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff + .long 0x03020100,0x0b0a0908,0xffffffff,0xffffffff + .long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 + .long 0xffffffff,0xffffffff,0x03020100,0x0b0a0908 + .asciz "SHA256 block transform for x86_64, CRYPTOGAMS by \@dot-asm" +___ +} else { +$code.=<<___; +.align 64 +.type $TABLE,\@object +$TABLE: + .quad 0x428a2f98d728ae22,0x7137449123ef65cd + .quad 0x428a2f98d728ae22,0x7137449123ef65cd + .quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc + .quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc + .quad 0x3956c25bf348b538,0x59f111f1b605d019 + .quad 0x3956c25bf348b538,0x59f111f1b605d019 + .quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 + .quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 + .quad 0xd807aa98a3030242,0x12835b0145706fbe + .quad 0xd807aa98a3030242,0x12835b0145706fbe + .quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 + .quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 + .quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 + .quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 + .quad 0x9bdc06a725c71235,0xc19bf174cf692694 + .quad 0x9bdc06a725c71235,0xc19bf174cf692694 + .quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 + .quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 + .quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 + .quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 + .quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 + .quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 + .quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 + .quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 + .quad 0x983e5152ee66dfab,0xa831c66d2db43210 + .quad 0x983e5152ee66dfab,0xa831c66d2db43210 + .quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 + .quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 + .quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 + .quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 + .quad 0x06ca6351e003826f,0x142929670a0e6e70 + .quad 0x06ca6351e003826f,0x142929670a0e6e70 + .quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 + .quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 + .quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df + .quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df + .quad 0x650a73548baf63de,0x766a0abb3c77b2a8 + .quad 0x650a73548baf63de,0x766a0abb3c77b2a8 + .quad 0x81c2c92e47edaee6,0x92722c851482353b + .quad 0x81c2c92e47edaee6,0x92722c851482353b + .quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 + .quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 + .quad 0xc24b8b70d0f89791,0xc76c51a30654be30 + .quad 0xc24b8b70d0f89791,0xc76c51a30654be30 + .quad 0xd192e819d6ef5218,0xd69906245565a910 + .quad 0xd192e819d6ef5218,0xd69906245565a910 + .quad 0xf40e35855771202a,0x106aa07032bbd1b8 + .quad 0xf40e35855771202a,0x106aa07032bbd1b8 + .quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 + .quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 + .quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 + .quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 + .quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb + .quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb + .quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 + .quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 + .quad 0x748f82ee5defb2fc,0x78a5636f43172f60 + .quad 0x748f82ee5defb2fc,0x78a5636f43172f60 + .quad 0x84c87814a1f0ab72,0x8cc702081a6439ec + .quad 0x84c87814a1f0ab72,0x8cc702081a6439ec + .quad 0x90befffa23631e28,0xa4506cebde82bde9 + .quad 0x90befffa23631e28,0xa4506cebde82bde9 + .quad 0xbef9a3f7b2c67915,0xc67178f2e372532b + .quad 0xbef9a3f7b2c67915,0xc67178f2e372532b + .quad 0xca273eceea26619c,0xd186b8c721c0c207 + .quad 0xca273eceea26619c,0xd186b8c721c0c207 + .quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 + .quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 + .quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 + .quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 + .quad 0x113f9804bef90dae,0x1b710b35131c471b + .quad 0x113f9804bef90dae,0x1b710b35131c471b + .quad 0x28db77f523047d84,0x32caab7b40c72493 + .quad 0x28db77f523047d84,0x32caab7b40c72493 + .quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c + .quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c + .quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a + .quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a + .quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + .quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + + .quad 0x0001020304050607,0x08090a0b0c0d0e0f + .quad 0x0001020304050607,0x08090a0b0c0d0e0f + +${TABLE}_nodup: + .quad 0x428a2f98d728ae22,0x7137449123ef65cd + .quad 0xb5c0fbcfec4d3b2f,0xe9b5dba58189dbbc + .quad 0x3956c25bf348b538,0x59f111f1b605d019 + .quad 0x923f82a4af194f9b,0xab1c5ed5da6d8118 + .quad 0xd807aa98a3030242,0x12835b0145706fbe + .quad 0x243185be4ee4b28c,0x550c7dc3d5ffb4e2 + .quad 0x72be5d74f27b896f,0x80deb1fe3b1696b1 + .quad 0x9bdc06a725c71235,0xc19bf174cf692694 + .quad 0xe49b69c19ef14ad2,0xefbe4786384f25e3 + .quad 0x0fc19dc68b8cd5b5,0x240ca1cc77ac9c65 + .quad 0x2de92c6f592b0275,0x4a7484aa6ea6e483 + .quad 0x5cb0a9dcbd41fbd4,0x76f988da831153b5 + .quad 0x983e5152ee66dfab,0xa831c66d2db43210 + .quad 0xb00327c898fb213f,0xbf597fc7beef0ee4 + .quad 0xc6e00bf33da88fc2,0xd5a79147930aa725 + .quad 0x06ca6351e003826f,0x142929670a0e6e70 + .quad 0x27b70a8546d22ffc,0x2e1b21385c26c926 + .quad 0x4d2c6dfc5ac42aed,0x53380d139d95b3df + .quad 0x650a73548baf63de,0x766a0abb3c77b2a8 + .quad 0x81c2c92e47edaee6,0x92722c851482353b + .quad 0xa2bfe8a14cf10364,0xa81a664bbc423001 + .quad 0xc24b8b70d0f89791,0xc76c51a30654be30 + .quad 0xd192e819d6ef5218,0xd69906245565a910 + .quad 0xf40e35855771202a,0x106aa07032bbd1b8 + .quad 0x19a4c116b8d2d0c8,0x1e376c085141ab53 + .quad 0x2748774cdf8eeb99,0x34b0bcb5e19b48a8 + .quad 0x391c0cb3c5c95a63,0x4ed8aa4ae3418acb + .quad 0x5b9cca4f7763e373,0x682e6ff3d6b2b8a3 + .quad 0x748f82ee5defb2fc,0x78a5636f43172f60 + .quad 0x84c87814a1f0ab72,0x8cc702081a6439ec + .quad 0x90befffa23631e28,0xa4506cebde82bde9 + .quad 0xbef9a3f7b2c67915,0xc67178f2e372532b + .quad 0xca273eceea26619c,0xd186b8c721c0c207 + .quad 0xeada7dd6cde0eb1e,0xf57d4f7fee6ed178 + .quad 0x06f067aa72176fba,0x0a637dc5a2c898a6 + .quad 0x113f9804bef90dae,0x1b710b35131c471b + .quad 0x28db77f523047d84,0x32caab7b40c72493 + .quad 0x3c9ebe0a15c9bebc,0x431d67c49c100d4c + .quad 0x4cc5d4becb3e42b6,0x597f299cfc657e2a + .quad 0x5fcb6fab3ad6faec,0x6c44198c4a475817 + + .asciz "SHA512 block transform for x86_64, CRYPTOGAMS by \@dot-asm" +___ +} + +###################################################################### +# SIMD code paths +# +if ($SZ==4 && $shaext) {{{ +###################################################################### +# Intel SHA Extensions implementation of SHA256 update function. +# +my ($ctx,$inp,$num,$Tbl)=("%rdi","%rsi","%rdx","%rcx"); + +my ($Wi,$ABEF,$CDGH,$TMP,$BSWAP,$ABEF_SAVE,$CDGH_SAVE)=map("%xmm$_",(0..2,7..10)); +my @MSG=map("%xmm$_",(3..6)); + +$code.=<<___; +.type sha256_block_data_order_shaext,\@function,3,"unwind" +.align 64 +sha256_block_data_order_shaext: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lshaext_shortcut: +___ +$code.=<<___ if ($win64); + sub \$0x50,%rsp +.cfi_alloca 0x50 + movaps %xmm6,-0x50(%rbp) + movaps %xmm7,-0x40(%rbp) + movaps %xmm8,-0x30(%rbp) + movaps %xmm9,-0x20(%rbp) + movaps %xmm10,-0x10(%rbp) +.cfi_offset %xmm6-%xmm10,-0x60 +___ +$code.=<<___; +.cfi_end_prologue + lea K256+0x80(%rip),$Tbl + movdqu ($ctx),$ABEF # DCBA + movdqu 16($ctx),$CDGH # HGFE + movdqa 0x200-0x80($Tbl),$TMP # byte swap mask + + pshufd \$0x1b,$ABEF,$Wi # ABCD + pshufd \$0xb1,$ABEF,$ABEF # CDAB + pshufd \$0x1b,$CDGH,$CDGH # EFGH + movdqa $TMP,$BSWAP # offload + palignr \$8,$CDGH,$ABEF # ABEF + punpcklqdq $Wi,$CDGH # CDGH + jmp .Loop_shaext + +.align 16 +.Loop_shaext: + movdqu ($inp),@MSG[0] + movdqu 0x10($inp),@MSG[1] + movdqu 0x20($inp),@MSG[2] + pshufb $TMP,@MSG[0] + movdqu 0x30($inp),@MSG[3] + + movdqa 0*32-0x80($Tbl),$Wi + paddd @MSG[0],$Wi + pshufb $TMP,@MSG[1] + movdqa $CDGH,$CDGH_SAVE # offload + sha256rnds2 $ABEF,$CDGH # 0-3 + pshufd \$0x0e,$Wi,$Wi + nop + movdqa $ABEF,$ABEF_SAVE # offload + sha256rnds2 $CDGH,$ABEF + + movdqa 1*32-0x80($Tbl),$Wi + paddd @MSG[1],$Wi + pshufb $TMP,@MSG[2] + sha256rnds2 $ABEF,$CDGH # 4-7 + pshufd \$0x0e,$Wi,$Wi + lea 0x40($inp),$inp + sha256msg1 @MSG[1],@MSG[0] + sha256rnds2 $CDGH,$ABEF + + movdqa 2*32-0x80($Tbl),$Wi + paddd @MSG[2],$Wi + pshufb $TMP,@MSG[3] + sha256rnds2 $ABEF,$CDGH # 8-11 + pshufd \$0x0e,$Wi,$Wi + movdqa @MSG[3],$TMP + palignr \$4,@MSG[2],$TMP + nop + paddd $TMP,@MSG[0] + sha256msg1 @MSG[2],@MSG[1] + sha256rnds2 $CDGH,$ABEF + + movdqa 3*32-0x80($Tbl),$Wi + paddd @MSG[3],$Wi + sha256msg2 @MSG[3],@MSG[0] + sha256rnds2 $ABEF,$CDGH # 12-15 + pshufd \$0x0e,$Wi,$Wi + movdqa @MSG[0],$TMP + palignr \$4,@MSG[3],$TMP + nop + paddd $TMP,@MSG[1] + sha256msg1 @MSG[3],@MSG[2] + sha256rnds2 $CDGH,$ABEF +___ +for($i=4;$i<16-3;$i++) { +$code.=<<___; + movdqa $i*32-0x80($Tbl),$Wi + paddd @MSG[0],$Wi + sha256msg2 @MSG[0],@MSG[1] + sha256rnds2 $ABEF,$CDGH # 16-19... + pshufd \$0x0e,$Wi,$Wi + movdqa @MSG[1],$TMP + palignr \$4,@MSG[0],$TMP + nop + paddd $TMP,@MSG[2] + sha256msg1 @MSG[0],@MSG[3] + sha256rnds2 $CDGH,$ABEF +___ + push(@MSG,shift(@MSG)); +} +$code.=<<___; + movdqa 13*32-0x80($Tbl),$Wi + paddd @MSG[0],$Wi + sha256msg2 @MSG[0],@MSG[1] + sha256rnds2 $ABEF,$CDGH # 52-55 + pshufd \$0x0e,$Wi,$Wi + movdqa @MSG[1],$TMP + palignr \$4,@MSG[0],$TMP + sha256rnds2 $CDGH,$ABEF + paddd $TMP,@MSG[2] + + movdqa 14*32-0x80($Tbl),$Wi + paddd @MSG[1],$Wi + sha256rnds2 $ABEF,$CDGH # 56-59 + pshufd \$0x0e,$Wi,$Wi + sha256msg2 @MSG[1],@MSG[2] + movdqa $BSWAP,$TMP + sha256rnds2 $CDGH,$ABEF + + movdqa 15*32-0x80($Tbl),$Wi + paddd @MSG[2],$Wi + nop + sha256rnds2 $ABEF,$CDGH # 60-63 + pshufd \$0x0e,$Wi,$Wi + dec $num + nop + sha256rnds2 $CDGH,$ABEF + + paddd $CDGH_SAVE,$CDGH + paddd $ABEF_SAVE,$ABEF + jnz .Loop_shaext + + pshufd \$0xb1,$CDGH,$CDGH # DCHG + pshufd \$0x1b,$ABEF,$TMP # FEBA + pshufd \$0xb1,$ABEF,$ABEF # BAFE + punpckhqdq $CDGH,$ABEF # DCBA + palignr \$8,$TMP,$CDGH # HGFE + + movdqu $ABEF,($ctx) + movdqu $CDGH,16($ctx) +___ +$code.=<<___ if ($win64); + movaps -0x50(%rbp),%xmm6 + movaps -0x40(%rbp),%xmm7 + movaps -0x30(%rbp),%xmm8 + movaps -0x20(%rbp),%xmm9 + movaps -0x10(%rbp),%xmm10 + mov %rbp,%rsp +___ +$code.=<<___; +.cfi_def_cfa_register %rsp + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size sha256_block_data_order_shaext,.-sha256_block_data_order_shaext +___ +}}} +if ($SZ==8 && $shaext && $avx>1) {{{ +###################################################################### +# Intel SHA Extensions implementation of SHA512 update function. +# +my ($ctx,$inp,$num,$Tbl)=("%rdi","%rsi","%rdx","%rcx"); + +my ($Wi,$ABEF,$CDGH,$TMP,$BSWAP,$ABEF_SAVE,$CDGH_SAVE)=map("%ymm$_",(4..10)); +my @MSG=map("%ymm$_",(0..3)); + +$code.=<<___; +.globl sha512_block_data_order_shaext +.type sha512_block_data_order_shaext,\@function,3,"unwind" +.align 64 +sha512_block_data_order_shaext: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lshaext_shortcut: +___ +$code.=<<___ if ($win64); + sub \$0x50,%rsp +.cfi_alloca 0x50 + movaps %xmm6,-0x50(%rbp) + movaps %xmm7,-0x40(%rbp) + movaps %xmm8,-0x30(%rbp) + movaps %xmm9,-0x20(%rbp) + movaps %xmm10,-0x10(%rbp) +.cfi_offset %xmm6-%xmm10,-0x60 +___ +$code.=<<___; +.cfi_end_prologue + lea K512_nodup+0x80(%rip),$Tbl + vmovdqu ($ctx),@MSG[0] # DCBA + vmovdqu 32($ctx),@MSG[1] # HGFE + vmovdqa -0xa0($Tbl),$BSWAP + + vpermq \$0b00011011,@MSG[0],@MSG[0] # ABCD + vpblendd \$0b00001111,@MSG[1],@MSG[0],$ABEF # ABFE + vpblendd \$0b00001111,@MSG[0],@MSG[1],$CDGH # HGCD + vpermq \$0b11100001,$ABEF,$ABEF # ABEF + vpermq \$0b01001011,$CDGH,$CDGH # CDGH + jmp .Loop_shaext + +.align 16 +.Loop_shaext: + vmovdqu ($inp),@MSG[0] + vmovdqu 0x20($inp),@MSG[1] + vmovdqu 0x40($inp),@MSG[2] + vpshufb $BSWAP,@MSG[0],@MSG[0] + vmovdqu 0x60($inp),@MSG[3] + + vpaddq 0*32-0x80($Tbl),@MSG[0],$Wi + vpshufb $BSWAP,@MSG[1],@MSG[1] + vmovdqa $CDGH,$CDGH_SAVE # offload + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 0-3 + vextracti128 \$1,$Wi,%x#$Wi + vmovdqa $ABEF,$ABEF_SAVE # offload + vsha512rnds2 %x#$Wi,$CDGH,$ABEF + + vpaddq 1*32-0x80($Tbl),@MSG[1],$Wi + vpshufb $BSWAP,@MSG[2],@MSG[2] + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 4-7 + vextracti128 \$1,$Wi,%x#$Wi + lea 0x80($inp),$inp + vsha512msg1 @MSG[1],@MSG[0] + vsha512rnds2 %x#$Wi,$CDGH,$ABEF + + vpaddq 2*32-0x80($Tbl),@MSG[2],$Wi + vpshufb $BSWAP,@MSG[3],@MSG[3] + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 8-11 + vextracti128 \$1,$Wi,%x#$Wi + vpblendd \$0x03,@MSG[3],@MSG[2],$TMP + vpermq \$0x39,$TMP,$TMP + vpaddq $TMP,@MSG[0],@MSG[0] + vsha512msg1 @MSG[2],@MSG[1] + vsha512rnds2 %x#$Wi,$CDGH,$ABEF + + vpaddq 3*32-0x80($Tbl),@MSG[3],$Wi + vsha512msg2 @MSG[3],@MSG[0] + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 12-15 + vextracti128 \$1,$Wi,%x#$Wi + vpblendd \$0x03,@MSG[0],@MSG[3],$TMP + vpermq \$0x39,$TMP,$TMP + vpaddq $TMP,@MSG[1],@MSG[1] + vsha512msg1 @MSG[3],@MSG[2] + vsha512rnds2 %x#$Wi,$CDGH,$ABEF +___ +for($i=4;$i<20-3;$i++) { +$code.=<<___; + vpaddq $i*32-0x80($Tbl),@MSG[0],$Wi + vsha512msg2 @MSG[0],@MSG[1] + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 16-19... + vextracti128 \$1,$Wi,%x#$Wi + vpblendd \$0x03,@MSG[1],@MSG[0],$TMP + vpermq \$0x39,$TMP,$TMP + vpaddq $TMP,@MSG[2],@MSG[2] + vsha512msg1 @MSG[0],@MSG[3] + vsha512rnds2 %x#$Wi,$CDGH,$ABEF +___ + push(@MSG,shift(@MSG)); +} +$code.=<<___; + vpaddq 17*32-0x80($Tbl),@MSG[0],$Wi + vsha512msg2 @MSG[0],@MSG[1] + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 68-71 + vextracti128 \$1,$Wi,%x#$Wi + vpblendd \$0x03,@MSG[1],@MSG[0],$TMP + vpermq \$0x39,$TMP,$TMP + vsha512rnds2 %x#$Wi,$CDGH,$ABEF + vpaddq $TMP,@MSG[2],@MSG[2] + + vpaddq 18*32-0x80($Tbl),@MSG[1],$Wi + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 72-75 + vextracti128 \$1,$Wi,%x#$Wi + vsha512msg2 @MSG[1],@MSG[2] + vsha512rnds2 %x#$Wi,$CDGH,$ABEF + + vpaddq 19*32-0x80($Tbl),@MSG[2],$Wi + vsha512rnds2 %x#$Wi,$ABEF,$CDGH # 76-79 + vextracti128 \$1,$Wi,%x#$Wi + dec $num + vsha512rnds2 %x#$Wi,$CDGH,$ABEF + + vpaddq $CDGH_SAVE,$CDGH,$CDGH + vpaddq $ABEF_SAVE,$ABEF,$ABEF + jnz .Loop_shaext + + vpermq \$0b01001011,$ABEF,$ABEF # EFBA + vpblendd \$0b11110000,$CDGH,$ABEF,@MSG[0] # CDBA + vpblendd \$0b11110000,$ABEF,$CDGH,@MSG[1] # EFGH + vpermq \$0b10110100,@MSG[0],@MSG[0] # DCBA + vpermq \$0b00011011,@MSG[1],@MSG[1] # HGFE + + vmovdqu @MSG[0],($ctx) + vmovdqu @MSG[1],32($ctx) + + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0x50(%rbp),%xmm6 + movaps -0x40(%rbp),%xmm7 + movaps -0x30(%rbp),%xmm8 + movaps -0x20(%rbp),%xmm9 + movaps -0x10(%rbp),%xmm10 + mov %rbp,%rsp +___ +$code.=<<___; +.cfi_def_cfa_register %rsp + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size sha512_block_data_order_shaext,.-sha512_block_data_order_shaext +___ +}}} +{{{ + +my $a4=$T1; +my ($a,$b,$c,$d,$e,$f,$g,$h); + +sub AUTOLOAD() # thunk [simplified] 32-bit style perlasm +{ my $opcode = $AUTOLOAD; $opcode =~ s/.*:://; + my $arg = pop; + $arg = "\$$arg" if ($arg*1 eq $arg); + $code .= "\t$opcode\t".join(',',$arg,reverse @_)."\n"; +} + +sub body_00_15 () { + ( + '($a,$b,$c,$d,$e,$f,$g,$h)=@ROT;'. + + '&ror ($a0,$Sigma1[2]-$Sigma1[1])', + '&mov ($a,$a1)', + '&mov ($a4,$f)', + + '&ror ($a1,$Sigma0[2]-$Sigma0[1])', + '&xor ($a0,$e)', + '&xor ($a4,$g)', # f^g + + '&ror ($a0,$Sigma1[1]-$Sigma1[0])', + '&xor ($a1,$a)', + '&and ($a4,$e)', # (f^g)&e + + '&xor ($a0,$e)', + '&add ($h,$SZ*($i&15)."(%rsp)")', # h+=X[i]+K[i] + '&mov ($a2,$a)', + + '&xor ($a4,$g)', # Ch(e,f,g)=((f^g)&e)^g + '&ror ($a1,$Sigma0[1]-$Sigma0[0])', + '&xor ($a2,$b)', # a^b, b^c in next round + + '&add ($h,$a4)', # h+=Ch(e,f,g) + '&ror ($a0,$Sigma1[0])', # Sigma1(e) + '&and ($a3,$a2)', # (b^c)&(a^b) + + '&xor ($a1,$a)', + '&add ($h,$a0)', # h+=Sigma1(e) + '&xor ($a3,$b)', # Maj(a,b,c)=Ch(a^b,c,b) + + '&ror ($a1,$Sigma0[0])', # Sigma0(a) + '&add ($d,$h)', # d+=h + '&add ($h,$a3)', # h+=Maj(a,b,c) + + '&mov ($a0,$d)', + '&add ($a1,$h);'. # h+=Sigma0(a) + '($a2,$a3) = ($a3,$a2); unshift(@ROT,pop(@ROT)); $i++;' + ); +} + +###################################################################### +# SSSE3 code path +# +if ($SZ==4) { # SHA256 only +my $Tbl = $inp; +my $_ctx="-64(%rbp)"; +my $_inp="-56(%rbp)"; +my $_end="-48(%rbp)"; +my $framesz=3*8+$win64*16*4; + +my @X = map("%xmm$_",(0..3)); +my ($t0,$t1,$t2,$t3, $t4,$t5) = map("%xmm$_",(4..9)); + +$code.=<<___; +.type ${func}_ssse3,\@function,3,"unwind" +.align 64 +${func}_ssse3: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lssse3_shortcut: + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + shl \$4,%rdx # num*16 + sub \$$framesz,%rsp +.cfi_alloca $framesz + lea ($inp,%rdx,$SZ),%rdx # inp+num*16*$SZ + mov $ctx,$_ctx # save ctx, 1st arg + #mov $inp,$_inp # save inp, 2nd arg + mov %rdx,$_end # save end pointer, "3rd" arg +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0x80(%rbp) + movaps %xmm7,-0x70(%rbp) + movaps %xmm8,-0x60(%rbp) + movaps %xmm9,-0x50(%rbp) +.cfi_offset %xmm6-%xmm9,-0x90 +___ +$code.=<<___; +.cfi_end_prologue + + lea -16*$SZ(%rsp),%rsp + mov $SZ*0($ctx),$A + and \$-64,%rsp # align stack + mov $SZ*1($ctx),$B + mov $SZ*2($ctx),$C + mov $SZ*3($ctx),$D + mov $SZ*4($ctx),$E + mov $SZ*5($ctx),$F + mov $SZ*6($ctx),$G + mov $SZ*7($ctx),$H +___ + +$code.=<<___; + #movdqa $TABLE+`$SZ*2*$rounds`+32(%rip),$t4 + #movdqa $TABLE+`$SZ*2*$rounds`+64(%rip),$t5 + jmp .Lloop_ssse3 +.align 16 +.Lloop_ssse3: + movdqa $TABLE+`$SZ*2*$rounds`(%rip),$t3 + mov $inp,$_inp # offload $inp + movdqu 0x00($inp),@X[0] + movdqu 0x10($inp),@X[1] + movdqu 0x20($inp),@X[2] + pshufb $t3,@X[0] + movdqu 0x30($inp),@X[3] + lea $TABLE(%rip),$Tbl + pshufb $t3,@X[1] + movdqa 0x00($Tbl),$t0 + movdqa 0x20($Tbl),$t1 + pshufb $t3,@X[2] + paddd @X[0],$t0 + movdqa 0x40($Tbl),$t2 + pshufb $t3,@X[3] + movdqa 0x60($Tbl),$t3 + paddd @X[1],$t1 + paddd @X[2],$t2 + paddd @X[3],$t3 + movdqa $t0,0x00(%rsp) + mov $A,$a1 + movdqa $t1,0x10(%rsp) + mov $B,$a3 + movdqa $t2,0x20(%rsp) + xor $C,$a3 # magic + movdqa $t3,0x30(%rsp) + mov $E,$a0 + jmp .Lssse3_00_47 + +.align 16 +.Lssse3_00_47: + sub \$`-16*2*$SZ`,$Tbl # size optimization +___ +sub Xupdate_256_SSSE3 () { + ( + '&movdqa ($t0,@X[1]);', + '&movdqa ($t3,@X[3])', + '&palignr ($t0,@X[0],$SZ)', # X[1..4] + '&palignr ($t3,@X[2],$SZ);', # X[9..12] + '&movdqa ($t1,$t0)', + '&movdqa ($t2,$t0);', + '&psrld ($t0,$sigma0[2])', + '&paddd (@X[0],$t3);', # X[0..3] += X[9..12] + '&psrld ($t2,$sigma0[0])', + '&pshufd ($t3,@X[3],0b11111010)',# X[14..15] + '&pslld ($t1,8*$SZ-$sigma0[1]);'. + '&pxor ($t0,$t2)', + '&psrld ($t2,$sigma0[1]-$sigma0[0]);'. + '&pxor ($t0,$t1)', + '&pslld ($t1,$sigma0[1]-$sigma0[0]);'. + '&pxor ($t0,$t2);', + '&movdqa ($t2,$t3)', + '&pxor ($t0,$t1);', # sigma0(X[1..4]) + '&psrld ($t3,$sigma1[2])', + '&paddd (@X[0],$t0);', # X[0..3] += sigma0(X[1..4]) + '&psrlq ($t2,$sigma1[0])', + '&pxor ($t3,$t2);', + '&psrlq ($t2,$sigma1[1]-$sigma1[0])', + '&pxor ($t3,$t2)', + '&pshufb ($t3,$t4)', # sigma1(X[14..15]) + '&paddd (@X[0],$t3)', # X[0..1] += sigma1(X[14..15]) + '&pshufd ($t3,@X[0],0b01010000)',# X[16..17] + '&movdqa ($t2,$t3);', + '&psrld ($t3,$sigma1[2])', + '&psrlq ($t2,$sigma1[0])', + '&pxor ($t3,$t2);', + '&psrlq ($t2,$sigma1[1]-$sigma1[0])', + '&pxor ($t3,$t2);', + '&movdqa ($t2,16*2*$j."($Tbl)")', + '&pshufb ($t3,$t5)', + '&paddd (@X[0],$t3)' # X[2..3] += sigma1(X[16..17]) + ); +} + +sub SSSE3_256_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body,&$body,&$body); # 104 instructions + + if (0) { + foreach (Xupdate_256_SSSE3()) { # 36 instructions + eval; + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + } + } else { # squeeze extra 4% on Westmere and 19% on Atom + eval(shift(@insns)); #@ + &movdqa ($t0,@X[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &movdqa ($t3,@X[3]); + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + eval(shift(@insns)); + &palignr ($t0,@X[0],$SZ); # X[1..4] + eval(shift(@insns)); + eval(shift(@insns)); + &palignr ($t3,@X[2],$SZ); # X[9..12] + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + &movdqa ($t1,$t0); + eval(shift(@insns)); + eval(shift(@insns)); + &movdqa ($t2,$t0); + eval(shift(@insns)); #@ + eval(shift(@insns)); + &psrld ($t0,$sigma0[2]); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &paddd (@X[0],$t3); # X[0..3] += X[9..12] + eval(shift(@insns)); #@ + eval(shift(@insns)); + &psrld ($t2,$sigma0[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &pshufd ($t3,@X[3],0b11111010); # X[4..15] + eval(shift(@insns)); + eval(shift(@insns)); #@ + &pslld ($t1,8*$SZ-$sigma0[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t0,$t2); + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + &psrld ($t2,$sigma0[1]-$sigma0[0]); + eval(shift(@insns)); + &pxor ($t0,$t1); + eval(shift(@insns)); + eval(shift(@insns)); + &pslld ($t1,$sigma0[1]-$sigma0[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t0,$t2); + eval(shift(@insns)); + eval(shift(@insns)); #@ + &movdqa ($t2,$t3); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t0,$t1); # sigma0(X[1..4]) + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + &psrld ($t3,$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &paddd (@X[0],$t0); # X[0..3] += sigma0(X[1..4]) + eval(shift(@insns)); #@ + eval(shift(@insns)); + &psrlq ($t2,$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t3,$t2); + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + &psrlq ($t2,$sigma1[1]-$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t3,$t2); + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + #&pshufb ($t3,$t4); # sigma1(X[14..15]) + &pshufd ($t3,$t3,0b10000000); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &psrldq ($t3,8); + eval(shift(@insns)); + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + &paddd (@X[0],$t3); # X[0..1] += sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &pshufd ($t3,@X[0],0b01010000); # X[16..17] + eval(shift(@insns)); + eval(shift(@insns)); #@ + eval(shift(@insns)); + &movdqa ($t2,$t3); + eval(shift(@insns)); + eval(shift(@insns)); + &psrld ($t3,$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); #@ + &psrlq ($t2,$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t3,$t2); + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + eval(shift(@insns)); + &psrlq ($t2,$sigma1[1]-$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &pxor ($t3,$t2); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); #@ + #&pshufb ($t3,$t5); + &pshufd ($t3,$t3,0b00001000); + eval(shift(@insns)); + eval(shift(@insns)); + &movdqa ($t2,16*2*$j."($Tbl)"); + eval(shift(@insns)); #@ + eval(shift(@insns)); + &pslldq ($t3,8); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &paddd (@X[0],$t3); # X[2..3] += sigma1(X[16..17]) + eval(shift(@insns)); #@ + eval(shift(@insns)); + eval(shift(@insns)); + } + &paddd ($t2,@X[0]); + foreach (@insns) { eval; } # remaining instructions + &movdqa (16*$j."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<4; $j++) { + &SSSE3_256_00_47($j,\&body_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &cmpb ($SZ-1+16*2*$SZ."($Tbl)",0); + &jne (".Lssse3_00_47"); + + for ($i=0; $i<16; ) { + foreach(body_00_15()) { eval; } + } +$code.=<<___; + mov $_ctx,$ctx + mov $a1,$A + mov $_inp,$inp + + add $SZ*0($ctx),$A + add $SZ*1($ctx),$B + add $SZ*2($ctx),$C + add $SZ*3($ctx),$D + add $SZ*4($ctx),$E + add $SZ*5($ctx),$F + add $SZ*6($ctx),$G + add $SZ*7($ctx),$H + + lea 16*$SZ($inp),$inp + cmp $_end,$inp + + mov $A,$SZ*0($ctx) + mov $B,$SZ*1($ctx) + mov $C,$SZ*2($ctx) + mov $D,$SZ*3($ctx) + mov $E,$SZ*4($ctx) + mov $F,$SZ*5($ctx) + mov $G,$SZ*6($ctx) + mov $H,$SZ*7($ctx) + jb .Lloop_ssse3 + +___ +$code.=<<___ if ($win64); + movaps -0x80(%rbp),%xmm6 + movaps -0x70(%rbp),%xmm7 + movaps -0x60(%rbp),%xmm8 + movaps -0x50(%rbp),%xmm9 +___ +$code.=<<___; + mov -40(%rbp),%r15 + mov -32(%rbp),%r14 + mov -24(%rbp),%r13 + mov -16(%rbp),%r12 + mov -8(%rbp),%rbx + mov %rbp,%rsp +.cfi_def_cfa_register %rsp + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size ${func}_ssse3,.-${func}_ssse3 +___ +} + +if ($avx) {{ +###################################################################### +# XOP code path +# +if ($SZ==8) { # SHA512 only +my $Tbl=$inp; +my $_ctx="-64(%rbp)"; +my $_inp="-56(%rbp)"; +my $_end="-48(%rbp)"; +my $framesz=3*8+$win64*16*6; + +$code.=<<___; +.type ${func}_xop,\@function,3,"unwind" +.align 64 +${func}_xop: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lxop_shortcut: + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + shl \$4,%rdx # num*16 + sub \$$framesz,%rsp +.cfi_alloca $framesz + lea ($inp,%rdx,$SZ),%rdx # inp+num*16*$SZ + mov $ctx,$_ctx # save ctx, 1st arg + #mov $inp,$_inp # save inp, 2nd arg + mov %rdx,$_end # save end pointer, "3rd" arg +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0xa0(%rbp) + movaps %xmm7,-0x90(%rbp) + movaps %xmm8,-0x80(%rbp) + movaps %xmm9,-0x70(%rbp) +.cfi_offset %xmm6-%xmm9,-0xb0 +___ +$code.=<<___ if ($win64 && $SZ>4); + movaps %xmm10,-0x60(%rbp) + movaps %xmm11,-0x50(%rbp) +.cfi_offset %xmm10-%xmm11,-0x70 +___ +$code.=<<___; +.cfi_end_prologue + + lea -16*$SZ(%rsp),%rsp + vzeroupper + and \$-64,%rsp # align stack + mov $SZ*0($ctx),$A + mov $SZ*1($ctx),$B + mov $SZ*2($ctx),$C + mov $SZ*3($ctx),$D + mov $SZ*4($ctx),$E + mov $SZ*5($ctx),$F + mov $SZ*6($ctx),$G + mov $SZ*7($ctx),$H + jmp .Lloop_xop +___ + if ($SZ==4) { # SHA256 + my @X = map("%xmm$_",(0..3)); + my ($t0,$t1,$t2,$t3) = map("%xmm$_",(4..7)); + +$code.=<<___; +.align 16 +.Lloop_xop: + vmovdqa $TABLE+`$SZ*2*$rounds`(%rip),$t3 + mov $inp,$_inp # offload $inp + vmovdqu 0x00($inp),@X[0] + vmovdqu 0x10($inp),@X[1] + vmovdqu 0x20($inp),@X[2] + vmovdqu 0x30($inp),@X[3] + vpshufb $t3,@X[0],@X[0] + lea $TABLE(%rip),$Tbl + vpshufb $t3,@X[1],@X[1] + vpshufb $t3,@X[2],@X[2] + vpaddd 0x00($Tbl),@X[0],$t0 + vpshufb $t3,@X[3],@X[3] + vpaddd 0x20($Tbl),@X[1],$t1 + vpaddd 0x40($Tbl),@X[2],$t2 + vpaddd 0x60($Tbl),@X[3],$t3 + vmovdqa $t0,0x00(%rsp) + mov $A,$a1 + vmovdqa $t1,0x10(%rsp) + mov $B,$a3 + vmovdqa $t2,0x20(%rsp) + xor $C,$a3 # magic + vmovdqa $t3,0x30(%rsp) + mov $E,$a0 + jmp .Lxop_00_47 + +.align 16 +.Lxop_00_47: + sub \$`-16*2*$SZ`,$Tbl # size optimization +___ +sub XOP_256_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body,&$body,&$body); # 104 instructions + + &vpalignr ($t0,@X[1],@X[0],$SZ); # X[1..4] + eval(shift(@insns)); + eval(shift(@insns)); + &vpalignr ($t3,@X[3],@X[2],$SZ); # X[9..12] + eval(shift(@insns)); + eval(shift(@insns)); + &vprotd ($t1,$t0,8*$SZ-$sigma0[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpsrld ($t0,$t0,$sigma0[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddd (@X[0],@X[0],$t3); # X[0..3] += X[9..12] + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vprotd ($t2,$t1,$sigma0[1]-$sigma0[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t0,$t0,$t1); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vprotd ($t3,@X[3],8*$SZ-$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t0,$t0,$t2); # sigma0(X[1..4]) + eval(shift(@insns)); + eval(shift(@insns)); + &vpsrld ($t2,@X[3],$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddd (@X[0],@X[0],$t0); # X[0..3] += sigma0(X[1..4]) + eval(shift(@insns)); + eval(shift(@insns)); + &vprotd ($t1,$t3,$sigma1[1]-$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t3,$t3,$t2); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t3,$t3,$t1); # sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpsrldq ($t3,$t3,8); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddd (@X[0],@X[0],$t3); # X[0..1] += sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vprotd ($t3,@X[0],8*$SZ-$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpsrld ($t2,@X[0],$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &vprotd ($t1,$t3,$sigma1[1]-$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t3,$t3,$t2); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t3,$t3,$t1); # sigma1(X[16..17]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpslldq ($t3,$t3,8); # 22 instructions + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddd (@X[0],@X[0],$t3); # X[2..3] += sigma1(X[16..17]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddd ($t2,@X[0],16*2*$j."($Tbl)"); + foreach (@insns) { eval; } # remaining instructions + &vmovdqa (16*$j."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<4; $j++) { + &XOP_256_00_47($j,\&body_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &cmpb ($SZ-1+16*2*$SZ."($Tbl)",0); + &jne (".Lxop_00_47"); + + for ($i=0; $i<16; ) { + foreach(body_00_15()) { eval; } + } + + } else { # SHA512 + my @X = map("%xmm$_",(0..7)); + my ($t0,$t1,$t2,$t3) = map("%xmm$_",(8..11)); + +$code.=<<___; +.align 16 +.Lloop_xop: + vmovdqa $TABLE+`$SZ*2*$rounds`(%rip),$t3 + mov $inp,$_inp # offload $inp + vmovdqu 0x00($inp),@X[0] + vmovdqu 0x10($inp),@X[1] + vmovdqu 0x20($inp),@X[2] + vpshufb $t3,@X[0],@X[0] + vmovdqu 0x30($inp),@X[3] + vpshufb $t3,@X[1],@X[1] + vmovdqu 0x40($inp),@X[4] + vpshufb $t3,@X[2],@X[2] + vmovdqu 0x50($inp),@X[5] + vpshufb $t3,@X[3],@X[3] + vmovdqu 0x60($inp),@X[6] + vpshufb $t3,@X[4],@X[4] + vmovdqu 0x70($inp),@X[7] + lea $TABLE+0x80(%rip),$Tbl # size optimization + vpshufb $t3,@X[5],@X[5] + vpaddq -0x80($Tbl),@X[0],$t0 + vpshufb $t3,@X[6],@X[6] + vpaddq -0x60($Tbl),@X[1],$t1 + vpshufb $t3,@X[7],@X[7] + vpaddq -0x40($Tbl),@X[2],$t2 + vpaddq -0x20($Tbl),@X[3],$t3 + vmovdqa $t0,0x00(%rsp) + vpaddq 0x00($Tbl),@X[4],$t0 + vmovdqa $t1,0x10(%rsp) + vpaddq 0x20($Tbl),@X[5],$t1 + vmovdqa $t2,0x20(%rsp) + vpaddq 0x40($Tbl),@X[6],$t2 + vmovdqa $t3,0x30(%rsp) + vpaddq 0x60($Tbl),@X[7],$t3 + vmovdqa $t0,0x40(%rsp) + mov $A,$a1 + vmovdqa $t1,0x50(%rsp) + mov $B,$a3 + vmovdqa $t2,0x60(%rsp) + xor $C,$a3 # magic + vmovdqa $t3,0x70(%rsp) + mov $E,$a0 + jmp .Lxop_00_47 + +.align 16 +.Lxop_00_47: + add \$`16*2*$SZ`,$Tbl +___ +sub XOP_512_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body); # 52 instructions + + &vpalignr ($t0,@X[1],@X[0],$SZ); # X[1..2] + eval(shift(@insns)); + eval(shift(@insns)); + &vpalignr ($t3,@X[5],@X[4],$SZ); # X[9..10] + eval(shift(@insns)); + eval(shift(@insns)); + &vprotq ($t1,$t0,8*$SZ-$sigma0[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpsrlq ($t0,$t0,$sigma0[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddq (@X[0],@X[0],$t3); # X[0..1] += X[9..10] + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vprotq ($t2,$t1,$sigma0[1]-$sigma0[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t0,$t0,$t1); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vprotq ($t3,@X[7],8*$SZ-$sigma1[1]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t0,$t0,$t2); # sigma0(X[1..2]) + eval(shift(@insns)); + eval(shift(@insns)); + &vpsrlq ($t2,@X[7],$sigma1[2]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddq (@X[0],@X[0],$t0); # X[0..1] += sigma0(X[1..2]) + eval(shift(@insns)); + eval(shift(@insns)); + &vprotq ($t1,$t3,$sigma1[1]-$sigma1[0]); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t3,$t3,$t2); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpxor ($t3,$t3,$t1); # sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddq (@X[0],@X[0],$t3); # X[0..1] += sigma1(X[14..15]) + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + &vpaddq ($t2,@X[0],16*2*$j-0x80."($Tbl)"); + foreach (@insns) { eval; } # remaining instructions + &vmovdqa (16*$j."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<8; $j++) { + &XOP_512_00_47($j,\&body_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &cmpb ($SZ-1+16*2*$SZ-0x80."($Tbl)",0); + &jne (".Lxop_00_47"); + + for ($i=0; $i<16; ) { + foreach(body_00_15()) { eval; } + } +} +$code.=<<___; + mov $_ctx,$ctx + mov $a1,$A + mov $_inp,$inp + + add $SZ*0($ctx),$A + add $SZ*1($ctx),$B + add $SZ*2($ctx),$C + add $SZ*3($ctx),$D + add $SZ*4($ctx),$E + add $SZ*5($ctx),$F + add $SZ*6($ctx),$G + add $SZ*7($ctx),$H + + lea 16*$SZ($inp),$inp + cmp $_end,$inp + + mov $A,$SZ*0($ctx) + mov $B,$SZ*1($ctx) + mov $C,$SZ*2($ctx) + mov $D,$SZ*3($ctx) + mov $E,$SZ*4($ctx) + mov $F,$SZ*5($ctx) + mov $G,$SZ*6($ctx) + mov $H,$SZ*7($ctx) + jb .Lloop_xop + + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0xa0(%rbp),%xmm6 + movaps -0x90(%rbp),%xmm7 + movaps -0x80(%rbp),%xmm8 + movaps -0x70(%rbp),%xmm9 +___ +$code.=<<___ if ($win64 && $SZ>4); + movaps -0x60(%rbp),%xmm10 + movaps -0x50(%rbp),%xmm11 +___ +$code.=<<___; + mov -40(%rbp),%r15 + mov -32(%rbp),%r14 + mov -24(%rbp),%r13 + mov -16(%rbp),%r12 + mov -8(%rbp),%rbx + mov %rbp,%rsp +.cfi_def_cfa_register %rsp + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size ${func}_xop,.-${func}_xop +___ +} +###################################################################### +# AVX+shrd code path +# +my $Tbl=$inp; +my $_ctx="-64(%rbp)"; +my $_inp="-56(%rbp)"; +my $_end="-48(%rbp)"; +my $framesz=3*8+$win64*16*6; + +local *ror = sub { &shrd(@_[0],@_) }; + +$code.=<<___; +.type ${func}_avx,\@function,3,"unwind" +.align 64 +${func}_avx: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lavx_shortcut: + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + shl \$4,%rdx # num*16 + sub \$$framesz,%rsp +.cfi_alloca $framesz + lea ($inp,%rdx,$SZ),%rdx # inp+num*16*$SZ + mov $ctx,$_ctx # save ctx, 1st arg + #mov $inp,$_inp # save inp, 2nd arg + mov %rdx,$_end # save end pointer, "3rd" arg +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0xa0(%rbp) + movaps %xmm7,-0x90(%rbp) + movaps %xmm8,-0x80(%rbp) + movaps %xmm9,-0x70(%rbp) +.cfi_offset %xmm6-%xmm9,-0xb0 +___ +$code.=<<___ if ($win64 && $SZ>4); + movaps %xmm10,-0x60(%rbp) + movaps %xmm11,-0x50(%rbp) +.cfi_offset %xmm10-%xmm11,-0x70 +___ +$code.=<<___; +.cfi_end_prologue + + lea -16*$SZ(%rsp),%rsp + vzeroupper + and \$-64,%rsp # align stack + mov $SZ*0($ctx),$A + mov $SZ*1($ctx),$B + mov $SZ*2($ctx),$C + mov $SZ*3($ctx),$D + mov $SZ*4($ctx),$E + mov $SZ*5($ctx),$F + mov $SZ*6($ctx),$G + mov $SZ*7($ctx),$H +___ + if ($SZ==4) { # SHA256 + my @X = map("%xmm$_",(0..3)); + my ($t0,$t1,$t2,$t3, $t4,$t5) = map("%xmm$_",(4..9)); + +$code.=<<___; + vmovdqa $TABLE+`$SZ*2*$rounds`+32(%rip),$t4 + vmovdqa $TABLE+`$SZ*2*$rounds`+64(%rip),$t5 + jmp .Lloop_avx +.align 16 +.Lloop_avx: + vmovdqa $TABLE+`$SZ*2*$rounds`(%rip),$t3 + mov $inp,$_inp # offload $inp + vmovdqu 0x00($inp),@X[0] + vmovdqu 0x10($inp),@X[1] + vmovdqu 0x20($inp),@X[2] + vmovdqu 0x30($inp),@X[3] + vpshufb $t3,@X[0],@X[0] + lea $TABLE(%rip),$Tbl + vpshufb $t3,@X[1],@X[1] + vpshufb $t3,@X[2],@X[2] + vpaddd 0x00($Tbl),@X[0],$t0 + vpshufb $t3,@X[3],@X[3] + vpaddd 0x20($Tbl),@X[1],$t1 + vpaddd 0x40($Tbl),@X[2],$t2 + vpaddd 0x60($Tbl),@X[3],$t3 + vmovdqa $t0,0x00(%rsp) + mov $A,$a1 + vmovdqa $t1,0x10(%rsp) + mov $B,$a3 + vmovdqa $t2,0x20(%rsp) + xor $C,$a3 # magic + vmovdqa $t3,0x30(%rsp) + mov $E,$a0 + jmp .Lavx_00_47 + +.align 16 +.Lavx_00_47: + sub \$`-16*2*$SZ`,$Tbl # size optimization +___ +sub Xupdate_256_AVX () { + ( + '&vpalignr ($t0,@X[1],@X[0],$SZ)', # X[1..4] + '&vpalignr ($t3,@X[3],@X[2],$SZ)', # X[9..12] + '&vpsrld ($t2,$t0,$sigma0[0]);', + '&vpaddd (@X[0],@X[0],$t3)', # X[0..3] += X[9..12] + '&vpsrld ($t3,$t0,$sigma0[2])', + '&vpslld ($t1,$t0,8*$SZ-$sigma0[1]);', + '&vpxor ($t0,$t3,$t2)', + '&vpshufd ($t3,@X[3],0b11111010)',# X[14..15] + '&vpsrld ($t2,$t2,$sigma0[1]-$sigma0[0]);', + '&vpxor ($t0,$t0,$t1)', + '&vpslld ($t1,$t1,$sigma0[1]-$sigma0[0]);', + '&vpxor ($t0,$t0,$t2)', + '&vpsrld ($t2,$t3,$sigma1[2]);', + '&vpxor ($t0,$t0,$t1)', # sigma0(X[1..4]) + '&vpsrlq ($t3,$t3,$sigma1[0]);', + '&vpaddd (@X[0],@X[0],$t0)', # X[0..3] += sigma0(X[1..4]) + '&vpxor ($t2,$t2,$t3);', + '&vpsrlq ($t3,$t3,$sigma1[1]-$sigma1[0])', + '&vpxor ($t2,$t2,$t3)', + '&vpshufb ($t2,$t2,$t4)', # sigma1(X[14..15]) + '&vpaddd (@X[0],@X[0],$t2)', # X[0..1] += sigma1(X[14..15]) + '&vpshufd ($t3,@X[0],0b01010000)',# X[16..17] + '&vpsrld ($t2,$t3,$sigma1[2])', + '&vpsrlq ($t3,$t3,$sigma1[0])', + '&vpxor ($t2,$t2,$t3);', + '&vpsrlq ($t3,$t3,$sigma1[1]-$sigma1[0])', + '&vpxor ($t2,$t2,$t3)', + '&vpshufb ($t2,$t2,$t5)', + '&vpaddd (@X[0],@X[0],$t2)' # X[2..3] += sigma1(X[16..17]) + ); +} + +sub AVX_256_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body,&$body,&$body); # 104 instructions + + foreach (Xupdate_256_AVX()) { # 29 instructions + eval; + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + } + &vpaddd ($t2,@X[0],16*2*$j."($Tbl)"); + foreach (@insns) { eval; } # remaining instructions + &vmovdqa (16*$j."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<4; $j++) { + &AVX_256_00_47($j,\&body_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &cmpb ($SZ-1+16*2*$SZ."($Tbl)",0); + &jne (".Lavx_00_47"); + + for ($i=0; $i<16; ) { + foreach(body_00_15()) { eval; } + } + + } else { # SHA512 + my @X = map("%xmm$_",(0..7)); + my ($t0,$t1,$t2,$t3) = map("%xmm$_",(8..11)); + +$code.=<<___; + jmp .Lloop_avx +.align 16 +.Lloop_avx: + vmovdqa $TABLE+`$SZ*2*$rounds`(%rip),$t3 + mov $inp,$_inp # offload $inp + vmovdqu 0x00($inp),@X[0] + vmovdqu 0x10($inp),@X[1] + vmovdqu 0x20($inp),@X[2] + vpshufb $t3,@X[0],@X[0] + vmovdqu 0x30($inp),@X[3] + vpshufb $t3,@X[1],@X[1] + vmovdqu 0x40($inp),@X[4] + vpshufb $t3,@X[2],@X[2] + vmovdqu 0x50($inp),@X[5] + vpshufb $t3,@X[3],@X[3] + vmovdqu 0x60($inp),@X[6] + vpshufb $t3,@X[4],@X[4] + vmovdqu 0x70($inp),@X[7] + lea $TABLE+0x80(%rip),$Tbl # size optimization + vpshufb $t3,@X[5],@X[5] + vpaddq -0x80($Tbl),@X[0],$t0 + vpshufb $t3,@X[6],@X[6] + vpaddq -0x60($Tbl),@X[1],$t1 + vpshufb $t3,@X[7],@X[7] + vpaddq -0x40($Tbl),@X[2],$t2 + vpaddq -0x20($Tbl),@X[3],$t3 + vmovdqa $t0,0x00(%rsp) + vpaddq 0x00($Tbl),@X[4],$t0 + vmovdqa $t1,0x10(%rsp) + vpaddq 0x20($Tbl),@X[5],$t1 + vmovdqa $t2,0x20(%rsp) + vpaddq 0x40($Tbl),@X[6],$t2 + vmovdqa $t3,0x30(%rsp) + vpaddq 0x60($Tbl),@X[7],$t3 + vmovdqa $t0,0x40(%rsp) + mov $A,$a1 + vmovdqa $t1,0x50(%rsp) + mov $B,$a3 + vmovdqa $t2,0x60(%rsp) + xor $C,$a3 # magic + vmovdqa $t3,0x70(%rsp) + mov $E,$a0 + jmp .Lavx_00_47 + +.align 16 +.Lavx_00_47: + add \$`16*2*$SZ`,$Tbl +___ +sub Xupdate_512_AVX () { + ( + '&vpalignr ($t0,@X[1],@X[0],$SZ)', # X[1..2] + '&vpalignr ($t3,@X[5],@X[4],$SZ)', # X[9..10] + '&vpsrlq ($t2,$t0,$sigma0[0])', + '&vpaddq (@X[0],@X[0],$t3);', # X[0..1] += X[9..10] + '&vpsrlq ($t3,$t0,$sigma0[2])', + '&vpsllq ($t1,$t0,8*$SZ-$sigma0[1]);', + '&vpxor ($t0,$t3,$t2)', + '&vpsrlq ($t2,$t2,$sigma0[1]-$sigma0[0]);', + '&vpxor ($t0,$t0,$t1)', + '&vpsllq ($t1,$t1,$sigma0[1]-$sigma0[0]);', + '&vpxor ($t0,$t0,$t2)', + '&vpsrlq ($t3,@X[7],$sigma1[2]);', + '&vpxor ($t0,$t0,$t1)', # sigma0(X[1..2]) + '&vpsllq ($t2,@X[7],8*$SZ-$sigma1[1]);', + '&vpaddq (@X[0],@X[0],$t0)', # X[0..1] += sigma0(X[1..2]) + '&vpsrlq ($t1,@X[7],$sigma1[0]);', + '&vpxor ($t3,$t3,$t2)', + '&vpsllq ($t2,$t2,$sigma1[1]-$sigma1[0]);', + '&vpxor ($t3,$t3,$t1)', + '&vpsrlq ($t1,$t1,$sigma1[1]-$sigma1[0]);', + '&vpxor ($t3,$t3,$t2)', + '&vpxor ($t3,$t3,$t1)', # sigma1(X[14..15]) + '&vpaddq (@X[0],@X[0],$t3)', # X[0..1] += sigma1(X[14..15]) + ); +} + +sub AVX_512_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body); # 52 instructions + + foreach (Xupdate_512_AVX()) { # 23 instructions + eval; + eval(shift(@insns)); + eval(shift(@insns)); + } + &vpaddq ($t2,@X[0],16*2*$j-0x80."($Tbl)"); + foreach (@insns) { eval; } # remaining instructions + &vmovdqa (16*$j."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<8; $j++) { + &AVX_512_00_47($j,\&body_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &cmpb ($SZ-1+16*2*$SZ-0x80."($Tbl)",0); + &jne (".Lavx_00_47"); + + for ($i=0; $i<16; ) { + foreach(body_00_15()) { eval; } + } +} +$code.=<<___; + mov $_ctx,$ctx + mov $a1,$A + mov $_inp,$inp + + add $SZ*0($ctx),$A + add $SZ*1($ctx),$B + add $SZ*2($ctx),$C + add $SZ*3($ctx),$D + add $SZ*4($ctx),$E + add $SZ*5($ctx),$F + add $SZ*6($ctx),$G + add $SZ*7($ctx),$H + + lea 16*$SZ($inp),$inp + cmp $_end,$inp + + mov $A,$SZ*0($ctx) + mov $B,$SZ*1($ctx) + mov $C,$SZ*2($ctx) + mov $D,$SZ*3($ctx) + mov $E,$SZ*4($ctx) + mov $F,$SZ*5($ctx) + mov $G,$SZ*6($ctx) + mov $H,$SZ*7($ctx) + jb .Lloop_avx + + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0xa0(%rbp),%xmm6 + movaps -0x90(%rbp),%xmm7 + movaps -0x80(%rbp),%xmm8 + movaps -0x70(%rbp),%xmm9 +___ +$code.=<<___ if ($win64 && $SZ>4); + movaps -0x60(%rbp),%xmm10 + movaps -0x50(%rbp),%xmm11 +___ +$code.=<<___; + mov -40(%rbp),%r15 + mov -32(%rbp),%r14 + mov -24(%rbp),%r13 + mov -16(%rbp),%r12 + mov -8(%rbp),%rbx + mov %rbp,%rsp +.cfi_def_cfa_register %rsp + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size ${func}_avx,.-${func}_avx +___ + +if ($avx>1) {{ +###################################################################### +# AVX2+BMI code path +# +my $Tbl=$inp; +my $_ctx="-64(%rbp)"; +my $_inp="-56(%rbp)"; +my $_end="-48(%rbp)"; +my $framesz=3*8+$win64*16*6; +my $PUSH8=8*2*$SZ; +use integer; + +sub bodyx_00_15 () { + # at start $a1 should be zero, $a3 - $b^$c and $a4 copy of $f + ( + '($a,$b,$c,$d,$e,$f,$g,$h)=@ROT;'. + + '&add ($h,(32*($i/(16/$SZ))+$SZ*($i%(16/$SZ)))%$PUSH8.$base)', # h+=X[i]+K[i] + '&and ($a4,$e)', # f&e + '&rorx ($a0,$e,$Sigma1[2])', + '&rorx ($a2,$e,$Sigma1[1])', + + '&lea ($a,"($a,$a1)")', # h+=Sigma0(a) from the past + '&lea ($h,"($h,$a4)")', + '&andn ($a4,$e,$g)', # ~e&g + '&xor ($a0,$a2)', + + '&rorx ($a1,$e,$Sigma1[0])', + '&lea ($h,"($h,$a4)")', # h+=Ch(e,f,g)=(e&f)+(~e&g) + '&xor ($a0,$a1)', # Sigma1(e) + '&mov ($a2,$a)', + + '&rorx ($a4,$a,$Sigma0[2])', + '&lea ($h,"($h,$a0)")', # h+=Sigma1(e) + '&xor ($a2,$b)', # a^b, b^c in next round + '&rorx ($a1,$a,$Sigma0[1])', + + '&rorx ($a0,$a,$Sigma0[0])', + '&lea ($d,"($d,$h)")', # d+=h + '&and ($a3,$a2)', # (b^c)&(a^b) + '&xor ($a1,$a4)', + + '&xor ($a3,$b)', # Maj(a,b,c)=Ch(a^b,c,b) + '&xor ($a1,$a0)', # Sigma0(a) + '&lea ($h,"($h,$a3)");'. # h+=Maj(a,b,c) + '&mov ($a4,$e)', # copy of f in future + + '($a2,$a3) = ($a3,$a2); unshift(@ROT,pop(@ROT)); $i++;' + ); + # and at the finish one has to $a+=$a1 +} + +$code.=<<___; +.type ${func}_avx2,\@function,3,"unwind" +.align 64 +${func}_avx2: +.cfi_startproc + push %rbp +.cfi_push %rbp + mov %rsp,%rbp +.cfi_def_cfa_register %rbp +.Lavx2_shortcut: + push %rbx +.cfi_push %rbx + push %r12 +.cfi_push %r12 + push %r13 +.cfi_push %r13 + push %r14 +.cfi_push %r14 + push %r15 +.cfi_push %r15 + shl \$4,%rdx # num*16 + sub \$$framesz,%rsp +.cfi_alloca $framesz + lea ($inp,%rdx,$SZ),%rdx # inp+num*16*$SZ + mov $ctx,$_ctx # save ctx, 1st arg + #mov $inp,$_inp # save inp, 2nd arg + mov %rdx,$_end # save end pointer, "3rd" arg +___ +$code.=<<___ if ($win64); + movaps %xmm6,-0xa0(%rbp) + movaps %xmm7,-0x90(%rbp) + movaps %xmm8,-0x80(%rbp) + movaps %xmm9,-0x70(%rbp) +.cfi_offset %xmm6-%xmm9,-0xb0 +___ +$code.=<<___ if ($win64 && $SZ>4); + movaps %xmm10,-0x60(%rbp) + movaps %xmm11,-0x50(%rbp) +.cfi_offset %xmm10-%xmm11,-0x70 +___ +$code.=<<___; +.cfi_end_prologue + + lea -$PUSH8(%rsp),%rsp + vzeroupper + and \$-$PUSH8,%rsp # align stack + sub \$-16*$SZ,$inp # inp++, size optimization + mov $SZ*0($ctx),$A + mov $inp,%r12 # borrow $T1 + mov $SZ*1($ctx),$B + cmp %rdx,$inp # $_end + mov $SZ*2($ctx),$C + cmove %rsp,%r12 # next block or random data + mov $SZ*3($ctx),$D + mov $SZ*4($ctx),$E + mov $SZ*5($ctx),$F + mov $SZ*6($ctx),$G + mov $SZ*7($ctx),$H +___ + if ($SZ==4) { # SHA256 + my @X = map("%ymm$_",(0..3)); + my ($t0,$t1,$t2,$t3, $t4,$t5) = map("%ymm$_",(4..9)); + +$code.=<<___; + vmovdqa $TABLE+`$SZ*2*$rounds`+32(%rip),$t4 + vmovdqa $TABLE+`$SZ*2*$rounds`+64(%rip),$t5 + jmp .Loop_avx2 +.align 16 +.Loop_avx2: + vmovdqa $TABLE+`$SZ*2*$rounds`(%rip),$t3 + mov $inp,$_inp # offload $inp + vmovdqu -16*$SZ+0($inp),%xmm0 + vmovdqu -16*$SZ+16($inp),%xmm1 + vmovdqu -16*$SZ+32($inp),%xmm2 + vmovdqu -16*$SZ+48($inp),%xmm3 + lea $TABLE(%rip),$Tbl + vinserti128 \$1,(%r12),@X[0],@X[0] + vinserti128 \$1,16(%r12),@X[1],@X[1] + vpshufb $t3,@X[0],@X[0] + vinserti128 \$1,32(%r12),@X[2],@X[2] + vpshufb $t3,@X[1],@X[1] + vinserti128 \$1,48(%r12),@X[3],@X[3] + + vpshufb $t3,@X[2],@X[2] + vpaddd 0x00($Tbl),@X[0],$t0 + vpshufb $t3,@X[3],@X[3] + vpaddd 0x20($Tbl),@X[1],$t1 + vpaddd 0x40($Tbl),@X[2],$t2 + vpaddd 0x60($Tbl),@X[3],$t3 + vmovdqa $t0,0x00(%rsp) + xor $a1,$a1 + vmovdqa $t1,0x20(%rsp) + lea -$PUSH8(%rsp),%rsp + mov $B,$a3 + vmovdqa $t2,0x00(%rsp) + xor $C,$a3 # magic + vmovdqa $t3,0x20(%rsp) + mov $F,$a4 + sub \$-16*2*$SZ,$Tbl # size optimization + jmp .Lavx2_00_47 + +.align 16 +.Lavx2_00_47: +___ + +sub AVX2_256_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body,&$body,&$body); # 96 instructions +my $base = "+2*$PUSH8(%rsp)"; + + &lea ("%rsp","-$PUSH8(%rsp)") if (($j%2)==0); + foreach (Xupdate_256_AVX()) { # 29 instructions + eval; + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + } + &vpaddd ($t2,@X[0],16*2*$j."($Tbl)"); + foreach (@insns) { eval; } # remaining instructions + &vmovdqa ((32*$j)%$PUSH8."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<4; $j++) { + &AVX2_256_00_47($j,\&bodyx_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &lea ($Tbl,16*2*$SZ."($Tbl)"); + &cmpb (($SZ-1)."($Tbl)",0); + &jne (".Lavx2_00_47"); + + for ($i=0; $i<16; ) { + my $base=$i<8?"+$PUSH8(%rsp)":"(%rsp)"; + foreach(bodyx_00_15()) { eval; } + } + } else { # SHA512 + my @X = map("%ymm$_",(0..7)); + my ($t0,$t1,$t2,$t3) = map("%ymm$_",(8..11)); + +$code.=<<___; + jmp .Loop_avx2 +.align 16 +.Loop_avx2: + vmovdqa $TABLE+`$SZ*2*$rounds`(%rip),$t2 + mov $inp,$_inp # offload $inp + vmovdqu -16*$SZ($inp),%xmm0 + vmovdqu -16*$SZ+16($inp),%xmm1 + vmovdqu -16*$SZ+32($inp),%xmm2 + vmovdqu -16*$SZ+48($inp),%xmm3 + vmovdqu -16*$SZ+64($inp),%xmm4 + vmovdqu -16*$SZ+80($inp),%xmm5 + vmovdqu -16*$SZ+96($inp),%xmm6 + vmovdqu -16*$SZ+112($inp),%xmm7 + lea $TABLE+0x80(%rip),$Tbl # size optimization + vinserti128 \$1,(%r12),@X[0],@X[0] + vinserti128 \$1,16(%r12),@X[1],@X[1] + vpshufb $t2,@X[0],@X[0] + vinserti128 \$1,32(%r12),@X[2],@X[2] + vpshufb $t2,@X[1],@X[1] + vinserti128 \$1,48(%r12),@X[3],@X[3] + vpshufb $t2,@X[2],@X[2] + vinserti128 \$1,64(%r12),@X[4],@X[4] + vpshufb $t2,@X[3],@X[3] + vinserti128 \$1,80(%r12),@X[5],@X[5] + vpshufb $t2,@X[4],@X[4] + vinserti128 \$1,96(%r12),@X[6],@X[6] + vpshufb $t2,@X[5],@X[5] + vinserti128 \$1,112(%r12),@X[7],@X[7] + + vpaddq -0x80($Tbl),@X[0],$t0 + vpshufb $t2,@X[6],@X[6] + vpaddq -0x60($Tbl),@X[1],$t1 + vpshufb $t2,@X[7],@X[7] + vpaddq -0x40($Tbl),@X[2],$t2 + vpaddq -0x20($Tbl),@X[3],$t3 + vmovdqa $t0,0x00(%rsp) + vpaddq 0x00($Tbl),@X[4],$t0 + vmovdqa $t1,0x20(%rsp) + vpaddq 0x20($Tbl),@X[5],$t1 + vmovdqa $t2,0x40(%rsp) + vpaddq 0x40($Tbl),@X[6],$t2 + vmovdqa $t3,0x60(%rsp) + lea -$PUSH8(%rsp),%rsp + vpaddq 0x60($Tbl),@X[7],$t3 + vmovdqa $t0,0x00(%rsp) + xor $a1,$a1 + vmovdqa $t1,0x20(%rsp) + mov $B,$a3 + vmovdqa $t2,0x40(%rsp) + xor $C,$a3 # magic + vmovdqa $t3,0x60(%rsp) + mov $F,$a4 + add \$16*2*$SZ,$Tbl + jmp .Lavx2_00_47 + +.align 16 +.Lavx2_00_47: +___ + +sub AVX2_512_00_47 () { +my $j = shift; +my $body = shift; +my @X = @_; +my @insns = (&$body,&$body); # 48 instructions +my $base = "+2*$PUSH8(%rsp)"; + + &lea ("%rsp","-$PUSH8(%rsp)") if (($j%4)==0); + foreach (Xupdate_512_AVX()) { # 23 instructions + eval; + if ($_ !~ /\;$/) { + eval(shift(@insns)); + eval(shift(@insns)); + eval(shift(@insns)); + } + } + &vpaddq ($t2,@X[0],16*2*$j-0x80."($Tbl)"); + foreach (@insns) { eval; } # remaining instructions + &vmovdqa ((32*$j)%$PUSH8."(%rsp)",$t2); +} + + for ($i=0,$j=0; $j<8; $j++) { + &AVX2_512_00_47($j,\&bodyx_00_15,@X); + push(@X,shift(@X)); # rotate(@X) + } + &lea ($Tbl,16*2*$SZ."($Tbl)"); + &cmpb (($SZ-1-0x80)."($Tbl)",0); + &jne (".Lavx2_00_47"); + + for ($i=0; $i<16; ) { + my $base=$i<8?"+$PUSH8(%rsp)":"(%rsp)"; + foreach(bodyx_00_15()) { eval; } + } +} +$code.=<<___; + mov $_ctx,$ctx + add $a1,$A + mov $_inp,$a4 + + add $SZ*0($ctx),$A + add $SZ*1($ctx),$B + add $SZ*2($ctx),$C + add $SZ*3($ctx),$D + add $SZ*4($ctx),$E + add $SZ*5($ctx),$F + add $SZ*6($ctx),$G + add $SZ*7($ctx),$H + + mov $A,$SZ*0($ctx) + mov $B,$SZ*1($ctx) + mov $C,$SZ*2($ctx) + mov $D,$SZ*3($ctx) + mov $E,$SZ*4($ctx) + mov $F,$SZ*5($ctx) + mov $G,$SZ*6($ctx) + mov $H,$SZ*7($ctx) + + cmp $_end,$a4 + je .Ldone_avx2 + + lea `2*$SZ*($rounds-8)`(%rsp),$Tbl + xor $a1,$a1 + mov $B,$a3 + xor $C,$a3 # magic + mov $F,$a4 + jmp .Lower_avx2 +.align 16 +.Lower_avx2: +___ + for ($i=0; $i<8; ) { + my $base="+16($Tbl)"; + foreach(bodyx_00_15()) { eval; } + } +$code.=<<___; + lea -$PUSH8($Tbl),$Tbl + cmp %rsp,$Tbl + jae .Lower_avx2 + + mov $_ctx,$ctx + add $a1,$A + mov $_inp,$inp + lea `2*$SZ*($rounds-8)`(%rsp),%rsp + + add $SZ*0($ctx),$A + add $SZ*1($ctx),$B + add $SZ*2($ctx),$C + add $SZ*3($ctx),$D + add $SZ*4($ctx),$E + add $SZ*5($ctx),$F + lea `2*16*$SZ`($inp),$inp # inp+=2 + add $SZ*6($ctx),$G + mov $inp,%r12 + add $SZ*7($ctx),$H + cmp $_end,$inp + + mov $A,$SZ*0($ctx) + cmove %rsp,%r12 # next block or stale data + mov $B,$SZ*1($ctx) + mov $C,$SZ*2($ctx) + mov $D,$SZ*3($ctx) + mov $E,$SZ*4($ctx) + mov $F,$SZ*5($ctx) + mov $G,$SZ*6($ctx) + mov $H,$SZ*7($ctx) + + jbe .Loop_avx2 + +.Ldone_avx2: + vzeroupper +___ +$code.=<<___ if ($win64); + movaps -0xa0(%rbp),%xmm6 + movaps -0x90(%rbp),%xmm7 + movaps -0x80(%rbp),%xmm8 + movaps -0x70(%rbp),%xmm9 +___ +$code.=<<___ if ($win64 && $SZ>4); + movaps -0x60(%rbp),%xmm10 + movaps -0x50(%rbp),%xmm11 +___ +$code.=<<___; + mov -40(%rbp),%r15 + mov -32(%rbp),%r14 + mov -24(%rbp),%r13 + mov -16(%rbp),%r12 + mov -8(%rbp),%rbx + mov %rbp,%rsp +.cfi_def_cfa_register %rsp + pop %rbp +.cfi_pop %rbp +.cfi_epilogue + ret +.cfi_endproc +.size ${func}_avx2,.-${func}_avx2 +___ +}} +}}}}} + +sub sha256op38 { + my $instr = shift; + my %opcodelet = ( + "sha256rnds2" => 0xcb, + "sha256msg1" => 0xcc, + "sha256msg2" => 0xcd ); + + if (defined($opcodelet{$instr}) && @_[0] =~ /%xmm([0-7]),\s*%xmm([0-7])/) { + my @opcode=(0x0f,0x38); + push @opcode,$opcodelet{$instr}; + push @opcode,0xc0|($1&7)|(($2&7)<<3); # ModR/M + return ".byte\t".join(',',@opcode); + } else { + return $instr."\t".@_[0]; + } +} + +sub vsha512rnds2 { + my $instr = shift; + + if (@_[0] =~ /%xmm([0-9]+),\s*%ymm([0-9]+),\s*%ymm([0-9]+)/) { + my @opcode=(0xc4,0xe2,0x7f,0xcb); + @opcode[1] ^= (($1>>3)<<5)|(($3>>3)<<7); + @opcode[2] ^= $2<<3; + push @opcode,0xc0|($1&7)|(($3&7)<<3); # ModR/M + return ".byte\t".join(',',@opcode); + } else { + return $instr."\t".@_[0]; + } +} + +sub vsha512msg { + my $instr = shift; + my $op = shift; + + if (@_[0] =~ /%[xy]mm([0-9]+),\s*%ymm([0-9]+)/) { + my @opcode=(0xc4,0xe2,0x7f,0xcb+$op); + @opcode[1] ^= (($1>>3)<<5)|(($2>>3)<<7); + push @opcode,0xc0|($1&7)|(($2&7)<<3); # ModR/M + return ".byte\t".join(',',@opcode); + } else { + return $instr.$op."\t".@_[0]; + } +} + +foreach (split("\n",$code)) { + s/\`([^\`]*)\`/eval $1/geo; + s/%x#%[yz]/%x/go; + + s/\b(sha256[^\s]*)\s+(.*)/sha256op38($1,$2)/eo or + s/\b(vsha512msg)([12])\s+(.*)/vsha512msg($1,$2,$3)/eo or + s/\b(vsha512rnds2)\s+(.*)/vsha512rnds2($1,$2)/eo; + + print $_,"\n"; +} +close STDOUT; diff --git a/cbits/asm/x86_64-xlate.pl b/cbits/asm/x86_64-xlate.pl new file mode 100644 --- /dev/null +++ b/cbits/asm/x86_64-xlate.pl @@ -0,0 +1,1943 @@ +#!/usr/bin/env perl + +# Ascetic x86_64 AT&T to MASM/NASM assembler translator by @dot-asm. +# +# Why AT&T to MASM and not vice versa? Several reasons. Because AT&T +# format is way easier to parse. Because it's simpler to "gear" from +# Unix ABI to Windows one [see cross-reference "card" at the end of +# file]. Because Linux targets were available first... +# +# In addition the script also "distills" code suitable for GNU +# assembler, so that it can be compiled with more rigid assemblers, +# such as Solaris /usr/ccs/bin/as. +# +# This translator is not designed to convert *arbitrary* assembler +# code from AT&T format to MASM one. It's designed to convert just +# enough to provide for dual-ABI OpenSSL modules development... +# There *are* limitations and you might have to modify your assembler +# code or this script to achieve the desired result... +# +# Currently recognized limitations: +# +# - can't use multiple ops per line; +# +# Dual-ABI styling rules. +# +# 1. Adhere to Unix register and stack layout [see cross-reference +# ABI "card" at the end for explanation]. +# 2. Forget about "red zone," stick to more traditional blended +# stack frame allocation. If volatile storage is actually required +# that is. If not, just leave the stack as is. +# 3. Functions tagged with ".type name,@function" get crafted with +# unified Win64 prologue and epilogue automatically. If you want +# to take care of ABI differences yourself, tag functions as +# ".type name,@abi-omnipotent" instead. +# 4. To optimize the Win64 prologue you can specify number of input +# arguments as ".type name,@function,N." Keep in mind that if N is +# larger than 6, then you *have to* write "abi-omnipotent" code, +# because >6 cases can't be addressed with unified prologue. +# 5. Name local labels as .L*, do *not* use dynamic labels such as 1: +# (sorry about latter). +# 6. Don't use [or hand-code with .byte] "rep ret." "ret" mnemonic is +# required to identify the spots, where to inject Win64 epilogue! +# But on the pros, it's then prefixed with rep automatically:-) +# 7. Stick to explicit ip-relative addressing. If you have to use +# GOTPCREL addressing, stick to mov symbol@GOTPCREL(%rip),%r??. +# Both are recognized and translated to proper Win64 addressing +# modes. +# +# 8. In order to provide for structured exception handling unified +# Win64 prologue copies %rsp value to %rax. [Unless function is +# tagged with additional .type tag.] For further details see SEH +# paragraph at the end. +# 9. .init segment is allowed to contain calls to functions only. +# a. If function accepts more than 4 arguments *and* >4th argument +# is declared as non 64-bit value, do clear its upper part. + + +use strict; + +my $flavour = shift; +my $output = shift; +if ($flavour =~ /\./) { $output = $flavour; undef $flavour; } + +open STDOUT,">$output" || die "can't open $output: $!" + if (defined($output)); + +my $gas=1; $gas=0 if ($output =~ /\.asm$/); +my $elf=1; $elf=0 if (!$gas); +my $dwarf=$elf; +my $win64=0; +my $prefix=""; +my $decor=".L"; + +my $masmref=8 + 50727*2**-32; # 8.00.50727 shipped with VS2005 +my $masm=0; +my $PTR=" PTR"; + +my $nasmref=2.03; +my $nasm=0; + +if ($flavour eq "mingw64") { $gas=1; $elf=0; $win64=1; + $prefix=`echo __USER_LABEL_PREFIX__ | \${CC:-false} -E -P -`; + $prefix =~ s|\R$||; # Better chomp + } +elsif ($flavour eq "macosx") { $gas=1; $elf=0; $prefix="_"; $decor="L\$"; } +elsif ($flavour eq "masm") { $gas=0; $elf=0; $masm=$masmref; $win64=1; $decor="\$L\$"; } +elsif ($flavour eq "nasm") { $gas=0; $elf=0; $nasm=$nasmref; $win64=1; $decor="\$L\$"; $PTR=""; } +elsif (!$gas) +{ if ($ENV{ASM} =~ m/nasm/ && `nasm -v` =~ m/version ([0-9]+)\.([0-9]+)/i) + { $nasm = $1 + $2*0.01; $PTR=""; } + elsif (`ml64 2>&1` =~ m/Version ([0-9]+)\.([0-9]+)(\.([0-9]+))?/) + { $masm = $1 + $2*2**-16 + $4*2**-32; } + die "no assembler found on %PATH%" if (!($nasm || $masm)); + $win64=1; + $elf=0; + $decor="\$L\$"; +} +my $colon= $masm ? "::" : ":"; + +$dwarf=0 if($win64); + +my $current_segment; +my $current_function; +my %globals; + +{ package opcode; # pick up opcodes + sub re { + my ($class, $line) = @_; + my $self = {}; + my $ret; + + if ($$line =~ /^([a-z][a-z0-9]*)/i) { + bless $self,$class; + $self->{op} = $1; + $ret = $self; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + + undef $self->{sz}; + if ($self->{op} =~ /^(movz)x?([bw]).*/) { # movz is pain... + $self->{op} = $1; + $self->{sz} = $2; + } elsif ($self->{op} =~ /cmov[n]?[lb]$/) { + # pass through + } elsif ($self->{op} =~ /call|jmp/) { + $self->{sz} = ""; + } elsif ($self->{op} =~ /^p/ && $' !~ /^(ush|op|insrw)/) { # SSEn + $self->{sz} = ""; + } elsif ($self->{op} =~ /^[vk]/) { # VEX or k* such as kmov + $self->{sz} = ""; + } elsif ($self->{op} =~ /mov[dq]/ && $$line =~ /%xmm/) { + $self->{sz} = ""; + } elsif ($self->{op} =~ /([a-z]{3,})([qlwb])$/) { + $self->{op} = $1; + $self->{sz} = $2; + } + } + $ret; + } + sub size { + my ($self, $sz) = @_; + $self->{sz} = $sz if (defined($sz) && !defined($self->{sz})); + $self->{sz}; + } + sub out { + my $self = shift; + if ($gas) { + if ($self->{op} eq "movz") { # movz is pain... + sprintf "%s%s%s",$self->{op},$self->{sz},shift; + } elsif ($self->{op} =~ /^set/) { + "$self->{op}"; + } elsif ($self->{op} eq "ret") { + my $epilogue = ""; + if ($win64 && $current_function->{abi} eq "svr4" + && !$current_function->{unwind}) { + $epilogue = "movq 8(%rsp),%rdi\n\t" . + "movq 16(%rsp),%rsi\n\t"; + } + $epilogue . ".byte 0xf3,0xc3"; + } elsif ($self->{op} eq "call" && !$elf && $current_segment eq ".init") { + ".p2align\t3\n\t.quad"; + } else { + "$self->{op}$self->{sz}"; + } + } else { + $self->{op} =~ s/^movz/movzx/; + if ($self->{op} eq "ret") { + $self->{op} = ""; + if ($win64 && $current_function->{abi} eq "svr4" + && !$current_function->{unwind}) { + $self->{op} = "mov rdi,QWORD$PTR\[8+rsp\]\t;WIN64 epilogue\n\t". + "mov rsi,QWORD$PTR\[16+rsp\]\n\t"; + } + $self->{op} .= "DB\t0F3h,0C3h\t\t;repret"; + } elsif ($self->{op} =~ /^(pop|push)f/) { + $self->{op} .= $self->{sz}; + } elsif ($self->{op} eq "call" && $current_segment eq ".CRT\$XCU") { + $self->{op} = "\tDQ"; + } + $self->{op}; + } + } + sub mnemonic { + my ($self, $op) = @_; + $self->{op}=$op if (defined($op)); + $self->{op}; + } +} +{ package const; # pick up constants, which start with $ + sub re { + my ($class, $line) = @_; + my $self = {}; + my $ret; + + if ($$line =~ /^\$([^,]+)/) { + bless $self, $class; + $self->{value} = $1; + $ret = $self; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + } + $ret; + } + sub out { + my $self = shift; + + $self->{value} =~ s/\b(0b[0-1]+)/oct($1)/eig; + if ($gas) { + # Solaris /usr/ccs/bin/as can't handle multiplications + # in $self->{value} + my $value = $self->{value}; + no warnings; # oct might complain about overflow, ignore here... + $value =~ s/(?{value} = $value; + } + sprintf "\$%s",$self->{value}; + } else { + my $value = $self->{value}; + $value =~ s/0x([0-9a-f]+)/0$1h/ig if ($masm); + sprintf "%s",$value; + } + } +} +{ package ea; # pick up effective addresses: expr(%reg,%reg,scale) + + my %szmap = ( b=>"BYTE$PTR", w=>"WORD$PTR", + l=>"DWORD$PTR", d=>"DWORD$PTR", + q=>"QWORD$PTR", o=>"OWORD$PTR", + x=>"XMMWORD$PTR", y=>"YMMWORD$PTR", + z=>"ZMMWORD$PTR" ) if (!$gas); + + my %sifmap = ( ss=>"d", sd=>"q", # broadcast only + i32x2=>"q", f32x2=>"q", + i32x4=>"x", i64x2=>"x", i128=>"x", + f32x4=>"x", f64x2=>"x", f128=>"x", + i32x8=>"y", i64x4=>"y", + f32x8=>"y", f64x4=>"y" ) if (!$gas); + + sub re { + my ($class, $line, $opcode) = @_; + my $self = {}; + my $ret; + + # optional * ----vvv--- appears in indirect jmp/call + if ($$line =~ /^(\*?)([^\(,]*)\(([%\w,\s]+)\)((?:{[^}]+})*)/) { + bless $self, $class; + $self->{asterisk} = $1; + $self->{label} = $2; + ($self->{base},$self->{index},$self->{scale})=split(/(?:,\s*)/,$3); + $self->{scale} = 1 if (!defined($self->{scale})); + $self->{opmask} = $4; + $ret = $self; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + + if ($win64 && $self->{label} =~ s/\@GOTPCREL//) { + die if ($opcode->mnemonic() ne "mov"); + $opcode->mnemonic("lea"); + } + $self->{base} =~ s/^%//; + $self->{index} =~ s/^%// if (defined($self->{index})); + $self->{opcode} = $opcode; + } + $ret; + } + sub size {} + sub out { + my ($self, $sz) = @_; + + $self->{label} =~ s/([_a-z][_a-z0-9\$]*)/$globals{$1} or $1/gei; + $self->{label} =~ s/\.L/$decor/g; + + # Silently convert all EAs to 64-bit. This is required for + # elder GNU assembler and results in more compact code, + # *but* most importantly AES module depends on this feature! + $self->{index} =~ s/^[er](.?[0-9xpi])[d]?$/r\1/; + $self->{base} =~ s/^[er](.?[0-9xpi])[d]?$/r\1/; + + # Solaris /usr/ccs/bin/as can't handle multiplications + # in $self->{label}... + use integer; + $self->{label} =~ s/(?{label} =~ s/\b([0-9]+\s*[\*\/\%]\s*[0-9]+)\b/eval($1)/eg; + + # Some assemblers insist on signed presentation of 32-bit + # offsets, but sign extension is a tricky business in perl... + $self->{label} =~ s/\b([0-9]+)\b/unpack("l",pack("L",$1))/eg; + + # if base register is %rbp or %r13, see if it's possible to + # flip base and index registers [for better performance] + if (!$self->{label} && $self->{index} && $self->{scale}==1 && + $self->{base} =~ /(rbp|r13)/) { + $self->{base} = $self->{index}; $self->{index} = $1; + } + + if ($gas) { + $self->{label} =~ s/^___imp_/__imp__/ if ($flavour eq "mingw64"); + + if (defined($self->{index})) { + sprintf "%s%s(%s,%%%s,%d)%s", + $self->{asterisk},$self->{label}, + $self->{base}?"%$self->{base}":"", + $self->{index},$self->{scale}, + $self->{opmask}; + } else { + sprintf "%s%s(%%%s)%s", $self->{asterisk},$self->{label}, + $self->{base},$self->{opmask}; + } + } else { + $self->{label} =~ s/\./\$/g; + $self->{label} =~ s/(?{label} = "($self->{label})" if ($self->{label} =~ /[\*\+\-\/]/); + + my $mnemonic = $self->{opcode}->mnemonic(); + ($self->{asterisk}) && ($sz="q") || + ($mnemonic =~ /^v?mov([qd])$/) && ($sz=$1) || + ($mnemonic =~ /^v?pinsr([qdwb])$/) && ($sz=$1) || + ($mnemonic =~ /^vpbroadcast([qdwb])$/) && ($sz=$1) || + ($mnemonic =~ /^v(?:broadcast|extract|insert)([sif]\w+)$/) + && ($sz=$sifmap{$1}); + + $self->{opmask} =~ s/%(k[0-7])/$1/; + + if (defined($self->{index})) { + sprintf "%s[%s%s*%d%s]%s",$szmap{$sz}, + $self->{label}?"$self->{label}+":"", + $self->{index},$self->{scale}, + $self->{base}?"+$self->{base}":"", + $self->{opmask}; + } elsif ($self->{base} eq "rip") { + sprintf "%s[%s]",$szmap{$sz},$self->{label}; + } else { + sprintf "%s[%s%s]%s", $szmap{$sz}, + $self->{label}?"$self->{label}+":"", + $self->{base},$self->{opmask}; + } + } + } +} +{ package register; # pick up registers, which start with %. + sub re { + my ($class, $line, $opcode) = @_; + my $self = {}; + my $ret; + + # optional * ----vvv--- appears in indirect jmp/call + if ($$line =~ /^(\*?)%(\w+)((?:{[^}]+})*)/) { + bless $self,$class; + $self->{asterisk} = $1; + $self->{value} = $2; + $self->{opmask} = $3; + $opcode->size($self->size()); + $ret = $self; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + } + $ret; + } + sub size { + my $self = shift; + my $ret; + + if ($self->{value} =~ /^r[\d]+b$/i) { $ret="b"; } + elsif ($self->{value} =~ /^r[\d]+w$/i) { $ret="w"; } + elsif ($self->{value} =~ /^r[\d]+d$/i) { $ret="l"; } + elsif ($self->{value} =~ /^r[\w]+$/i) { $ret="q"; } + elsif ($self->{value} =~ /^[a-d][hl]$/i){ $ret="b"; } + elsif ($self->{value} =~ /^[\w]{2}l$/i) { $ret="b"; } + elsif ($self->{value} =~ /^[\w]{2}$/i) { $ret="w"; } + elsif ($self->{value} =~ /^e[a-z]{2}$/i){ $ret="l"; } + + $ret; + } + sub out { + my $self = shift; + if ($gas) { sprintf "%s%%%s%s", $self->{asterisk}, + $self->{value}, + $self->{opmask}; } + else { $self->{opmask} =~ s/%(k[0-7])/$1/; + $self->{value}.$self->{opmask}; } + } +} +{ package label; # pick up labels, which end with : + sub re { + my ($class, $line) = @_; + my $self = {}; + my $ret; + + if ($$line =~ /(^[\.\w\$]+)\:/) { + bless $self,$class; + $self->{value} = $1; + $ret = $self; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + + $self->{value} =~ s/^\.L/$decor/; + } + $ret; + } + sub win64_args { + my $narg = $current_function->{narg} // 6; + return undef if ($narg < 0); + my $arg5 = 4*8 - cfi_directive::cfa_rsp(); + my $arg6 = $arg5 + 8; + my $args; + if ($gas) { + $args .= " movq %rcx,%rdi\n" if ($narg>0); + $args .= " movq %rdx,%rsi\n" if ($narg>1); + $args .= " movq %r8,%rdx\n" if ($narg>2); + $args .= " movq %r9,%rcx\n" if ($narg>3); + $args .= " movq $arg5(%rsp),%r8\n" if ($narg>4); + $args .= " movq $arg6(%rsp),%r9\n" if ($narg>5); + } else { + $args .= " mov rdi,rcx\n" if ($narg>0); + $args .= " mov rsi,rdx\n" if ($narg>1); + $args .= " mov rdx,r8\n" if ($narg>2); + $args .= " mov rcx,r9\n" if ($narg>3); + $args .= " mov r8,QWORD$PTR\[$arg5+rsp\]\n" if ($narg>4); + $args .= " mov r9,QWORD$PTR\[$arg6+rsp\]\n" if ($narg>5); + } + $current_function->{narg} = -1; + $args; + } + sub out { + my $self = shift; + + if ($gas) { + my $func = ($globals{$self->{value}} or $self->{value}) . ":"; + if ($current_function->{name} eq $self->{value}) { + $current_function->{pc} = 0; + $func .= "\n.cfi_".cfi_directive::startproc() if ($dwarf); + $func .= "\n .byte 0xf3,0x0f,0x1e,0xfa\n"; # endbranch + if ($win64) { + if ($current_function->{abi} eq "svr4") { + my $fp = $current_function->{unwind} ? "%r11" : "%rax"; + $func .= " movq %rdi,8(%rsp)\n"; + $func .= " movq %rsi,16(%rsp)\n"; + $func .= " movq %rsp,$fp\n"; + $func .= "${decor}SEH_begin_$current_function->{name}:\n"; + } elsif ($current_function->{unwind}) { + $func .= " movq %rsp,%r11\n"; + $func .= "${decor}SEH_begin_$current_function->{name}:\n"; + } + } + } elsif ($win64 && $current_function->{abi} eq "svr4" + && $current_function->{pc} >= 0) { + $func = win64_args().$func; + } + $func; + } elsif ($self->{value} ne "$current_function->{name}") { + my $func; + if ($win64 && $current_function->{abi} eq "svr4" + && $current_function->{pc} >= 0) { + $func = win64_args(); + } + $func .= $self->{value} . $colon; + $func; + } else { + $current_function->{pc} = 0; + my $func = "$current_function->{name}" . + ($nasm ? ":" : "\tPROC $current_function->{scope}") . + "\n"; + $func .= " DB 243,15,30,250\n"; # endbranch + if ($current_function->{abi} eq "svr4") { + my $fp = $current_function->{unwind} ? "r11" : "rax"; + $func .= " mov QWORD$PTR\[8+rsp\],rdi\t;WIN64 prologue\n"; + $func .= " mov QWORD$PTR\[16+rsp\],rsi\n"; + $func .= " mov $fp,rsp\n"; + $func .= "${decor}SEH_begin_$current_function->{name}${colon}\n"; + } elsif ($current_function->{unwind}) { + $func .= " mov r11,rsp\n"; + $func .= "${decor}SEH_begin_$current_function->{name}${colon}\n"; + } + $func; + } + } +} +{ package expr; # pick up expressions + sub re { + my ($class, $line, $opcode) = @_; + my $self = {}; + my $ret; + + if ($$line =~ /(^[^,]+)/) { + bless $self,$class; + $self->{value} = $1; + $ret = $self; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + + $self->{value} =~ s/\@PLT// if (!$elf); + $self->{value} =~ s/([_a-z][_a-z0-9\$]*)/$globals{$1} or $1/gei; + $self->{value} =~ s/\.L/$decor/g; + $self->{opcode} = $opcode; + } + $ret; + } + sub out { + my $self = shift; + $self->{value}; + } +} + +my @xdata_seg = (".section .xdata", ".align 8"); +my @pdata_seg = (".section .pdata", ".align 4"); + +{ package cfi_directive; + # CFI directives annotate instructions that are significant for + # stack unwinding procedure compliant with DWARF specification, + # see http://dwarfstd.org/. Besides naturally expected for this + # script platform-specific filtering function, this module adds + # four auxiliary synthetic directives not recognized by [GNU] + # assembler: + # + # - .cfi_push to annotate push instructions in prologue, which + # translates to .cfi_adjust_cfa_offset (if needed) and + # .cfi_offset; + # - .cfi_pop to annotate pop instructions in epilogue, which + # translates to .cfi_adjust_cfa_offset (if needed) and + # .cfi_restore; + # - .cfi_alloca to annotate stack pointer adjustments, which + # translates to .cfi_adjust_cfa_offset as needed; + # - [and most notably] .cfi_cfa_expression which encodes + # DW_CFA_def_cfa_expression and passes it to .cfi_escape as + # byte vector; + # + # CFA expressions were introduced in DWARF specification version + # 3 and describe how to deduce CFA, Canonical Frame Address. This + # becomes handy if your stack frame is variable and you can't + # spare register for [previous] frame pointer. Suggested directive + # syntax is made-up mix of DWARF operator suffixes [subset of] + # and references to registers with optional bias. Following example + # describes offloaded *original* stack pointer at specific offset + # from *current* stack pointer: + # + # .cfi_cfa_expression %rsp+40,deref,+8 + # + # Final +8 has everything to do with the fact that CFA is defined + # as reference to top of caller's stack, and on x86_64 call to + # subroutine pushes 8-byte return address. In other words original + # stack pointer upon entry to a subroutine is 8 bytes off from CFA. + # + # In addition the .cfi directives are re-purposed even for Win64 + # stack unwinding. Two more synthetic directives were added: + # + # - .cfi_end_prologue to denote point when all non-volatile + # registers are saved and stack or [chosen] frame pointer is + # stable; + # - .cfi_epilogue to denote point when all non-volatile registers + # are restored [and it even adds missing .cfi_restore-s]; + # + # Though it's not universal "miracle cure," it has its limitations. + # Most notably .cfi_cfa_expression won't start working... For more + # information see the end of this file. + + # Below constants are taken from "DWARF Expressions" section of the + # DWARF specification, section is numbered 7.7 in versions 3 and 4. + my %DW_OP_simple = ( # no-arg operators, mapped directly + deref => 0x06, dup => 0x12, + drop => 0x13, over => 0x14, + pick => 0x15, swap => 0x16, + rot => 0x17, xderef => 0x18, + + abs => 0x19, and => 0x1a, + div => 0x1b, minus => 0x1c, + mod => 0x1d, mul => 0x1e, + neg => 0x1f, not => 0x20, + or => 0x21, plus => 0x22, + shl => 0x24, shr => 0x25, + shra => 0x26, xor => 0x27, + ); + + my %DW_OP_complex = ( # used in specific subroutines + constu => 0x10, # uleb128 + consts => 0x11, # sleb128 + plus_uconst => 0x23, # uleb128 + lit0 => 0x30, # add 0-31 to opcode + reg0 => 0x50, # add 0-31 to opcode + breg0 => 0x70, # add 0-31 to opcole, sleb128 + regx => 0x90, # uleb28 + fbreg => 0x91, # sleb128 + bregx => 0x92, # uleb128, sleb128 + piece => 0x93, # uleb128 + ); + + # Following constants are defined in x86_64 ABI supplement, for + # example available at https://www.uclibc.org/docs/psABI-x86_64.pdf, + # see section 3.7 "Stack Unwind Algorithm". + my %DW_reg_idx = ( + "%rax"=>0, "%rdx"=>1, "%rcx"=>2, "%rbx"=>3, + "%rsi"=>4, "%rdi"=>5, "%rbp"=>6, "%rsp"=>7, + "%r8" =>8, "%r9" =>9, "%r10"=>10, "%r11"=>11, + "%r12"=>12, "%r13"=>13, "%r14"=>14, "%r15"=>15 + ); + + my ($cfa_reg, $cfa_off, $cfa_rsp, %saved_regs); + my @cfa_stack; + + sub cfa_rsp { return $cfa_rsp // -8; } + + # [us]leb128 format is variable-length integer representation base + # 2^128, with most significant bit of each byte being 0 denoting + # *last* most significant digit. See "Variable Length Data" in the + # DWARF specification, numbered 7.6 at least in versions 3 and 4. + sub sleb128 { + use integer; # get right shift extend sign + + my $val = shift; + my $sign = ($val < 0) ? -1 : 0; + my @ret = (); + + while(1) { + push @ret, $val&0x7f; + + # see if remaining bits are same and equal to most + # significant bit of the current digit, if so, it's + # last digit... + last if (($val>>6) == $sign); + + @ret[-1] |= 0x80; + $val >>= 7; + } + + return @ret; + } + sub uleb128 { + my $val = shift; + my @ret = (); + + while(1) { + push @ret, $val&0x7f; + + # see if it's last significant digit... + last if (($val >>= 7) == 0); + + @ret[-1] |= 0x80; + } + + return @ret; + } + sub const { + my $val = shift; + + if ($val >= 0 && $val < 32) { + return ($DW_OP_complex{lit0}+$val); + } + return ($DW_OP_complex{consts}, sleb128($val)); + } + sub reg { + my $val = shift; + + return if ($val !~ m/^(%r\w+)(?:([\+\-])((?:0x)?[0-9a-f]+))?/); + + my $reg = $DW_reg_idx{$1}; + my $off = eval ("0 $2 $3"); + + return (($DW_OP_complex{breg0} + $reg), sleb128($off)); + # Yes, we use DW_OP_bregX+0 to push register value and not + # DW_OP_regX, because latter would require even DW_OP_piece, + # which would be a waste under the circumstances. If you have + # to use DWP_OP_reg, use "regx:N"... + } + sub cfa_expression { + my $line = shift; + my @ret; + + foreach my $token (split(/,\s*/,$line)) { + if ($token =~ /^%r/) { + push @ret,reg($token); + } elsif ($token =~ /((?:0x)?[0-9a-f]+)\((%r\w+)\)/) { + push @ret,reg("$2+$1"); + } elsif ($token =~ /(\w+):(\-?(?:0x)?[0-9a-f]+)(U?)/i) { + my $i = 1*eval($2); + push @ret,$DW_OP_complex{$1}, ($3 ? uleb128($i) : sleb128($i)); + } elsif (my $i = 1*eval($token) or $token eq "0") { + if ($token =~ /^\+/) { + push @ret,$DW_OP_complex{plus_uconst},uleb128($i); + } else { + push @ret,const($i); + } + } else { + push @ret,$DW_OP_simple{$token}; + } + } + + # Finally we return DW_CFA_def_cfa_expression, 15, followed by + # length of the expression and of course the expression itself. + return (15,scalar(@ret),@ret); + } + + # Following constants are defined in "x64 exception handling" at + # https://docs.microsoft.com/ and match the register sequence in + # CONTEXT structure defined in winnt.h. + my %WIN64_reg_idx = ( + "%rax"=>0, "%rcx"=>1, "%rdx"=>2, "%rbx"=>3, + "%rsp"=>4, "%rbp"=>5, "%rsi"=>6, "%rdi"=>7, + "%r8" =>8, "%r9" =>9, "%r10"=>10, "%r11"=>11, + "%r12"=>12, "%r13"=>13, "%r14"=>14, "%r15"=>15 + ); + sub xdata { + our @dat = (); + our $len = 0; + + sub savereg { + my ($key, $offset) = @_; + + if ($key =~ /%xmm([0-9]+)/) { + if ($offset < 0x100000) { + push @dat, [0,($1<<4)|8,unpack("C2",pack("v",$offset>>4))]; + } else { + push @dat, [0,($1<<4)|9,unpack("C4",pack("V",$offset))]; + } + } else { + if ($offset < 0x80000) { + push @dat, [0,(($WIN64_reg_idx{$key})<<4)|4, + unpack("C2",pack("v",$offset>>3))]; + } else { + push @dat, [0,(($WIN64_reg_idx{$key})<<4)|5, + unpack("C4",pack("V",$offset))]; + } + } + $len += $#{@dat[-1]}+1; + } + + my $fp_info = 0; + + # allocate stack frame + if ($cfa_rsp < -8) { + my $offset = -8 - $cfa_rsp; + if ($cfa_reg ne "%rsp" && $saved_regs{$cfa_reg} == -16) { + $fp_info = $WIN64_reg_idx{$cfa_reg}; + push @dat, [0,$fp_info<<4]; # UWOP_PUSH_NONVOL + $len += $#{@dat[-1]}+1; + $offset -= 8; + } + if ($offset <= 128) { + my $alloc = ($offset - 8) >> 3; + push @dat, [0,$alloc<<4|2]; # UWOP_ALLOC_SMALL + } elsif ($offset < 0x80000) { + push @dat, [0,0x01,unpack("C2",pack("v",$offset>>3))]; + } else { + push @dat, [0,0x11,unpack("C4",pack("V",$offset))]; + } + $len += $#{@dat[-1]}+1; + } + + # save frame pointer [if not pushed already] + if ($cfa_reg ne "%rsp" && $fp_info == 0) { + $fp_info = $WIN64_reg_idx{$cfa_reg}; + if (defined(my $offset = $saved_regs{$cfa_reg})) { + $offset -= $cfa_rsp; + savereg($cfa_reg, $offset); + } + } + + # set up frame pointer + if ($fp_info) { + push @dat, [0,($fp_info<<4)|3]; # UWOP_SET_FPREG + $len += $#{@dat[-1]}+1; + my $fp_off = $cfa_off - $cfa_rsp; + ($fp_off > 240 or $fp_off&0xf) and die "invalid FP offset $fp_off"; + $fp_info |= $fp_off&-16; + } + + # save registers + foreach my $key (sort { $saved_regs{$b} <=> $saved_regs{$a} } + keys(%saved_regs)) { + next if ($cfa_reg ne "%rsp" && $cfa_reg eq $key); + my $offset = $saved_regs{$key} - $cfa_rsp; + savereg($key, $offset); + } + + my @ret; + # generate 4-byte descriptor + push @ret, ".byte 1,0,".($len/2).",$fp_info"; + $len += 4; + # keep objdump happy, pad to 4*n and add a 32-bit zero + unshift @dat, [(0)x(((-$len)&3)+4)]; + $len += $#{@dat[0]}+1; + # pad to 8*n + unshift @dat, [(0)x((-$len)&7)] if ($len&7); + # emit data + while(defined(my $row = pop @dat)) { + push @ret, ".byte ". join(",", + map { sprintf "0x%02x",$_ } @{$row}); + } + + return @ret; + } + sub startproc { + return if ($cfa_rsp == -8); + ($cfa_reg, $cfa_off, $cfa_rsp) = ("%rsp", -8, -8); + %saved_regs = (); + return "startproc"; + } + sub endproc { + return if ($cfa_rsp == 0); + ($cfa_reg, $cfa_off, $cfa_rsp) = ("%rsp", 0, 0); + %saved_regs = (); + return "endproc"; + } + sub re { + my ($class, $line) = @_; + my $self = {}; + my $ret; + + if ($$line =~ s/^\s*\.cfi_(\w+)\s*//) { + bless $self,$class; + $ret = $self; + undef $self->{value}; + my $dir = $1; + + SWITCH: for ($dir) { + # What is $cfa_rsp? Effectively it's difference between %rsp + # value and current CFA, Canonical Frame Address, which is + # why it starts with -8. Recall that CFA is top of caller's + # stack... + /startproc/ && do { $dir = startproc(); last; }; + /endproc/ && do { $dir = endproc(); + # .cfi_remember_state directives that are not + # matched with .cfi_restore_state are + # unnecessary. + die "unpaired .cfi_remember_state" if (@cfa_stack); + last; + }; + /def_cfa_register/ + && do { $cfa_off = $cfa_rsp if ($cfa_reg eq "%rsp"); + $cfa_reg = $$line; + $cfa_rsp = $cfa_off if ($cfa_reg eq "%rsp"); + last; + }; + /def_cfa_offset/ + && do { $cfa_off = -1*eval($$line); + $cfa_rsp = $cfa_off if ($cfa_reg eq "%rsp"); + last; + }; + /adjust_cfa_offset/ + && do { my $val = 1*eval($$line); + $cfa_off -= $val; + if ($cfa_reg eq "%rsp") { + $cfa_rsp -= $val; + } + $$line = "$val"; + last; + }; + /alloca/ && do { $dir = undef; + my $val = 1*eval($$line); + $cfa_rsp -= $val; + if ($cfa_reg eq "%rsp") { + $cfa_off -= $val; + $dir = "adjust_cfa_offset"; + } + $$line = "$val"; + last; + }; + /def_cfa/ && do { if ($$line =~ /(%r\w+)\s*(?:,\s*(.+))?/) { + $cfa_reg = $1; + if ($cfa_reg eq "%rsp" && !defined($2)) { + $cfa_off = $cfa_rsp; + $$line .= ",".(-$cfa_rsp); + } else { + $cfa_off = -1*eval($2); + $cfa_rsp = $cfa_off if ($cfa_reg eq "%rsp"); + } + } + last; + }; + /push/ && do { $dir = undef; + $cfa_rsp -= 8; + if ($cfa_reg eq "%rsp") { + $cfa_off = $cfa_rsp; + $self->{value} = ".cfi_adjust_cfa_offset\t8\n"; + } + $saved_regs{$$line} = $cfa_rsp; + $self->{value} .= ".cfi_offset\t$$line,$cfa_rsp"; + last; + }; + /pop/ && do { $dir = undef; + $cfa_rsp += 8; + if ($cfa_reg eq "%rsp") { + $cfa_off = $cfa_rsp; + $self->{value} = ".cfi_adjust_cfa_offset\t-8\n"; + } + $self->{value} .= ".cfi_restore\t$$line"; + delete $saved_regs{$$line}; + last; + }; + /cfa_expression/ + && do { $dir = undef; + $self->{value} = ".cfi_escape\t" . + join(",", map(sprintf("0x%02x", $_), + cfa_expression($$line))); + last; + }; + /remember_state/ + && do { push @cfa_stack, + [$cfa_reg,$cfa_off,$cfa_rsp,%saved_regs]; + last; + }; + /restore_state/ + && do { ($cfa_reg,$cfa_off,$cfa_rsp,%saved_regs) + = @{pop @cfa_stack}; + last; + }; + /offset/ && do { if ($$line =~ /(%\w+)(?:-%xmm(\d+))?\s*,\s*(.+)/) { + my ($reg, $off, $xmmlast) = ($1, 1*eval($3), $2); + if ($reg !~ /%xmm(\d+)/) { + $saved_regs{$reg} = $off; + } else { + $dir = undef; + $xmmlast //= $1; + for (my $i=$1; $i<=$xmmlast; $i++) { + $saved_regs{"%xmm$i"} = $off; + $off += 16; + } + } + } + last; + }; + /restore/ && do { delete $saved_regs{$$line}; last; }; + /end_prologue/ + && do { $dir = undef; + $self->{win64} = ".endprolog"; + last; + }; + /epilogue/ && do { $dir = undef; + $self->{win64} = ".epilogue"; + $self->{value} = join("\n", + map { ".cfi_restore\t$_" } + sort keys(%saved_regs)); + %saved_regs = (); + last; + }; + } + + $self->{value} = ".cfi_$dir\t$$line" if ($dir); + + $$line = ""; + } + + return $ret; + } + sub out { + my $self = shift; + return $self->{value} if ($dwarf); + + if ($win64 and $current_function->{unwind} + and my $ret = $self->{win64}) { + my ($reg, $off) = ($cfa_reg =~ /%(?!rsp)/) ? ($', $cfa_off) + : ("rsp", $cfa_rsp); + my $fname = $current_function->{name}; + + if ($ret eq ".endprolog") { + $ret = ""; + if ($current_function->{abi} eq "svr4") { + $ret .= label::win64_args(); + $saved_regs{"%rdi"} = 0; # relative to CFA, remember? + $saved_regs{"%rsi"} = 8; + } + + push @pdata_seg, + ".rva .LSEH_begin_${fname}", + ".rva .LSEH_body_${fname}", + ".rva .LSEH_info_${fname}_prologue",""; + push @xdata_seg, + ".LSEH_info_${fname}_prologue:"; + if ($current_function->{unwind} eq "%rbp") { + if ($current_function->{abi} eq "svr4") { + push @xdata_seg, + ".byte 1,4,6,0x05", # 6 unwind codes, %rbp is FP + ".byte 4,0x74,2,0", # %rdi at 16(%rsp) + ".byte 4,0x64,3,0", # %rsi at 24(%rsp) + ".byte 4,0x53", # mov %rsp, %rbp + ".byte 1,0x50", # push %rbp + ".long 0,0" # pad to keep objdump happy + ; + } else { + push @xdata_seg, + ".byte 1,4,2,0x05", # 2 unwind codes, %rbp is FP + ".byte 4,0x53", # mov %rsp, %rbp + ".byte 1,0x50", # push %rbp + ".long 0,0" # pad to keep objdump happy + ; + } + } else { + if ($current_function->{abi} eq "svr4") { + push @xdata_seg, + ".byte 1,0,5,0x0b", # 5 unwind codes, %r11 is FP + ".byte 0,0x74,1,0", # %rdi at 8(%rsp) + ".byte 0,0x64,2,0", # %rsi at 16(%rsp) + ".byte 0,0xb3", # set frame pointer + ".byte 0,0", # padding + ".long 0,0" # pad to keep objdump happy + ; + } else { + push @xdata_seg, + ".byte 1,0,1,0x0b", # 1 unwind code, %r11 is FP + ".byte 0,0xb3", # set frame pointer + ".byte 0,0", # padding + ".long 0,0" # pad to keep objdump happy + ; + } + } + push @pdata_seg, + ".rva .LSEH_body_${fname}", + ".rva .LSEH_epilogue_${fname}", + ".rva .LSEH_info_${fname}_body",""; + push @xdata_seg,".LSEH_info_${fname}_body:", xdata(); + $ret .= "${decor}SEH_body_${fname}${colon}\n"; + } elsif ($ret eq ".epilogue") { + %saved_regs = (); + $cfa_rsp = $cfa_off; + $ret = "${decor}SEH_epilogue_${fname}${colon}\n"; + if ($current_function->{abi} eq "svr4") { + $saved_regs{"%rdi"} = 0; # relative to CFA, remember? + $saved_regs{"%rsi"} = 8; + + push @pdata_seg, + ".rva .LSEH_epilogue_${fname}", + ".rva .LSEH_end_${fname}", + ".rva .LSEH_info_${fname}_epilogue",""; + push @xdata_seg,".LSEH_info_${fname}_epilogue:", xdata(), ""; + if ($gas) { + $ret .= " mov ".(0-$off)."(%$reg),%rdi\n"; + $ret .= " mov ".(8-$off)."(%$reg),%rsi\n"; + } else { + $ret .= " mov rdi,QWORD$PTR\[".(0-$off)."+$reg\]"; + $ret .= " ;WIN64 epilogue\n"; + $ret .= " mov rsi,QWORD$PTR\[".(8-$off)."+$reg\]\n"; + } + } + } + return $ret; + } + return; + } +} +{ package directive; # pick up directives, which start with . + sub re { + my ($class, $line) = @_; + my $self = {}; + my $ret; + my $dir; + + # chain-call to cfi_directive + $ret = cfi_directive->re($line) and return $ret; + + if ($$line =~ /^\s*(\.\w+)/) { + bless $self,$class; + $dir = $1; + $ret = $self; + undef $self->{value}; + $$line = substr($$line,@+[0]); $$line =~ s/^\s+//; + + SWITCH: for ($dir) { + /\.global|\.globl|\.extern|\.comm/ + && do { $$line =~ s/([_a-z][_a-z0-9\$]*)/$prefix\1/gi; + $globals{$1} = $prefix.$1 if ($1); + last; + }; + /\.type/ && do { my ($sym,$type,$narg,$unwind) = split(',',$$line); + if ($type eq "\@function") { + undef $current_function; + $current_function->{name} = $sym; + $current_function->{abi} = "svr4"; + $current_function->{narg} = $narg; + $current_function->{scope} = defined($globals{$sym})?"PUBLIC":"PRIVATE"; + $current_function->{unwind} = $unwind; + $current_function->{pc} = -1; + } elsif ($type eq "\@abi-omnipotent") { + undef $current_function; + $current_function->{name} = $sym; + $current_function->{scope} = defined($globals{$sym})?"PUBLIC":"PRIVATE"; + $current_function->{unwind} = $unwind; + $current_function->{pc} = -1; + } + $$line =~ s/\@abi\-omnipotent/\@function/; + $$line =~ s/\@function.*/\@function/; + last; + }; + /\.asciz/ && do { if ($$line =~ /^"(.*)"$/) { + $dir = ".byte"; + $$line = join(",",unpack("C*",$1),0); + } + last; + }; + /\.rva|\.long|\.quad/ + && do { $$line =~ s/([_a-z][_a-z0-9\$]*)/$globals{$1} or $1/gei; + $$line =~ s/\.L/$decor/g; + last; + }; + } + + if ($gas) { + $self->{value} = $dir . "\t" . $$line; + + if ($dir =~ /\.extern/) { + $self->{value} = ""; # swallow extern + } elsif (!$elf && $dir =~ /\.type/) { + $self->{value} = ""; + $self->{value} = ".def\t" . ($globals{$1} or $1) . ";\t" . + (defined($globals{$1})?".scl 2;":".scl 3;") . + "\t.type 32;\t.endef" + if ($win64 && $$line =~ /([^,]+),\@function/); + } elsif ($dir =~ /\.size/) { + $self->{value} = "" if (!$elf); + if ($dwarf and my $endproc = cfi_directive::endproc()) { + $self->{value} = ".cfi_$endproc\n$self->{value}"; + } elsif (!$elf && defined($current_function)) { + $self->{value} .= "${decor}SEH_end_$current_function->{name}:" + if ($win64 && $current_function->{abi} eq "svr4"); + undef $current_function; + } + } elsif (!$elf && $dir =~ /\.align/) { + $self->{value} = ".p2align\t" . (log($$line)/log(2)); + } elsif ($dir eq ".section") { + $current_segment=$$line; + if (!$elf && $current_segment eq ".init") { + if ($flavour eq "macosx") { $self->{value} = ".mod_init_func"; } + elsif ($flavour eq "mingw64") { $self->{value} = ".section\t.ctors"; } + } + if (!$elf && $current_segment eq ".rodata") { + if ($flavour eq "macosx") { $self->{value} = ".section\t__TEXT,__const"; } + elsif ($flavour eq "mingw64") { $self->{value} = ".section\t.rdata"; } + } + } elsif ($dir =~ /\.(text|data)/) { + $current_segment=".$1"; + } elsif ($dir =~ /\.hidden/) { + if ($flavour eq "macosx") { $self->{value} = ".private_extern\t$prefix$$line"; } + elsif ($flavour eq "mingw64") { $self->{value} = ""; } + } elsif ($dir =~ /\.comm/) { + $self->{value} = "$dir\t$$line"; + $self->{value} =~ s|,([0-9]+),([0-9]+)$|",$1,".log($2)/log(2)|e if ($flavour eq "macosx"); + } + $$line = ""; + return $self; + } + + # non-gas case or nasm/masm + SWITCH: for ($dir) { + /\.text/ && do { my $v=undef; + if ($nasm) { + $v="section .text code align=64\n"; + } else { + $v="$current_segment\tENDS\n" if ($current_segment); + $current_segment = ".text\$"; + $v.="$current_segment\tSEGMENT "; + $v.=$masm>=$masmref ? "ALIGN(256)" : "PAGE"; + $v.=" 'CODE'"; + } + $self->{value} = $v; + last; + }; + /\.data/ && do { my $v=undef; + if ($nasm) { + $v="section .data data align=8\n"; + } else { + $v="$current_segment\tENDS\n" if ($current_segment); + $current_segment = "_DATA"; + $v.="$current_segment\tSEGMENT"; + } + $self->{value} = $v; + last; + }; + /\.section/ && do { my $v=undef; + $$line =~ s/([^,]*).*/$1/; + $$line = ".CRT\$XCU" if ($$line eq ".init"); + $$line = ".rdata" if ($$line eq ".rodata"); + my %align = ( p=>4, x=>8, r=>256); + if ($nasm) { + $v="section $$line"; + if ($$line=~/\.([pxr])data/) { + $v.=" rdata align=$align{$1}"; + } elsif ($$line=~/\.CRT\$/i) { + $v.=" rdata align=8"; + } + } else { + $v="$current_segment\tENDS\n" if ($current_segment); + $v.="$$line\tSEGMENT"; + if ($$line=~/\.([pxr])data/) { + $v.=" READONLY"; + $v.=" ALIGN($align{$1})" if ($masm>=$masmref); + } elsif ($$line=~/\.CRT\$/i) { + $v.=" READONLY "; + $v.=$masm>=$masmref ? "ALIGN(8)" : "DWORD"; + } + } + $current_segment = $$line; + $self->{value} = $v; + last; + }; + /\.extern/ && do { $self->{value} = "EXTERN\t".$$line; + $self->{value} .= ":NEAR" if ($masm); + last; + }; + /\.globl|.global/ + && do { $self->{value} = $masm?"PUBLIC":"global"; + $self->{value} .= "\t".$$line; + last; + }; + /\.size/ && do { if (defined($current_function)) { + undef $self->{value}; + if ($current_function->{abi} eq "svr4") { + $self->{value}="${decor}SEH_end_$current_function->{name}${colon}\n"; + } + $self->{value}.="$current_function->{name}\tENDP" if($masm && $current_function->{name}); + undef $current_function; + } + last; + }; + /\.align/ && do { my $max = ($masm && $masm>=$masmref) ? 256 : 4096; + $self->{value} = "ALIGN\t".($$line>$max?$max:$$line); + last; + }; + /\.(value|long|rva|quad)/ + && do { my $sz = substr($1,0,1); + my @arr = split(/,\s*/,$$line); + my $last = pop(@arr); + my $conv = sub { my $var=shift; + $var=~s/^(0b[0-1]+)/oct($1)/eig; + $var=~s/^0x([0-9a-f]+)/0$1h/ig if ($masm); + if ($sz eq "D" && ($current_segment=~/.[px]data/ || $dir eq ".rva")) + { $var=~s/^([_a-z\$\@][_a-z0-9\$\@]*)/$nasm?"$1 wrt ..imagebase":"imagerel $1"/egi; } + $var; + }; + + $sz =~ tr/bvlrq/BWDDQ/; + $self->{value} = "\tD$sz\t"; + for (@arr) { $self->{value} .= &$conv($_).","; } + $self->{value} .= &$conv($last); + last; + }; + /\.byte/ && do { my @str=split(/,\s*/,$$line); + map(s/(0b[0-1]+)/oct($1)/eig,@str); + map(s/0x([0-9a-f]+)/0$1h/ig,@str) if ($masm); + while ($#str>15) { + $self->{value}.="DB\t" + .join(",",@str[0..15])."\n"; + foreach (0..15) { shift @str; } + } + $self->{value}.="DB\t" + .join(",",@str) if (@str); + last; + }; + /\.comm/ && do { my @str=split(/,\s*/,$$line); + my $v=undef; + if ($nasm) { + $v.="common $prefix@str[0] @str[1]"; + } else { + $v="$current_segment\tENDS\n" if ($current_segment); + $current_segment = "_DATA"; + $v.="$current_segment\tSEGMENT\n"; + $v.="COMM @str[0]:DWORD:".@str[1]/4; + } + $self->{value} = $v; + last; + }; + } + $$line = ""; + } + + $ret; + } + sub out { + my $self = shift; + $self->{value}; + } +} + +# Upon initial x86_64 introduction SSE>2 extensions were not introduced +# yet. In order not to be bothered by tracing exact assembler versions, +# but at the same time to provide a bare security minimum of AES-NI, we +# hard-code some instructions. Extensions past AES-NI on the other hand +# are traced by examining assembler version in individual perlasm +# modules... + +my %regrm = ( "%eax"=>0, "%ecx"=>1, "%edx"=>2, "%ebx"=>3, + "%esp"=>4, "%ebp"=>5, "%esi"=>6, "%edi"=>7 ); + +sub rex { + my $opcode=shift; + my ($dst,$src,$rex)=@_; + + $rex|=0x04 if($dst>=8); + $rex|=0x01 if($src>=8); + push @$opcode,($rex|0x40) if ($rex); +} + +my $movq = sub { # elderly gas can't handle inter-register movq + my $arg = shift; + my @opcode=(0x66); + if ($arg =~ /%xmm([0-9]+),\s*%r(\w+)/) { + my ($src,$dst)=($1,$2); + if ($dst !~ /[0-9]+/) { $dst = $regrm{"%e$dst"}; } + rex(\@opcode,$src,$dst,0x8); + push @opcode,0x0f,0x7e; + push @opcode,0xc0|(($src&7)<<3)|($dst&7); # ModR/M + @opcode; + } elsif ($arg =~ /%r(\w+),\s*%xmm([0-9]+)/) { + my ($src,$dst)=($2,$1); + if ($dst !~ /[0-9]+/) { $dst = $regrm{"%e$dst"}; } + rex(\@opcode,$src,$dst,0x8); + push @opcode,0x0f,0x6e; + push @opcode,0xc0|(($src&7)<<3)|($dst&7); # ModR/M + @opcode; + } else { + (); + } +}; + +my $pextrd = sub { + if (shift =~ /\$([0-9]+),\s*%xmm([0-9]+),\s*(%\w+)/) { + my @opcode=(0x66); + my $imm=$1; + my $src=$2; + my $dst=$3; + if ($dst =~ /%r([0-9]+)d/) { $dst = $1; } + elsif ($dst =~ /%e/) { $dst = $regrm{$dst}; } + rex(\@opcode,$src,$dst); + push @opcode,0x0f,0x3a,0x16; + push @opcode,0xc0|(($src&7)<<3)|($dst&7); # ModR/M + push @opcode,$imm; + @opcode; + } else { + (); + } +}; + +my $pinsrd = sub { + if (shift =~ /\$([0-9]+),\s*(%\w+),\s*%xmm([0-9]+)/) { + my @opcode=(0x66); + my $imm=$1; + my $src=$2; + my $dst=$3; + if ($src =~ /%r([0-9]+)/) { $src = $1; } + elsif ($src =~ /%e/) { $src = $regrm{$src}; } + rex(\@opcode,$dst,$src); + push @opcode,0x0f,0x3a,0x22; + push @opcode,0xc0|(($dst&7)<<3)|($src&7); # ModR/M + push @opcode,$imm; + @opcode; + } else { + (); + } +}; + +my $pshufb = sub { + if (shift =~ /%xmm([0-9]+),\s*%xmm([0-9]+)/) { + my @opcode=(0x66); + rex(\@opcode,$2,$1); + push @opcode,0x0f,0x38,0x00; + push @opcode,0xc0|($1&7)|(($2&7)<<3); # ModR/M + @opcode; + } else { + (); + } +}; + +my $palignr = sub { + if (shift =~ /\$([0-9]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) { + my @opcode=(0x66); + rex(\@opcode,$3,$2); + push @opcode,0x0f,0x3a,0x0f; + push @opcode,0xc0|($2&7)|(($3&7)<<3); # ModR/M + push @opcode,$1; + @opcode; + } else { + (); + } +}; + +my $pclmulqdq = sub { + if (shift =~ /\$([x0-9a-f]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) { + my @opcode=(0x66); + rex(\@opcode,$3,$2); + push @opcode,0x0f,0x3a,0x44; + push @opcode,0xc0|($2&7)|(($3&7)<<3); # ModR/M + my $c=$1; + push @opcode,$c=~/^0/?oct($c):$c; + @opcode; + } else { + (); + } +}; + +my $rdrand = sub { + if (shift =~ /%[er](\w+)/) { + my @opcode=(); + my $dst=$1; + if ($dst !~ /[0-9]+/) { $dst = $regrm{"%e$dst"}; } + rex(\@opcode,0,$dst,8); + push @opcode,0x0f,0xc7,0xf0|($dst&7); + @opcode; + } else { + (); + } +}; + +my $rdseed = sub { + if (shift =~ /%[er](\w+)/) { + my @opcode=(); + my $dst=$1; + if ($dst !~ /[0-9]+/) { $dst = $regrm{"%e$dst"}; } + rex(\@opcode,0,$dst,8); + push @opcode,0x0f,0xc7,0xf8|($dst&7); + @opcode; + } else { + (); + } +}; + +# Not all AVX-capable assemblers recognize AMD XOP extension. Since we +# are using only two instructions hand-code them in order to be excused +# from chasing assembler versions... + +sub rxb { + my $opcode=shift; + my ($dst,$src1,$src2,$rxb)=@_; + + $rxb|=0x7<<5; + $rxb&=~(0x04<<5) if($dst>=8); + $rxb&=~(0x01<<5) if($src1>=8); + $rxb&=~(0x02<<5) if($src2>=8); + push @$opcode,$rxb; +} + +my $vprotd = sub { + if (shift =~ /\$([x0-9a-f]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) { + my @opcode=(0x8f); + rxb(\@opcode,$3,$2,-1,0x08); + push @opcode,0x78,0xc2; + push @opcode,0xc0|($2&7)|(($3&7)<<3); # ModR/M + my $c=$1; + push @opcode,$c=~/^0/?oct($c):$c; + @opcode; + } else { + (); + } +}; + +my $vprotq = sub { + if (shift =~ /\$([x0-9a-f]+),\s*%xmm([0-9]+),\s*%xmm([0-9]+)/) { + my @opcode=(0x8f); + rxb(\@opcode,$3,$2,-1,0x08); + push @opcode,0x78,0xc3; + push @opcode,0xc0|($2&7)|(($3&7)<<3); # ModR/M + my $c=$1; + push @opcode,$c=~/^0/?oct($c):$c; + @opcode; + } else { + (); + } +}; + +# Intel Control-flow Enforcement Technology extension. All functions and +# indirect branch targets will have to start with this instruction... +# However, it should not be used in functions' prologues explicitly, as +# it's added automatically [and in the right spot]. Which leaves only +# non-function indirect branch targets, such as in a case-like dispatch +# table, as application area. + +my $endbr64 = sub { + (0xf3,0x0f,0x1e,0xfa); +}; + +######################################################################## + +my $preproc_prefix = "#"; + +if ($nasm) { + $preproc_prefix = "%"; + print <<___; +default rel +%define XMMWORD +%define YMMWORD +%define ZMMWORD +___ +} elsif ($masm) { + $preproc_prefix = ""; + print <<___; +OPTION DOTNAME +___ +} + +sub process { + my $line = shift; + + $line =~ s|\R$||; # Better chomp + + if ($line =~ m/^#\s*(if|elif|else|endif)(.*)/) { # pass through preproc + if ($win64 && $current_function->{abi} eq "svr4" + && $current_function->{narg} >= 0) { + print label::win64_args(); + } + print $preproc_prefix,$1,$2,"\n"; + next; + } + + print $1 if ($line =~ s|(\{\w+\})||); + + $line =~ s|[#!].*$||; # get rid of asm-style comments... + $line =~ s|/\*.*\*/||; # ... and C-style comments... + $line =~ s|^\s+||; # ... and skip white spaces in beginning + $line =~ s|\s+$||; # ... and at the end + + if (my $label=label->re(\$line)) { print $label->out(); } + + if (my $directive=directive->re(\$line)) { + printf "%s",$directive->out(); + } elsif (my $opcode=opcode->re(\$line)) { + my $asm = eval("\$".$opcode->mnemonic()); + + if ((ref($asm) eq 'CODE') && scalar(my @bytes=&$asm($line))) { + print $gas?".byte\t":"DB\t",join(',',@bytes),"\n"; + next; + } + + my @args; + ARGUMENT: while (1) { + my $arg; + + ($arg=register->re(\$line, $opcode))|| + ($arg=const->re(\$line)) || + ($arg=ea->re(\$line, $opcode)) || + ($arg=expr->re(\$line, $opcode)) || + last ARGUMENT; + + push @args,$arg; + + last ARGUMENT if ($line !~ /^,/); + + $line =~ s/^,\s*//; + } # ARGUMENT: + + if ($win64 && $current_function->{abi} eq "svr4" + && $current_function->{narg} >= 0) { + my $pc = $current_function->{pc}; + my $op = $opcode->{op}; + my $a0 = @args[0]->{value} if ($#args>=0); + if (!$current_function->{unwind} + || $pc == 0 && !($op eq "push" && $a0 eq "rbp") + || $pc == 1 && !($op eq "mov" && $a0 eq "rsp" + && @args[1]->{value} eq "rbp" + && ($current_function->{unwind} = "%rbp")) + || $pc > 1) { + print label::win64_args(); + } + } + + if ($#args>=0) { + my $insn; + my $sz=$opcode->size(); + + if ($gas) { + $insn = $opcode->out($#args>=1?$args[$#args]->size():$sz); + @args = map($_->out($sz),@args); + printf "\t%s\t%s",$insn,join(",",@args); + } else { + $insn = $opcode->out(); + foreach (@args) { + my $arg = $_->out(); + # $insn.=$sz compensates for movq, pinsrw, ... + if ($arg =~ /^xmm[0-9]+$/) { $insn.=$sz; $sz="x" if(!$sz); last; } + if ($arg =~ /^ymm[0-9]+$/) { $insn.=$sz; $sz="y" if(!$sz); last; } + if ($arg =~ /^zmm[0-9]+$/) { $insn.=$sz; $sz="z" if(!$sz); last; } + if ($arg =~ /^mm[0-9]+$/) { $insn.=$sz; $sz="q" if(!$sz); last; } + } + @args = reverse(@args); + undef $sz if ($nasm && $opcode->mnemonic() eq "lea"); + printf "\t%s\t%s",$insn,join(",",map($_->out($sz),@args)); + } + } else { + printf "\t%s",$opcode->out(); + } + + ++$current_function->{pc} if (defined($current_function)); + } + + print $line,"\n"; +} + +while(<>) { process($_); } + +map { process($_) } @pdata_seg if ($win64 && $#pdata_seg>1); +map { process($_) } @xdata_seg if ($win64 && $#xdata_seg>1); + +# platform-specific epilogue +if ($masm) { + print "\n$current_segment\tENDS\n" if ($current_segment); + print "END\n"; +} elsif ($elf) { + # -fcf-protection segment, snatched from compiler -S output + my $align = ($flavour =~ /elf32/) ? 4 : 8; + print <<___; + +.section .note.gnu.property,"a",\@note + .long 4,2f-1f,5 + .byte 0x47,0x4E,0x55,0 +1: .long 0xc0000002,4,3 +.align $align +2: +___ +} + +close STDOUT; + + ################################################# +# Cross-reference x86_64 ABI "card" +# +# Unix Win64 +# %rax * * +# %rbx - - +# %rcx #4 #1 +# %rdx #3 #2 +# %rsi #2 - +# %rdi #1 - +# %rbp - - +# %rsp - - +# %r8 #5 #3 +# %r9 #6 #4 +# %r10 * * +# %r11 * * +# %r12 - - +# %r13 - - +# %r14 - - +# %r15 - - +# +# (*) volatile register +# (-) preserved by callee +# (#) Nth argument, volatile +# +# In Unix terms top of stack is argument transfer area for arguments +# which could not be accommodated in registers. Or in other words 7th +# [integer] argument resides at 8(%rsp) upon function entry point. +# 128 bytes above %rsp constitute a "red zone" which is not touched +# by signal handlers and can be used as temporal storage without +# allocating a frame. +# +# In Win64 terms N*8 bytes on top of stack is argument transfer area, +# which belongs to/can be overwritten by callee. N is the number of +# arguments passed to callee, *but* not less than 4! This means that +# upon function entry point 5th argument resides at 40(%rsp), as well +# as that 32 bytes from 8(%rsp) can always be used as temporal +# storage [without allocating a frame]. One can actually argue that +# one can assume a "red zone" above stack pointer under Win64 as well. +# Point is that at apparently no occasion Windows kernel would alter +# the area above user stack pointer in true asynchronous manner... +# +# All the above means that if assembler programmer adheres to Unix +# register and stack layout, but disregards the "red zone" existence, +# it's possible to use following prologue and epilogue to "gear" from +# Unix to Win64 ABI in leaf functions with not more than 6 arguments. +# +# omnipotent_function: +# ifdef WIN64 +# movq %rdi,8(%rsp) +# movq %rsi,16(%rsp) +# movq %rcx,%rdi ; if 1st argument is actually present +# movq %rdx,%rsi ; if 2nd argument is actually ... +# movq %r8,%rdx ; if 3rd argument is ... +# movq %r9,%rcx ; if 4th argument ... +# movq 40(%rsp),%r8 ; if 5th ... +# movq 48(%rsp),%r9 ; if 6th ... +# endif +# ... +# ifdef WIN64 +# movq 8(%rsp),%rdi +# movq 16(%rsp),%rsi +# endif +# ret +# + ################################################# +# Win64 SEH, Structured Exception Handling. +# +# Unlike on Unix systems(*) lack of Win64 stack unwinding information +# has undesired side-effect at run-time: if an exception is raised in +# assembler subroutine such as those in question (basically we're +# referring to segmentation violations caused by malformed input +# parameters), the application is briskly terminated without invoking +# any exception handlers, most notably without generating memory dump +# or any user notification whatsoever. This poses a problem. It's +# possible to address it by registering custom language-specific +# handler that would restore processor context to the state at +# subroutine entry point and return "exception is not handled, keep +# unwinding" code. Writing such handler can be a challenge... But it's +# doable, though requires certain coding convention. Consider following +# snippet: +# +# .type function,@function +# function: +# movq %rsp,%rax # copy rsp to volatile register +# pushq %r15 # save non-volatile registers +# pushq %rbx +# pushq %rbp +# movq %rsp,%r11 +# subq %rdi,%r11 # prepare [variable] stack frame +# andq $-64,%r11 +# movq %rax,0(%r11) # check for exceptions +# movq %r11,%rsp # allocate [variable] stack frame +# movq %rax,0(%rsp) # save original rsp value +# magic_point: +# ... +# movq 0(%rsp),%rcx # pull original rsp value +# movq -24(%rcx),%rbp # restore non-volatile registers +# movq -16(%rcx),%rbx +# movq -8(%rcx),%r15 +# movq %rcx,%rsp # restore original rsp +# magic_epilogue: +# ret +# .size function,.-function +# +# The key is that up to magic_point copy of original rsp value remains +# in chosen volatile register and no non-volatile register, except for +# rsp, is modified. While past magic_point rsp remains constant till +# the very end of the function. In this case custom language-specific +# exception handler would look like this: +# +# EXCEPTION_DISPOSITION handler (EXCEPTION_RECORD *rec,ULONG64 frame, +# CONTEXT *context,DISPATCHER_CONTEXT *disp) +# { ULONG64 *rsp = (ULONG64 *)context->Rax; +# ULONG64 rip = context->Rip; +# +# if (rip >= magic_point) +# { rsp = (ULONG64 *)context->Rsp; +# if (rip < magic_epilogue) +# { rsp = (ULONG64 *)rsp[0]; +# context->Rbp = rsp[-3]; +# context->Rbx = rsp[-2]; +# context->R15 = rsp[-1]; +# } +# } +# context->Rsp = (ULONG64)rsp; +# context->Rdi = rsp[1]; +# context->Rsi = rsp[2]; +# +# memcpy (disp->ContextRecord,context,sizeof(CONTEXT)); +# RtlVirtualUnwind(UNW_FLAG_NHANDLER,disp->ImageBase, +# dips->ControlPc,disp->FunctionEntry,disp->ContextRecord, +# &disp->HandlerData,&disp->EstablisherFrame,NULL); +# return ExceptionContinueSearch; +# } +# +# It's appropriate to implement this handler in assembler, directly in +# function's module. In order to do that one has to know members' +# offsets in CONTEXT and DISPATCHER_CONTEXT structures and some constant +# values. Here they are: +# +# CONTEXT.Rax 120 +# CONTEXT.Rcx 128 +# CONTEXT.Rdx 136 +# CONTEXT.Rbx 144 +# CONTEXT.Rsp 152 +# CONTEXT.Rbp 160 +# CONTEXT.Rsi 168 +# CONTEXT.Rdi 176 +# CONTEXT.R8 184 +# CONTEXT.R9 192 +# CONTEXT.R10 200 +# CONTEXT.R11 208 +# CONTEXT.R12 216 +# CONTEXT.R13 224 +# CONTEXT.R14 232 +# CONTEXT.R15 240 +# CONTEXT.Rip 248 +# CONTEXT.Xmm6 512 +# sizeof(CONTEXT) 1232 +# DISPATCHER_CONTEXT.ControlPc 0 +# DISPATCHER_CONTEXT.ImageBase 8 +# DISPATCHER_CONTEXT.FunctionEntry 16 +# DISPATCHER_CONTEXT.EstablisherFrame 24 +# DISPATCHER_CONTEXT.TargetIp 32 +# DISPATCHER_CONTEXT.ContextRecord 40 +# DISPATCHER_CONTEXT.LanguageHandler 48 +# DISPATCHER_CONTEXT.HandlerData 56 +# UNW_FLAG_NHANDLER 0 +# ExceptionContinueSearch 1 +# +# In order to tie the handler to the function one has to compose +# couple of structures: one for .xdata segment and one for .pdata. +# +# UNWIND_INFO structure for .xdata segment would be +# +# function_unwind_info: +# .byte 9,0,0,0 +# .rva handler +# +# This structure designates exception handler for a function with +# zero-length prologue, no stack frame or frame register. +# +# To facilitate composing of .pdata structures, auto-generated "gear" +# prologue copies rsp value to rax and denotes next instruction with +# .LSEH_begin_{function_name} label. This essentially defines the SEH +# styling rule mentioned in the beginning. Position of this label is +# chosen in such manner that possible exceptions raised in the "gear" +# prologue would be accounted to caller and unwound from latter's frame. +# End of function is marked with respective .LSEH_end_{function_name} +# label. To summarize, .pdata segment would contain +# +# .rva .LSEH_begin_function +# .rva .LSEH_end_function +# .rva function_unwind_info +# +# Reference to function_unwind_info from .xdata segment is the anchor. +# In case you wonder why references are 32-bit .rvas and not 64-bit +# .quads. References put into these two segments are required to be +# *relative* to the base address of the current binary module, a.k.a. +# image base. No Win64 module, be it .exe or .dll, can be larger than +# 2GB and thus such relative references can be and are accommodated in +# 32 bits. +# +# Having reviewed the example function code, one can argue that "movq +# %rsp,%rax" above is redundant. It is not! Keep in mind that on Unix +# rax would contain an undefined value. If this "offends" you, use +# another register and refrain from modifying rax till magic_point is +# reached, i.e. as if it was a non-volatile register. If more registers +# are required prior [variable] frame setup is completed, note that +# nobody says that you can have only one "magic point." You can +# "liberate" non-volatile registers by denoting last stack off-load +# instruction and reflecting it in finer grade unwind logic in handler. +# After all, isn't it why it's called *language-specific* handler... +# +# SE handlers are also involved in unwinding stack when executable is +# profiled or debugged. Profiling implies additional limitations that +# are too subtle to discuss here. For now it's sufficient to say that +# in order to simplify handlers one should either a) offload original +# %rsp to stack (like discussed above); or b) if you have a register to +# spare for frame pointer, choose volatile one. +# +# (*) Note that we're talking about run-time, not debug-time. Lack of +# unwind information makes debugging hard on both Windows and +# Unix. "Unlike" refers to the fact that on Unix signal handler +# will always be invoked, core dumped and appropriate exit code +# returned to parent (for user notification). +# +######################################################################## +# As of May 2020 an alternative approach that works with both exceptions +# and debugging/profiling was implemented by re-purposing DWARF .cfi +# annotations even for Win64 unwind tables' generation. Unfortunately, +# but not really unexpectedly, it imposes additional limitations on +# coding style. Probably the most significant limitation is that the +# frame pointer has to be at 16*n distance from the stack pointer at the +# exit from prologue. But first things first. There are two additional +# synthetic .cfi directives, .cfi_end_prologue and .cfi_epilogue, +# that need to be added to all functions marked with additional .type +# tag (see example below). There are "do's and don'ts" for prologue +# and epilogue. It shouldn't come as a surprise that in prologue one may +# not modify non-volatile registers, but one may not modify %r11 either. +# This is because it's used as a temporary frame pointer(*). There are +# two exceptions to this rule. 1) One can set up a non-volatile register +# or %r11 as a frame pointer, but it must be last instruction in the +# prologue. 2) One can use 'push %rbp' as first instruction immediately +# followed by 'mov %rsp,%rbp' to use %rbp as "legacy" frame pointer. +# Constraints for epilogue, or rather on its boundary, depend on whether +# the frame is fixed- or variable-length. In fixed-frame subroutine +# stack pointer has to be restored in the last instruction prior to the +# .cfi_epilogue directive. If it's a variable-frame subroutine, and a +# non-volatile register was used as a frame pointer, then the last +# instruction prior to the directive has to restore its original value. +# This means that final stack pointer adjustment would have to be +# pushed past the directive. Normally this would render the epilogue +# non-unwindable, so special care has to be taken. To resolve the +# dilemma, copy the frame pointer to a volatile register in advance. +# To give an example: +# +# .type rbp_as_frame_pointer,\@function,3,"unwind" # mind extra tag! +# rbp_as_frame_pointer: +# .cfi_startproc +# push %rbp +# .cfi_push %rbp +# push %rbx +# .cfi_push %rbx +# mov %rsp,%rbp # last instruction in prologue +# .cfi_def_cfa_register %rbp # %rsp-%rbp has to be 16*n, e.g. 16*0 +# .cfi_end_prologue +# sub \$40,%rsp +# and \$-64,%rsp +# ... +# mov %rbp,%r11 +# .cfi_def_cfa_register %r11 # copy frame pointer to volatile %r11 +# mov 0(%rbp),%rbx +# mov 8(%rbp),%rbp # last instruction prior epilogue +# .cfi_epilogue # may not change %r11 in epilogue +# lea 16(%r11),%rsp +# ret +# .cfi_endproc +# .size rbp_as_frame_pointer,.-rbp_as_frame_pointer +# +# An example of "legacy" frame pointer: +# +# .type legacy_frame_pointer,\@function,3,"unwind" # mind extra tag! +# legacy_frame_pointer: +# .cfi_startproc +# push %rbp +# .cfi_push %rbp +# mov %rsp,%rbp +# .cfi_def_cfa_register %rbp +# push %rbx +# .cfi_push %rbx +# sub \$40,%rsp +# .cfi_alloca 40 +# .cfi_end_prologue # %rsp-%rbp has to be 16*n +# and \$-64,%rsp +# ... +# mov -8(%rbp),%rbx +# mov %rbp,%rsp +# .cfi_def_cfa_register %rsp +# pop %rbp # recognized by Windows +# .cfi_pop %rbp +# .cfi_epilogue +# ret +# .cfi_endproc +# .size legacy_frame_pointer,.-legacy_frame_pointer +# +# To give an example of fixed-frame subroutine for reference: +# +# .type fixed_frame,\@function,3,"unwind" # mind extra tag! +# fixed_frame: +# .cfi_startproc +# push %rbp +# .cfi_push %rbp +# push %rbx +# .cfi_push %rbx +# sub \$40,%rsp +# .cfi_adjust_cfa_offset 40 +# .cfi_end_prologue +# ... +# mov 40(%rsp),%rbx +# mov 48(%rsp),%rbp +# lea 56(%rsp),%rsp +# .cfi_adjust_cfa_offset -56 +# .cfi_epilogue +# ret +# .cfi_endproc +# .size fixed_frame,.-fixed_frame +# +# As for epilogue itself, one can only work on non-volatile registers. +# "Non-volatile" in "Windows" sense, i.e. minus %rdi and %rsi. +# +# On a final note, mixing old-style and modernized subroutines in the +# same file takes some trickery. Ones of the new kind have to appear +# after old-style ones. This has everything to do with the fact that +# entries in the .pdata segment have to appear in strictly same order +# as corresponding subroutines, and auto-generated RUNTIME_FUNCTION +# structures get mechanically appended to whatever existing .pdata. +# +# (*) Just in case, why %r11 and not %rax. This has everything to do +# with the way UNWIND_INFO is, one just can't designate %rax as +# frame pointer. diff --git a/cbits/chacha_avx2.c b/cbits/chacha_avx2.c new file mode 100644 --- /dev/null +++ b/cbits/chacha_avx2.c @@ -0,0 +1,146 @@ +/* + * ChaCha with AVX2, eight blocks at a time. + * + * The same arrangement as the SSE and NEON versions, twice as wide: word i + * of eight blocks goes in lane i of one 256-bit register. Eight blocks is + * where the register file stops being the constraint -- sixteen registers + * hold the working state either way, so the wider ones are free. + * + * AVX2 is not part of any baseline, so this is reached only after + * crypton_x86_simd_features() has said the CPU has it and the OS saves the + * wider registers. It is compiled into a translation unit that is + * otherwise baseline, through a function attribute, so nothing here can be + * emitted anywhere else. + */ + +#include +#include +#include "crypton_chacha.h" + +#ifdef WITH_TARGET_ATTRIBUTES + +#define TARGET __attribute__((target("avx2"))) + +/* rotating a 32-bit lane by sixteen or eight is a byte shuffle, which AVX2 + * does within each 128-bit half -- which is all this needs */ +static const int8_t rot16_tbl[32] = { + 2,3,0,1, 6,7,4,5, 10,11,8,9, 14,15,12,13, + 2,3,0,1, 6,7,4,5, 10,11,8,9, 14,15,12,13, +}; +static const int8_t rot8_tbl[32] = { + 3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14, + 3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14, +}; + +#define ROL(x, n) \ + ((n) == 16 ? _mm256_shuffle_epi8((x), _mm256_loadu_si256((const __m256i *) rot16_tbl)) \ + : (n) == 8 ? _mm256_shuffle_epi8((x), _mm256_loadu_si256((const __m256i *) rot8_tbl)) \ + : _mm256_or_si256(_mm256_slli_epi32((x), (n)), _mm256_srli_epi32((x), 32 - (n)))) + +TARGET +static inline void core8(int rounds, const crypton_chacha_state *in, + const uint8_t *src, uint8_t *dst, int combine) +{ + __m256i v0, v1, v2, v3, v4, v5, v6, v7; + __m256i v8, v9, v10, v11, v12, v13, v14, v15; + const uint32_t c = in->d[12]; + const __m256i ctr = _mm256_setr_epi32((int) c, (int) (c + 1), (int) (c + 2), + (int) (c + 3), (int) (c + 4), (int) (c + 5), + (int) (c + 6), (int) (c + 7)); + int i; + +#define SET(n) v##n = _mm256_set1_epi32((int) in->d[n]) + SET(0); SET(1); SET(2); SET(3); + SET(4); SET(5); SET(6); SET(7); + SET(8); SET(9); SET(10); SET(11); + SET(13); SET(14); SET(15); +#undef SET + v12 = ctr; + +#define QR(a, b, cc, d) \ + a = _mm256_add_epi32(a, b); d = ROL(_mm256_xor_si256(d, a), 16); \ + cc = _mm256_add_epi32(cc, d); b = ROL(_mm256_xor_si256(b, cc), 12); \ + a = _mm256_add_epi32(a, b); d = ROL(_mm256_xor_si256(d, a), 8); \ + cc = _mm256_add_epi32(cc, d); b = ROL(_mm256_xor_si256(b, cc), 7) + + for (i = rounds; i > 0; i -= 2) { + QR(v0, v4, v8, v12); + QR(v1, v5, v9, v13); + QR(v2, v6, v10, v14); + QR(v3, v7, v11, v15); + + QR(v0, v5, v10, v15); + QR(v1, v6, v11, v12); + QR(v2, v7, v8, v13); + QR(v3, v4, v9, v14); + } +#undef QR + +#define ADD(n) v##n = _mm256_add_epi32(v##n, _mm256_set1_epi32((int) in->d[n])) + ADD(0); ADD(1); ADD(2); ADD(3); + ADD(4); ADD(5); ADD(6); ADD(7); + ADD(8); ADD(9); ADD(10); ADD(11); + ADD(13); ADD(14); ADD(15); +#undef ADD + v12 = _mm256_add_epi32(v12, ctr); + + /* + * The interleave works within each 128-bit half, so four registers + * holding word w of blocks 0..7 come apart into words w..w+3 of + * blocks 0..3 in the low halves and of blocks 4..7 in the high ones. + * + * Each piece is exclusive-ored with the input and stored where it + * belongs as it comes out. Writing the keystream to a buffer and + * reading it back to combine it cost a pass over every byte, which is + * a tenth of what this loop does. + */ +#define OUT(j, g, v) \ + do { \ + __m128i o_ = (v); \ + if (combine) \ + o_ = _mm_xor_si128(o_, _mm_loadu_si128( \ + (const __m128i *) (src + 64 * (j) + 4 * (g)))); \ + _mm_storeu_si128((__m128i *) (dst + 64 * (j) + 4 * (g)), o_);\ + } while (0) + +#define GROUP(g, qa, qb, qc, qd) \ + do { \ + __m256i t0_ = _mm256_unpacklo_epi32(qa, qb); \ + __m256i t1_ = _mm256_unpackhi_epi32(qa, qb); \ + __m256i t2_ = _mm256_unpacklo_epi32(qc, qd); \ + __m256i t3_ = _mm256_unpackhi_epi32(qc, qd); \ + __m256i u0_ = _mm256_unpacklo_epi64(t0_, t2_); \ + __m256i u1_ = _mm256_unpackhi_epi64(t0_, t2_); \ + __m256i u2_ = _mm256_unpacklo_epi64(t1_, t3_); \ + __m256i u3_ = _mm256_unpackhi_epi64(t1_, t3_); \ + OUT(0, (g), _mm256_castsi256_si128(u0_)); \ + OUT(1, (g), _mm256_castsi256_si128(u1_)); \ + OUT(2, (g), _mm256_castsi256_si128(u2_)); \ + OUT(3, (g), _mm256_castsi256_si128(u3_)); \ + OUT(4, (g), _mm256_extracti128_si256(u0_, 1)); \ + OUT(5, (g), _mm256_extracti128_si256(u1_, 1)); \ + OUT(6, (g), _mm256_extracti128_si256(u2_, 1)); \ + OUT(7, (g), _mm256_extracti128_si256(u3_, 1)); \ + } while (0) + GROUP(0, v0, v1, v2, v3); + GROUP(4, v4, v5, v6, v7); + GROUP(8, v8, v9, v10, v11); + GROUP(12, v12, v13, v14, v15); +#undef GROUP +#undef OUT +} + +TARGET +void crypton_chacha_avx2_combine(int rounds, uint8_t *dst, const uint8_t *src, + const crypton_chacha_state *in) +{ + core8(rounds, in, src, dst, 1); +} + +TARGET +void crypton_chacha_avx2_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in) +{ + core8(rounds, in, NULL, dst, 0); +} + +#endif /* WITH_TARGET_ATTRIBUTES */ diff --git a/cbits/chacha_neon.c b/cbits/chacha_neon.c new file mode 100644 --- /dev/null +++ b/cbits/chacha_neon.c @@ -0,0 +1,145 @@ +/* + * ChaCha with NEON, four blocks at a time. + * + * The state is sixteen 32-bit words and the quarter rounds touch four of + * them at once, so a single block vectorises only by shuffling lanes + * between the column and diagonal rounds. Four blocks vectorise without + * any shuffling at all: word i of the four blocks goes in lane i of one + * register, every quarter round is then the same operation on whole + * registers, and the blocks are independent because only the counter + * differs between them. + * + * NEON is part of the AArch64 baseline, so unlike the AES, PMULL and SHA + * work there is nothing to ask about at runtime and no target attribute + * to attach. + */ + +#include +#include +#include +#include "crypton_chacha.h" + +/* rotate each 32-bit lane left by n */ +#define ROL(x, n) vsriq_n_u32(vshlq_n_u32((x), (n)), (x), 32 - (n)) +/* by 16 it is a halfword swap, and by 8 a byte shuffle; both beat the pair + * of shifts */ +#define ROL16(x) vreinterpretq_u32_u16(vrev32q_u16(vreinterpretq_u16_u32(x))) +#define ROL8(x) vreinterpretq_u32_u8(vqtbl1q_u8(vreinterpretq_u8_u32(x), rot8)) + +#define QR(a, b, c, d) \ + a = vaddq_u32(a, b); d = ROL16(veorq_u32(d, a)); \ + c = vaddq_u32(c, d); b = ROL(veorq_u32(b, c), 12); \ + a = vaddq_u32(a, b); d = ROL8(veorq_u32(d, a)); \ + c = vaddq_u32(c, d); b = ROL(veorq_u32(b, c), 7) + +/* + * Turn four registers holding word w of blocks 0..3 into four holding + * words w..w+3 of one block each, which is the order they are written in. + */ +#define TRANSPOSE(a, b, c, d) \ + do { \ + uint32x4x2_t t0_ = vtrnq_u32((a), (b)); \ + uint32x4x2_t t1_ = vtrnq_u32((c), (d)); \ + (a) = vcombine_u32(vget_low_u32(t0_.val[0]), \ + vget_low_u32(t1_.val[0])); \ + (b) = vcombine_u32(vget_low_u32(t0_.val[1]), \ + vget_low_u32(t1_.val[1])); \ + (c) = vcombine_u32(vget_high_u32(t0_.val[0]), \ + vget_high_u32(t1_.val[0])); \ + (d) = vcombine_u32(vget_high_u32(t0_.val[1]), \ + vget_high_u32(t1_.val[1])); \ + } while (0) + +/* + * Four blocks with counters d[12], d[12]+1, d[12]+2 and d[12]+3. The + * caller keeps the state's counter, and only calls this when those four + * do not carry into d[13]. + */ +static inline void core4(int rounds, const crypton_chacha_state *in, + const uint8_t *src, uint8_t *dst, int combine) +{ + static const uint8_t rot8_tbl[16] = + { 3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14 }; + const uint8x16_t rot8 = vld1q_u8(rot8_tbl); + uint32x4_t v0, v1, v2, v3, v4, v5, v6, v7; + uint32x4_t v8, v9, v10, v11, v12, v13, v14, v15; + const uint32_t c = in->d[12]; + const uint32_t ctr4[4] = { c, c + 1, c + 2, c + 3 }; + int i; + + /* + * Only the working state is kept in registers. The initial state has + * to be added back at the end, but holding a second copy of it would + * want thirty-two registers for that alone, and the machine has + * thirty-two in total; read it again instead, from memory that is + * certainly warm. + */ +#define SET(n) v##n = vdupq_n_u32(in->d[n]) + SET(0); SET(1); SET(2); SET(3); + SET(4); SET(5); SET(6); SET(7); + SET(8); SET(9); SET(10); SET(11); + SET(13); SET(14); SET(15); +#undef SET + v12 = vld1q_u32(ctr4); + + for (i = rounds; i > 0; i -= 2) { + QR(v0, v4, v8, v12); + QR(v1, v5, v9, v13); + QR(v2, v6, v10, v14); + QR(v3, v7, v11, v15); + + QR(v0, v5, v10, v15); + QR(v1, v6, v11, v12); + QR(v2, v7, v8, v13); + QR(v3, v4, v9, v14); + } + +#define ADD(n) v##n = vaddq_u32(v##n, vdupq_n_u32(in->d[n])) + ADD(0); ADD(1); ADD(2); ADD(3); + ADD(4); ADD(5); ADD(6); ADD(7); + ADD(8); ADD(9); ADD(10); ADD(11); + ADD(13); ADD(14); ADD(15); +#undef ADD + v12 = vaddq_u32(v12, vld1q_u32(ctr4)); + + TRANSPOSE(v0, v1, v2, v3); + TRANSPOSE(v4, v5, v6, v7); + TRANSPOSE(v8, v9, v10, v11); + TRANSPOSE(v12, v13, v14, v15); + + /* + * Each piece is exclusive-ored with the input and stored where it + * belongs as it comes out. Writing the keystream to a buffer and + * reading it back to combine it cost a pass over every byte. + */ +#define ST(j, g, v) \ + do { \ + uint8x16_t o_ = vreinterpretq_u8_u32(v); \ + if (combine) \ + o_ = veorq_u8(o_, vld1q_u8(src + 64 * (j) \ + + 4 * (g))); \ + vst1q_u8(dst + 64 * (j) + 4 * (g), o_); \ + } while (0) + ST(0, 0, v0); ST(1, 0, v1); ST(2, 0, v2); ST(3, 0, v3); + ST(0, 4, v4); ST(1, 4, v5); ST(2, 4, v6); ST(3, 4, v7); + ST(0, 8, v8); ST(1, 8, v9); ST(2, 8, v10); ST(3, 8, v11); + ST(0, 12, v12); ST(1, 12, v13); ST(2, 12, v14); ST(3, 12, v15); +#undef ST +} + +void crypton_chacha_simd_combine(int rounds, uint8_t *dst, const uint8_t *src, + const crypton_chacha_state *in) +{ + core4(rounds, in, src, dst, 1); +} + +void crypton_chacha_simd_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in) +{ + core4(rounds, in, NULL, dst, 0); +} + +/* NEON has no wider sibling to choose between, so the answer is fixed. */ +int crypton_chacha_simd_width(void) +{ + return 4; +} diff --git a/cbits/chacha_sse2.c b/cbits/chacha_sse2.c new file mode 100644 --- /dev/null +++ b/cbits/chacha_sse2.c @@ -0,0 +1,105 @@ +/* + * ChaCha with SSE, four blocks at a time, and the choice of which x86 + * version to run. + * + * Word i of four blocks goes in lane i of one register, so every quarter + * round is one operation on whole registers and no lane moves between the + * column and the diagonal rounds. Only the counter differs between the + * four blocks. + * + * SSE2 is part of the x86-64 baseline and needs no check. SSSE3 takes the + * rotates by sixteen and eight in one instruction each, and AVX2 -- in + * chacha_avx2.c -- carries eight blocks instead of four; both are reached + * only after crypton_x86_simd_features() says so. Both also need function + * attributes to sit in a translation unit that is otherwise baseline, so + * with use_target_attributes turned off only the SSE2 version is built. + */ + +#include +#include +#ifdef WITH_TARGET_ATTRIBUTES +#include +#endif +#include "crypton_chacha.h" +#include "crypton_cpu.h" + +#define SIZED(n) n##_sse2 +#define TARGET +#define ROL(x, n) _mm_or_si128(_mm_slli_epi32((x), (n)), _mm_srli_epi32((x), 32 - (n))) +#include +#undef SIZED +#undef TARGET +#undef ROL + +#ifdef WITH_TARGET_ATTRIBUTES + +static const int8_t rot16_tbl[16] = { 2,3,0,1, 6,7,4,5, 10,11,8,9, 14,15,12,13 }; +static const int8_t rot8_tbl[16] = { 3,0,1,2, 7,4,5,6, 11,8,9,10, 15,12,13,14 }; + +#define SIZED(n) n##_ssse3 +#define TARGET __attribute__((target("ssse3"))) +#define ROL(x, n) \ + ((n) == 16 ? _mm_shuffle_epi8((x), _mm_loadu_si128((const __m128i *) rot16_tbl)) \ + : (n) == 8 ? _mm_shuffle_epi8((x), _mm_loadu_si128((const __m128i *) rot8_tbl)) \ + : _mm_or_si128(_mm_slli_epi32((x), (n)), _mm_srli_epi32((x), 32 - (n)))) +#include +#undef SIZED +#undef TARGET +#undef ROL + +void crypton_chacha_avx2_combine(int rounds, uint8_t *dst, const uint8_t *src, + const crypton_chacha_state *in); +void crypton_chacha_avx2_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in); + +#endif + +/* how many blocks a call covers, and which version does it */ +enum { IMPL_UNRESOLVED = 0, IMPL_SSE2, IMPL_SSSE3, IMPL_AVX2 }; + +static int impl = IMPL_UNRESOLVED; + +/* Two threads racing to answer this both write the same value. */ +static int resolve(void) +{ +#ifdef WITH_TARGET_ATTRIBUTES + uint32_t f = crypton_x86_simd_features(); + + if (f & CRYPTON_X86_AVX2) + impl = IMPL_AVX2; + else if (f & CRYPTON_X86_SSSE3) + impl = IMPL_SSSE3; + else +#endif + impl = IMPL_SSE2; + return impl; +} + +int crypton_chacha_simd_width(void) +{ + int i = impl ? impl : resolve(); + + return i == IMPL_AVX2 ? 8 : 4; +} + +void crypton_chacha_simd_combine(int rounds, uint8_t *dst, const uint8_t *src, + const crypton_chacha_state *in) +{ + switch (impl ? impl : resolve()) { +#ifdef WITH_TARGET_ATTRIBUTES + case IMPL_AVX2: crypton_chacha_avx2_combine(rounds, dst, src, in); return; + case IMPL_SSSE3: combine_ssse3(rounds, dst, src, in); return; +#endif + default: combine_sse2(rounds, dst, src, in); return; + } +} + +void crypton_chacha_simd_generate(int rounds, uint8_t *dst, const crypton_chacha_state *in) +{ + switch (impl ? impl : resolve()) { +#ifdef WITH_TARGET_ATTRIBUTES + case IMPL_AVX2: crypton_chacha_avx2_generate(rounds, dst, in); return; + case IMPL_SSSE3: generate_ssse3(rounds, dst, in); return; +#endif + default: generate_sse2(rounds, dst, in); return; + } +} diff --git a/cbits/chacha_sse_impl.c b/cbits/chacha_sse_impl.c new file mode 100644 --- /dev/null +++ b/cbits/chacha_sse_impl.c @@ -0,0 +1,114 @@ +/* + * Included from chacha_sse2.c once per instruction set, with SIZED() + * naming the functions, ROL() rotating a lane and TARGET saying what the + * functions may use. The body is identical; only the two rotates by + * sixteen and eight differ, and only because SSSE3 can do each in one + * PSHUFB where SSE2 needs a shift, a shift and an or. + */ + +/* + * Four blocks with counters d[12] .. d[12]+3. The caller keeps the + * state's counter and only calls this when those four do not carry into + * d[13]. + */ +TARGET +static inline void SIZED(core4)(int rounds, const crypton_chacha_state *in, + const uint8_t *src, uint8_t *dst, int combine) +{ + __m128i v0, v1, v2, v3, v4, v5, v6, v7; + __m128i v8, v9, v10, v11, v12, v13, v14, v15; + const uint32_t c = in->d[12]; + int i; + + /* + * Sixteen registers hold the working state and the machine has + * sixteen, so the initial state is read again at the end rather than + * kept in a second set. + */ +#define SET(n) v##n = _mm_set1_epi32((int) in->d[n]) + SET(0); SET(1); SET(2); SET(3); + SET(4); SET(5); SET(6); SET(7); + SET(8); SET(9); SET(10); SET(11); + SET(13); SET(14); SET(15); +#undef SET + v12 = _mm_setr_epi32((int) c, (int) (c + 1), (int) (c + 2), (int) (c + 3)); + +#define QR(a, b, cc, d) \ + a = _mm_add_epi32(a, b); d = ROL(_mm_xor_si128(d, a), 16); \ + cc = _mm_add_epi32(cc, d); b = ROL(_mm_xor_si128(b, cc), 12); \ + a = _mm_add_epi32(a, b); d = ROL(_mm_xor_si128(d, a), 8); \ + cc = _mm_add_epi32(cc, d); b = ROL(_mm_xor_si128(b, cc), 7) + + for (i = rounds; i > 0; i -= 2) { + QR(v0, v4, v8, v12); + QR(v1, v5, v9, v13); + QR(v2, v6, v10, v14); + QR(v3, v7, v11, v15); + + QR(v0, v5, v10, v15); + QR(v1, v6, v11, v12); + QR(v2, v7, v8, v13); + QR(v3, v4, v9, v14); + } +#undef QR + +#define ADD(n) v##n = _mm_add_epi32(v##n, _mm_set1_epi32((int) in->d[n])) + ADD(0); ADD(1); ADD(2); ADD(3); + ADD(4); ADD(5); ADD(6); ADD(7); + ADD(8); ADD(9); ADD(10); ADD(11); + ADD(13); ADD(14); ADD(15); +#undef ADD + v12 = _mm_add_epi32(v12, _mm_setr_epi32((int) c, (int) (c + 1), + (int) (c + 2), (int) (c + 3))); + + /* four registers holding word w of blocks 0..3 become four holding + * words w..w+3 of one block each, the order they are written in */ +#define TRANSPOSE(qa, qb, qc, qd) \ + do { \ + __m128i t0_ = _mm_unpacklo_epi32(qa, qb); \ + __m128i t1_ = _mm_unpackhi_epi32(qa, qb); \ + __m128i t2_ = _mm_unpacklo_epi32(qc, qd); \ + __m128i t3_ = _mm_unpackhi_epi32(qc, qd); \ + qa = _mm_unpacklo_epi64(t0_, t2_); \ + qb = _mm_unpackhi_epi64(t0_, t2_); \ + qc = _mm_unpacklo_epi64(t1_, t3_); \ + qd = _mm_unpackhi_epi64(t1_, t3_); \ + } while (0) + TRANSPOSE(v0, v1, v2, v3); + TRANSPOSE(v4, v5, v6, v7); + TRANSPOSE(v8, v9, v10, v11); + TRANSPOSE(v12, v13, v14, v15); +#undef TRANSPOSE + + /* + * Each piece is exclusive-ored with the input and stored where it + * belongs as it comes out. Writing the keystream to a buffer and + * reading it back to combine it cost a pass over every byte. + */ +#define ST(j, g, v) \ + do { \ + __m128i o_ = (v); \ + if (combine) \ + o_ = _mm_xor_si128(o_, _mm_loadu_si128( \ + (const __m128i *) (src + 64 * (j) + 4 * (g)))); \ + _mm_storeu_si128((__m128i *) (dst + 64 * (j) + 4 * (g)), o_); \ + } while (0) + ST(0, 0, v0); ST(1, 0, v1); ST(2, 0, v2); ST(3, 0, v3); + ST(0, 4, v4); ST(1, 4, v5); ST(2, 4, v6); ST(3, 4, v7); + ST(0, 8, v8); ST(1, 8, v9); ST(2, 8, v10); ST(3, 8, v11); + ST(0, 12, v12); ST(1, 12, v13); ST(2, 12, v14); ST(3, 12, v15); +#undef ST +} + +TARGET +static void SIZED(combine)(int rounds, uint8_t *dst, const uint8_t *src, + const crypton_chacha_state *in) +{ + SIZED(core4)(rounds, in, src, dst, 1); +} + +TARGET +static void SIZED(generate)(int rounds, uint8_t *dst, const crypton_chacha_state *in) +{ + SIZED(core4)(rounds, in, NULL, dst, 0); +} diff --git a/cbits/crypton_aes.c b/cbits/crypton_aes.c --- a/cbits/crypton_aes.c +++ b/cbits/crypton_aes.c @@ -56,6 +56,30 @@ void crypton_aes_generic_ccm_encrypt(uint8_t *output, aes_ccm *ccm, aes_key *key, uint8_t *input, uint32_t length); void crypton_aes_generic_ccm_decrypt(uint8_t *output, aes_ccm *ccm, aes_key *key, uint8_t *input, uint32_t length); +#ifdef WITH_ARMV8_CRYPTO +void crypton_aes_armv8_init(aes_key *key, uint8_t *origkey, uint8_t size); +#define ARMV8_DECLS(sz) \ + void crypton_aes_armv8_encrypt_block##sz(aes_block *output, aes_key *key, aes_block *input); \ + void crypton_aes_armv8_decrypt_block##sz(aes_block *output, aes_key *key, aes_block *input); \ + void crypton_aes_armv8_encrypt_ecb##sz(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks); \ + void crypton_aes_armv8_decrypt_ecb##sz(aes_block *output, aes_key *key, aes_block *input, uint32_t nb_blocks); \ + void crypton_aes_armv8_encrypt_cbc##sz(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); \ + void crypton_aes_armv8_decrypt_cbc##sz(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); \ + void crypton_aes_armv8_encrypt_ctr##sz(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len); \ + void crypton_aes_armv8_gcm_encrypt##sz(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length); \ + void crypton_aes_armv8_gcm_decrypt##sz(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length); \ + void crypton_aes_armv8_encrypt_xts##sz(aes_block *output, aes_key *k1, aes_key *k2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks); \ + void crypton_aes_armv8_decrypt_xts##sz(aes_block *output, aes_key *k1, aes_key *k2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks); +ARMV8_DECLS(128) +ARMV8_DECLS(192) +ARMV8_DECLS(256) +int crypton_aes_armv8_available(void); +int crypton_aes_armv8_pmull_available(void); +void crypton_aes_armv8_hinit_pmull(block128 *htable, const block128 *h); +void crypton_aes_armv8_gf_mul_pmull(block128 *a, const block128 *htable); +void crypton_aes_armv8_gf_mul4_pmull(block128 *a, const block128 *blocks, const block128 *htable); +#endif + enum { /* init */ INIT_128, INIT_192, INIT_256, @@ -85,7 +109,7 @@ ENCRYPT_CCM_128, ENCRYPT_CCM_192, ENCRYPT_CCM_256, DECRYPT_CCM_128, DECRYPT_CCM_192, DECRYPT_CCM_256, /* ghash */ - GHASH_HINIT, GHASH_GF_MUL, + GHASH_HINIT, GHASH_GF_MUL, GHASH_GF_MUL4, }; void *crypton_aes_branch_table[] = { @@ -153,6 +177,7 @@ /* GHASH */ [GHASH_HINIT] = crypton_aes_generic_hinit, [GHASH_GF_MUL] = crypton_aes_generic_gf_mul, + [GHASH_GF_MUL4] = crypton_aes_generic_gf_mul4, }; typedef void (*init_f)(aes_key *, uint8_t *, uint8_t); @@ -166,8 +191,9 @@ typedef void (*block_f)(aes_block *output, aes_key *key, aes_block *input); typedef void (*hinit_f)(table_4bit htable, const block128 *h); typedef void (*gf_mul_f)(block128 *a, const table_4bit htable); +typedef void (*gf_mul4_f)(block128 *a, const block128 *blocks, const table_4bit htable); -#ifdef WITH_AESNI +#if defined(WITH_AESNI) || defined(WITH_ARMV8_CRYPTO) #define GET_INIT(strength) \ ((init_f) (crypton_aes_branch_table[INIT_128 + strength])) #define GET_ECB_ENCRYPT(strength) \ @@ -206,6 +232,8 @@ (((hinit_f) (crypton_aes_branch_table[GHASH_HINIT]))(t,h)) #define crypton_gf_mul(a,t) \ (((gf_mul_f) (crypton_aes_branch_table[GHASH_GF_MUL]))(a,t)) +#define crypton_gf_mul4(a,b,t) \ + (((gf_mul4_f) (crypton_aes_branch_table[GHASH_GF_MUL4]))(a,b,t)) #else #define GET_INIT(strenght) crypton_aes_generic_init #define GET_ECB_ENCRYPT(strength) crypton_aes_generic_encrypt_ecb @@ -226,6 +254,7 @@ #define crypton_aes_decrypt_block(o,k,i) crypton_aes_generic_decrypt_block(o,k,i) #define crypton_hinit(t,h) crypton_aes_generic_hinit(t,h) #define crypton_gf_mul(a,t) crypton_aes_generic_gf_mul(a,t) +#define crypton_gf_mul4(a,b,t) crypton_aes_generic_gf_mul4(a,b,t) #endif #define CPU_AESNI 0 @@ -242,39 +271,61 @@ crypton_aes_cpu_options[CPU_AESNI] = 1; crypton_aes_branch_table[INIT_128] = crypton_aesni_init; + crypton_aes_branch_table[INIT_192] = crypton_aesni_init; crypton_aes_branch_table[INIT_256] = crypton_aesni_init; crypton_aes_branch_table[ENCRYPT_BLOCK_128] = crypton_aesni_encrypt_block128; crypton_aes_branch_table[DECRYPT_BLOCK_128] = crypton_aesni_decrypt_block128; + crypton_aes_branch_table[ENCRYPT_BLOCK_192] = crypton_aesni_encrypt_block192; crypton_aes_branch_table[ENCRYPT_BLOCK_256] = crypton_aesni_encrypt_block256; + crypton_aes_branch_table[DECRYPT_BLOCK_192] = crypton_aesni_decrypt_block192; crypton_aes_branch_table[DECRYPT_BLOCK_256] = crypton_aesni_decrypt_block256; /* ECB */ crypton_aes_branch_table[ENCRYPT_ECB_128] = crypton_aesni_encrypt_ecb128; crypton_aes_branch_table[DECRYPT_ECB_128] = crypton_aesni_decrypt_ecb128; + crypton_aes_branch_table[ENCRYPT_ECB_192] = crypton_aesni_encrypt_ecb192; crypton_aes_branch_table[ENCRYPT_ECB_256] = crypton_aesni_encrypt_ecb256; + crypton_aes_branch_table[DECRYPT_ECB_192] = crypton_aesni_decrypt_ecb192; crypton_aes_branch_table[DECRYPT_ECB_256] = crypton_aesni_decrypt_ecb256; /* CBC */ crypton_aes_branch_table[ENCRYPT_CBC_128] = crypton_aesni_encrypt_cbc128; crypton_aes_branch_table[DECRYPT_CBC_128] = crypton_aesni_decrypt_cbc128; + crypton_aes_branch_table[ENCRYPT_CBC_192] = crypton_aesni_encrypt_cbc192; crypton_aes_branch_table[ENCRYPT_CBC_256] = crypton_aesni_encrypt_cbc256; + crypton_aes_branch_table[DECRYPT_CBC_192] = crypton_aesni_decrypt_cbc192; crypton_aes_branch_table[DECRYPT_CBC_256] = crypton_aesni_decrypt_cbc256; /* CTR */ crypton_aes_branch_table[ENCRYPT_CTR_128] = crypton_aesni_encrypt_ctr128; + crypton_aes_branch_table[ENCRYPT_CTR_192] = crypton_aesni_encrypt_ctr192; crypton_aes_branch_table[ENCRYPT_CTR_256] = crypton_aesni_encrypt_ctr256; /* CTR with 32-bit wrapping */ crypton_aes_branch_table[ENCRYPT_C32_128] = crypton_aesni_encrypt_c32_128; + crypton_aes_branch_table[ENCRYPT_C32_192] = crypton_aesni_encrypt_c32_192; crypton_aes_branch_table[ENCRYPT_C32_256] = crypton_aesni_encrypt_c32_256; /* XTS */ crypton_aes_branch_table[ENCRYPT_XTS_128] = crypton_aesni_encrypt_xts128; + crypton_aes_branch_table[ENCRYPT_XTS_192] = crypton_aesni_encrypt_xts192; crypton_aes_branch_table[ENCRYPT_XTS_256] = crypton_aesni_encrypt_xts256; + crypton_aes_branch_table[DECRYPT_XTS_128] = crypton_aesni_decrypt_xts128; + crypton_aes_branch_table[DECRYPT_XTS_192] = crypton_aesni_decrypt_xts192; + crypton_aes_branch_table[DECRYPT_XTS_256] = crypton_aesni_decrypt_xts256; /* GCM */ + /* GCM, where the build has the carry-less multiply, waits below until + * the processor is known to have it too: the loop calls the multiply + * rather than reaching it through the branch pointer, so that it can + * be scheduled against the rounds, and is compiled with the + * instruction. The AArch64 table waits for PMULL for the same reason. + */ +#ifndef WITH_PCLMUL crypton_aes_branch_table[ENCRYPT_GCM_128] = crypton_aesni_gcm_encrypt128; + crypton_aes_branch_table[ENCRYPT_GCM_192] = crypton_aesni_gcm_encrypt192; crypton_aes_branch_table[ENCRYPT_GCM_256] = crypton_aesni_gcm_encrypt256; - /* OCB */ - /* - crypton_aes_branch_table[ENCRYPT_OCB_128] = crypton_aesni_ocb_encrypt128; - crypton_aes_branch_table[ENCRYPT_OCB_256] = crypton_aesni_ocb_encrypt256; - */ + crypton_aes_branch_table[DECRYPT_GCM_128] = crypton_aesni_gcm_decrypt128; + crypton_aes_branch_table[DECRYPT_GCM_192] = crypton_aesni_gcm_decrypt192; + crypton_aes_branch_table[DECRYPT_GCM_256] = crypton_aesni_gcm_decrypt256; +#endif + /* OCB drives the ECB paths above a group at a time, so it has no + * entries of its own */ #ifdef WITH_PCLMUL if (!pclmul) return; @@ -283,16 +334,90 @@ /* GHASH */ crypton_aes_branch_table[GHASH_HINIT] = crypton_aesni_hinit_pclmul, crypton_aes_branch_table[GHASH_GF_MUL] = crypton_aesni_gf_mul_pclmul, + crypton_aes_branch_table[GHASH_GF_MUL4] = crypton_aesni_gf_mul4_pclmul, crypton_aesni_init_pclmul(); + + /* and GCM, which needs both halves */ + crypton_aes_branch_table[ENCRYPT_GCM_128] = crypton_aesni_gcm_encrypt128; + crypton_aes_branch_table[ENCRYPT_GCM_192] = crypton_aesni_gcm_encrypt192; + crypton_aes_branch_table[ENCRYPT_GCM_256] = crypton_aesni_gcm_encrypt256; + crypton_aes_branch_table[DECRYPT_GCM_128] = crypton_aesni_gcm_decrypt128; + crypton_aes_branch_table[DECRYPT_GCM_192] = crypton_aesni_gcm_decrypt192; + crypton_aes_branch_table[DECRYPT_GCM_256] = crypton_aesni_gcm_decrypt256; #endif } #endif +#ifdef WITH_ARMV8_CRYPTO +static void initialize_table_armv8(void) +{ + if (!crypton_aes_armv8_available()) + return; + crypton_aes_cpu_options[CPU_AESNI] = 1; + + crypton_aes_branch_table[INIT_128] = crypton_aes_armv8_init; + crypton_aes_branch_table[INIT_192] = crypton_aes_armv8_init; + crypton_aes_branch_table[INIT_256] = crypton_aes_armv8_init; + + crypton_aes_branch_table[ENCRYPT_BLOCK_128] = crypton_aes_armv8_encrypt_block128; + crypton_aes_branch_table[DECRYPT_BLOCK_128] = crypton_aes_armv8_decrypt_block128; + crypton_aes_branch_table[ENCRYPT_BLOCK_192] = crypton_aes_armv8_encrypt_block192; + crypton_aes_branch_table[ENCRYPT_BLOCK_256] = crypton_aes_armv8_encrypt_block256; + crypton_aes_branch_table[DECRYPT_BLOCK_192] = crypton_aes_armv8_decrypt_block192; + crypton_aes_branch_table[DECRYPT_BLOCK_256] = crypton_aes_armv8_decrypt_block256; + /* ECB */ + crypton_aes_branch_table[ENCRYPT_ECB_128] = crypton_aes_armv8_encrypt_ecb128; + crypton_aes_branch_table[DECRYPT_ECB_128] = crypton_aes_armv8_decrypt_ecb128; + crypton_aes_branch_table[ENCRYPT_ECB_192] = crypton_aes_armv8_encrypt_ecb192; + crypton_aes_branch_table[ENCRYPT_ECB_256] = crypton_aes_armv8_encrypt_ecb256; + crypton_aes_branch_table[DECRYPT_ECB_192] = crypton_aes_armv8_decrypt_ecb192; + crypton_aes_branch_table[DECRYPT_ECB_256] = crypton_aes_armv8_decrypt_ecb256; + /* CBC */ + crypton_aes_branch_table[ENCRYPT_CBC_128] = crypton_aes_armv8_encrypt_cbc128; + crypton_aes_branch_table[DECRYPT_CBC_128] = crypton_aes_armv8_decrypt_cbc128; + crypton_aes_branch_table[ENCRYPT_CBC_192] = crypton_aes_armv8_encrypt_cbc192; + crypton_aes_branch_table[ENCRYPT_CBC_256] = crypton_aes_armv8_encrypt_cbc256; + crypton_aes_branch_table[DECRYPT_CBC_192] = crypton_aes_armv8_decrypt_cbc192; + crypton_aes_branch_table[DECRYPT_CBC_256] = crypton_aes_armv8_decrypt_cbc256; + /* CTR, which the generic loop would otherwise drive one block at a time */ + crypton_aes_branch_table[ENCRYPT_CTR_128] = crypton_aes_armv8_encrypt_ctr128; + crypton_aes_branch_table[ENCRYPT_CTR_192] = crypton_aes_armv8_encrypt_ctr192; + crypton_aes_branch_table[ENCRYPT_CTR_256] = crypton_aes_armv8_encrypt_ctr256; + /* XTS, likewise, in both directions */ + crypton_aes_branch_table[ENCRYPT_XTS_128] = crypton_aes_armv8_encrypt_xts128; + crypton_aes_branch_table[DECRYPT_XTS_128] = crypton_aes_armv8_decrypt_xts128; + crypton_aes_branch_table[ENCRYPT_XTS_192] = crypton_aes_armv8_encrypt_xts192; + crypton_aes_branch_table[ENCRYPT_XTS_256] = crypton_aes_armv8_encrypt_xts256; + crypton_aes_branch_table[DECRYPT_XTS_192] = crypton_aes_armv8_decrypt_xts192; + crypton_aes_branch_table[DECRYPT_XTS_256] = crypton_aes_armv8_decrypt_xts256; + + /* GHASH, which GCM spends its time in once AES itself is fast */ + if (!crypton_aes_armv8_pmull_available()) + return; + crypton_aes_cpu_options[CPU_PCLMUL] = 1; + crypton_aes_branch_table[GHASH_HINIT] = crypton_aes_armv8_hinit_pmull; + crypton_aes_branch_table[GHASH_GF_MUL] = crypton_aes_armv8_gf_mul_pmull; + crypton_aes_branch_table[GHASH_GF_MUL4] = crypton_aes_armv8_gf_mul4_pmull; + + /* GCM, which needs both halves and so waits until PMULL is known to + * be there; the generic loop stands in otherwise */ + crypton_aes_branch_table[ENCRYPT_GCM_128] = crypton_aes_armv8_gcm_encrypt128; + crypton_aes_branch_table[DECRYPT_GCM_128] = crypton_aes_armv8_gcm_decrypt128; + crypton_aes_branch_table[ENCRYPT_GCM_192] = crypton_aes_armv8_gcm_encrypt192; + crypton_aes_branch_table[ENCRYPT_GCM_256] = crypton_aes_armv8_gcm_encrypt256; + crypton_aes_branch_table[DECRYPT_GCM_192] = crypton_aes_armv8_gcm_decrypt192; + crypton_aes_branch_table[DECRYPT_GCM_256] = crypton_aes_armv8_gcm_decrypt256; +} +#endif + uint8_t *crypton_aes_cpu_init(void) { #if defined(ARCH_X86) && defined(WITH_AESNI) crypton_aesni_initialize_hw(initialize_table_ni); #endif +#ifdef WITH_ARMV8_CRYPTO + initialize_table_armv8(); +#endif return crypton_aes_cpu_options; } @@ -332,33 +457,6 @@ d(output, key, iv, input, nb_blocks); } -void crypton_aes_gen_ctr(aes_block *output, aes_key *key, const aes_block *iv, uint32_t nb_blocks) -{ - aes_block block; - - /* preload IV in block */ - block128_copy(&block, iv); - - for ( ; nb_blocks-- > 0; output++, block128_inc_be(&block)) { - crypton_aes_encrypt_block(output, key, &block); - } -} - -void crypton_aes_gen_ctr_cont(aes_block *output, aes_key *key, aes_block *iv, uint32_t nb_blocks) -{ - aes_block block; - - /* preload IV in block */ - block128_copy(&block, iv); - - for ( ; nb_blocks-- > 0; output++, block128_inc_be(&block)) { - crypton_aes_encrypt_block(output, key, &block); - } - - /* copy back the IV */ - block128_copy(iv, &block); -} - void crypton_aes_encrypt_ctr(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len) { ctr_f e = GET_CTR_ENCRYPT(key->strength); @@ -381,7 +479,8 @@ void crypton_aes_decrypt_xts(aes_block *output, aes_key *k1, aes_key *k2, aes_block *dataunit, uint32_t spoint, aes_block *input, uint32_t nb_blocks) { - crypton_aes_generic_decrypt_xts(output, k1, k2, dataunit, spoint, input, nb_blocks); + xts_f d = GET_XTS_DECRYPT(k1->strength); + d(output, k1, k2, dataunit, spoint, input, nb_blocks); } void crypton_aes_gcm_encrypt(uint8_t *output, aes_gcm *gcm, aes_key *key, uint8_t *input, uint32_t length) @@ -426,6 +525,13 @@ crypton_gf_mul(&gcm->tag, gcm->htable); } +/* Same, for four consecutive blocks. Where the multiply is a carry-less + * instruction this costs one reduction instead of four. */ +static void gcm_ghash_add4(aes_gcm *gcm, const block128 *b) +{ + crypton_gf_mul4(&gcm->tag, b, gcm->htable); +} + void crypton_aes_gcm_init(aes_gcm *gcm, aes_key *key, uint8_t *iv, uint32_t len) { block128 h; @@ -465,6 +571,9 @@ void crypton_aes_gcm_aad(aes_gcm *gcm, uint8_t *input, uint32_t length) { gcm->length_aad += length; + for (; length >= 64; input += 64, length -= 64) { + gcm_ghash_add4(gcm, (const block128 *) input); + } for (; length >= 16; input += 16, length -= 16) { gcm_ghash_add(gcm, (block128 *) input); } @@ -655,6 +764,21 @@ #undef L_CACHED } +/* + * OCB's offsets are a running exclusive-or, so they have to be worked out in + * order, but the block cipher calls under them do not depend on each other: + * every block is offset, encrypted, and offset again. So the offsets are + * computed a group at a time and the group goes through ECB together, which + * is where the code written for the AES instructions interleaves eight blocks + * and covers the latency of AESENC. One block at a time left that idle and + * cost four times what GCM costs on the same machine, for a mode that does + * less work than GCM. + * + * Eight is what the ECB paths interleave; a group beyond that gains nothing + * and only makes the buffers larger. + */ +#define OCB_WAY 8 + void crypton_aes_ocb_init(aes_ocb *ocb, aes_key *key, uint8_t *iv, uint32_t len, uint32_t taglen) { block128 tmp, nonce, ktop; @@ -714,9 +838,23 @@ void crypton_aes_ocb_aad(aes_ocb *ocb, aes_key *key, uint8_t *input, uint32_t length) { block128 tmp; - unsigned int i; + block128 buf[OCB_WAY]; + uint32_t blocks = length / 16; + unsigned int i = 1, j; - for (i=1; i<= length/16; i++, input=input+16) { + for (; blocks >= OCB_WAY; blocks -= OCB_WAY, input += 16 * OCB_WAY) { + for (j = 0; j < OCB_WAY; j++, i++) { + ocb_get_L_i(&tmp, ocb->li, i); + block128_xor_aligned(&ocb->offset_aad, &tmp); + block128_vxor(&buf[j], &ocb->offset_aad, + (block128 *) (input + 16 * j)); + } + crypton_aes_encrypt_ecb(buf, key, buf, OCB_WAY); + for (j = 0; j < OCB_WAY; j++) + block128_xor_aligned(&ocb->sum_aad, &buf[j]); + } + + for (; blocks > 0; blocks--, i++, input += 16) { ocb_get_L_i(&tmp, ocb->li, i); block128_xor_aligned(&ocb->offset_aad, &tmp); @@ -882,6 +1020,20 @@ aes_block out; gcm->length_input += length; + /* four blocks at a time, so GHASH can fold them into one reduction */ + for (; length >= 64; input += 64, output += 64, length -= 64) { + aes_block buf[4]; + int i; + + for (i = 0; i < 4; i++) { + block128_inc32_be(&gcm->civ); + crypton_aes_encrypt_block(&buf[i], key, &gcm->civ); + block128_xor(&buf[i], (block128 *) (input + 16 * i)); + } + gcm_ghash_add4(gcm, buf); + for (i = 0; i < 4; i++) + block128_copy((block128 *) (output + 16 * i), &buf[i]); + } for (; length >= 16; input += 16, output += 16, length -= 16) { block128_inc32_be(&gcm->civ); @@ -915,6 +1067,19 @@ aes_block out; gcm->length_input += length; + /* GHASH all four ciphertext blocks before writing any plaintext, since + * output may be input */ + for (; length >= 64; input += 64, output += 64, length -= 64) { + int i; + + gcm_ghash_add4(gcm, (const block128 *) input); + for (i = 0; i < 4; i++) { + block128_inc32_be(&gcm->civ); + crypton_aes_encrypt_block(&out, key, &gcm->civ); + block128_xor(&out, (block128 *) (input + 16 * i)); + block128_copy((block128 *) (output + 16 * i), &out); + } + } for (; length >= 16; input += 16, output += 16, length -= 16) { block128_inc32_be(&gcm->civ); @@ -946,10 +1111,37 @@ uint8_t *input, uint32_t length, int encrypt) { block128 tmp, pad; - unsigned int i; + block128 offsets[OCB_WAY], buf[OCB_WAY]; + uint32_t blocks = length / 16; + unsigned int i = 1, j; - for (i = 1; i <= length/16; i++, input += 16, output += 16) { - /* Offset_i = Offset_{i-1} xor L_{ntz(i)} */ + for (; blocks >= OCB_WAY; + blocks -= OCB_WAY, input += 16 * OCB_WAY, output += 16 * OCB_WAY) { + for (j = 0; j < OCB_WAY; j++, i++) { + /* Offset_i = Offset_{i-1} xor L_{ntz(i)} */ + ocb_get_L_i(&tmp, ocb->li, i); + block128_xor_aligned(&ocb->offset_enc, &tmp); + block128_copy_aligned(&offsets[j], &ocb->offset_enc); + block128_vxor(&buf[j], &ocb->offset_enc, + (block128 *) (input + 16 * j)); + } + + if (encrypt) + crypton_aes_encrypt_ecb(buf, key, buf, OCB_WAY); + else + crypton_aes_decrypt_ecb(buf, key, buf, OCB_WAY); + + for (j = 0; j < OCB_WAY; j++) { + block128_vxor((block128 *) (output + 16 * j), + &offsets[j], &buf[j]); + block128_xor(&ocb->sum_enc, + (block128 *) ((encrypt ? input : output) + + 16 * j)); + } + } + + /* and what is left of the message, a block at a time */ + for (; blocks > 0; blocks--, i++, input += 16, output += 16) { ocb_get_L_i(&tmp, ocb->li, i); block128_xor_aligned(&ocb->offset_enc, &tmp); diff --git a/cbits/crypton_aes.h b/cbits/crypton_aes.h --- a/cbits/crypton_aes.h +++ b/cbits/crypton_aes.h @@ -95,8 +95,8 @@ void crypton_aes_encrypt_cbc(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); void crypton_aes_decrypt_cbc(aes_block *output, aes_key *key, aes_block *iv, aes_block *input, uint32_t nb_blocks); -void crypton_aes_gen_ctr(aes_block *output, aes_key *key, const aes_block *iv, uint32_t nb_blocks); -void crypton_aes_gen_ctr_cont(aes_block *output, aes_key *key, aes_block *iv, uint32_t nb_blocks); +void crypton_aes_encrypt_ctr(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len); +void crypton_aes_encrypt_c32(uint8_t *output, aes_key *key, aes_block *iv, uint8_t *input, uint32_t len); void crypton_aes_encrypt_xts(aes_block *output, aes_key *key, aes_key *key2, aes_block *sector, uint32_t spoint, aes_block *input, uint32_t nb_blocks); diff --git a/cbits/crypton_bignum.h b/cbits/crypton_bignum.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_bignum.h @@ -0,0 +1,268 @@ +/* + * Arithmetic on numbers held as arrays of limbs, least significant first. + * + * The modular multiplication is Montgomery's, and every choice it makes -- + * which of two numbers to keep after the final subtraction, which entry of a + * table to take -- is made with a mask rather than a branch, so that the + * values being worked on do not steer the work. The exponentiation in + * crypton_powm.c and the curve arithmetic in crypton_ecc.c are both built on + * this. + * + * Everything here is static inline: each file that includes it gets its own + * copy, which the compiler can specialise to the sizes it uses. + */ +#ifndef CRYPTON_BIGNUM_H +#define CRYPTON_BIGNUM_H + +#include +#include + +#if defined(__SIZEOF_INT128__) +typedef uint64_t limb_t; +typedef unsigned __int128 dlimb_t; +#define LIMB_BITS 64 +#else +typedef uint32_t limb_t; +typedef uint64_t dlimb_t; +#define LIMB_BITS 32 +#endif + +#define LIMB_BYTES (LIMB_BITS / 8) + +/* four bits of exponent per window, so a table of sixteen and no leftover + * bits: a byte holds exactly two windows */ +#define WINDOW_BITS 4 +#define TABLE_SIZE (1 << WINDOW_BITS) + +/* r = a - b, returning the borrow out of the top */ +static inline limb_t sub_n(limb_t *r, const limb_t *a, const limb_t *b, uint32_t n) +{ + limb_t borrow = 0; + uint32_t i; + + for (i = 0; i < n; i++) { + limb_t ai = a[i], bi = b[i]; + limb_t d = ai - bi - borrow; + /* borrow out, without branching */ + borrow = ((~ai & bi) | (~(ai ^ bi) & d)) >> (LIMB_BITS - 1); + r[i] = d; + } + return borrow; +} + +/* r = a + b, returning the carry out of the top */ +static inline limb_t add_n(limb_t *r, const limb_t *a, const limb_t *b, + uint32_t n) +{ + limb_t carry = 0; + uint32_t i; + + for (i = 0; i < n; i++) { + dlimb_t s = (dlimb_t) a[i] + b[i] + carry; + + r[i] = (limb_t) s; + carry = (limb_t) (s >> LIMB_BITS); + } + return carry; +} + +/* a = 2a, returning the bit shifted out of the top */ +static inline limb_t shl1(limb_t *a, uint32_t n) +{ + limb_t carry = 0; + uint32_t i; + + for (i = 0; i < n; i++) { + limb_t next = a[i] >> (LIMB_BITS - 1); + a[i] = (a[i] << 1) | carry; + carry = next; + } + return carry; +} + +/* r = take ? a : b */ +static inline void select_n(limb_t *r, const limb_t *a, const limb_t *b, limb_t take, + uint32_t n) +{ + limb_t mask = (limb_t) 0 - take; + uint32_t i; + + for (i = 0; i < n; i++) + r[i] = (a[i] & mask) | (b[i] & ~mask); +} + +/* all ones when a and b are equal, zero otherwise */ +static inline limb_t eq_mask(limb_t a, limb_t b) +{ + limb_t d = a ^ b; + limb_t nz = d | ((limb_t) 0 - d); /* top bit set unless d is zero */ + + return (limb_t) 0 - ((nz >> (LIMB_BITS - 1)) ^ 1); +} + +/* -m^-1 mod 2^LIMB_BITS, for odd m */ +static inline limb_t mont_n0(limb_t m0) +{ + limb_t inv = 1; + int i; + + /* Newton's iteration doubles the number of correct bits each time */ + for (i = 0; i < 6; i++) + inv *= (limb_t) 2 - m0 * inv; + return (limb_t) 0 - inv; +} + +/* t += a * b over n limbs, returning the carry. This is where nearly all of + * the time goes, so the limbs are taken eight at a time; what is left over at + * the end is taken one at a time. */ +#define ADDMUL_STEP(k) \ + p = (dlimb_t) a[i + (k)] * b + t[i + (k)] + carry; \ + t[i + (k)] = (limb_t) p; \ + carry = (limb_t) (p >> LIMB_BITS); + +static inline limb_t addmul_1(limb_t *t, const limb_t *a, uint32_t n, limb_t b) +{ + limb_t carry = 0; + uint32_t i = 0; + dlimb_t p; + + for (; i + 8 <= n; i += 8) { + ADDMUL_STEP(0) ADDMUL_STEP(1) ADDMUL_STEP(2) ADDMUL_STEP(3) + ADDMUL_STEP(4) ADDMUL_STEP(5) ADDMUL_STEP(6) ADDMUL_STEP(7) + } + for (; i + 4 <= n; i += 4) { + ADDMUL_STEP(0) ADDMUL_STEP(1) ADDMUL_STEP(2) ADDMUL_STEP(3) + } + for (; i + 2 <= n; i += 2) { + ADDMUL_STEP(0) ADDMUL_STEP(1) + } + for (; i < n; i++) { + ADDMUL_STEP(0) + } + return carry; +} + +/* r = t * R^-1 mod m, with t of 2n limbs and destroyed on the way */ +static inline void mont_reduce(limb_t *r, limb_t *t, const limb_t *m, limb_t n0, + uint32_t n) +{ + limb_t borrow, take, carry = 0; + uint32_t i; + + for (i = 0; i < n; i++) { + limb_t u = t[i] * n0; + limb_t c = addmul_1(t + i, m, n, u); + dlimb_t s = (dlimb_t) t[n + i] + c + carry; + + t[n + i] = (limb_t) s; + carry = (limb_t) (s >> LIMB_BITS); + } + + /* what is left is under 2m, so at most one subtraction; which of the two + * to keep is a mask */ + borrow = sub_n(r, t + n, m, n); + take = carry | (borrow ^ 1); + select_n(r, r, t + n, take & 1, n); +} + +/* r = a * b * R^-1 mod m, with t of 2n limbs */ +static inline void mont_mul(limb_t *r, const limb_t *a, const limb_t *b, + const limb_t *m, limb_t n0, uint32_t n, limb_t *t) +{ + uint32_t i; + + memset(t, 0, 2 * n * sizeof(limb_t)); + for (i = 0; i < n; i++) + t[n + i] = addmul_1(t + i, a, n, b[i]); + mont_reduce(r, t, m, n0, n); +} + +/* r = a * a * R^-1 mod m, with t of 2n limbs. A square is its own mirror + * image, so each product off the diagonal is worth two and only half of them + * are worked out: their sum is doubled, and then the diagonal is added in. */ +static inline void mont_sqr(limb_t *r, const limb_t *a, const limb_t *m, limb_t n0, + uint32_t n, limb_t *t) +{ + limb_t carry = 0; + uint32_t i; + + memset(t, 0, 2 * n * sizeof(limb_t)); + for (i = 0; i + 1 < n; i++) + t[n + i] = addmul_1(t + i + i + 1, a + i + 1, n - 1 - i, a[i]); + shl1(t, 2 * n); /* their sum is under half of what 2n limbs hold */ + for (i = 0; i < n; i++) { + dlimb_t p = (dlimb_t) a[i] * a[i] + t[i + i] + carry; + + t[i + i] = (limb_t) p; + p = (dlimb_t) t[i + i + 1] + (limb_t) (p >> LIMB_BITS); + t[i + i + 1] = (limb_t) p; + carry = (limb_t) (p >> LIMB_BITS); + } + mont_reduce(r, t, m, n0, n); +} + +/* r2 = R^2 mod m, by doubling + * + * Doubling starts at the highest power of two under the modulus rather than + * at one, since everything below that power is where doubling would go + * anyway: for a modulus that fills its limbs that is half the steps. + */ +static inline void mont_r2(limb_t *r2, const limb_t *m, uint32_t n, limb_t *tmp) +{ + uint32_t i, k = 0, steps; + + for (i = n; i > 0 && k == 0; i--) + if (m[i - 1] != 0) { + limb_t top = m[i - 1]; + + k = (i - 1) * LIMB_BITS; + while (top != 0) { + k++; + top >>= 1; + } + } + memset(r2, 0, n * sizeof(limb_t)); + if (k == 0) + return; /* a modulus of nothing, which the caller rules out */ + r2[(k - 1) / LIMB_BITS] = (limb_t) 1 << ((k - 1) % LIMB_BITS); + steps = 2 * n * LIMB_BITS - (k - 1); + for (i = 0; i < steps; i++) { + limb_t carry = shl1(r2, n); + limb_t borrow = sub_n(tmp, r2, m, n); + select_n(r2, tmp, r2, (carry | (borrow ^ 1)) & 1, n); + } +} + +/* big-endian bytes into limbs, least significant limb first; anything above + * n limbs has to be zero, which is what the contract on the base asks for */ +static inline int from_be(limb_t *r, uint32_t n, const uint8_t *src, uint32_t len) +{ + uint32_t i; + + memset(r, 0, n * sizeof(limb_t)); + for (i = 0; i < len; i++) { + uint8_t byte = src[len - 1 - i]; + + if (i / LIMB_BYTES >= n) { + if (byte != 0) + return 1; + continue; + } + r[i / LIMB_BYTES] |= (limb_t) byte << (8 * (i % LIMB_BYTES)); + } + return 0; +} + +static inline void to_be(uint8_t *dst, uint32_t len, const limb_t *a, uint32_t n) +{ + uint32_t i; + + for (i = 0; i < len; i++) { + uint32_t pos = len - 1 - i; + uint32_t li = i / LIMB_BYTES; + + dst[pos] = li < n ? (uint8_t) (a[li] >> (8 * (i % LIMB_BYTES))) : 0; + } +} + +#endif diff --git a/cbits/crypton_blowfish.c b/cbits/crypton_blowfish.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_blowfish.c @@ -0,0 +1,456 @@ +/* + * Blowfish, and the key setup bcrypt wraps around it. + * + * The cipher is the plain one: sixteen Feistel rounds over a schedule of + * eighteen P words and four S boxes of 256, all of which the key is stirred + * into. What makes bcrypt out of it is doing that stirring twice for every + * count the cost asks for, with the salt in the mix, so that the work is + * whatever the cost says and cannot be skipped. + * + * None of this is constant time, and it is not meant to be: what it is given + * is a password, and what it leaks by timing is how long the password is, + * which the format says out loud anyway. What matters here is that a round + * costs what it costs, since that is the whole point of the cost parameter. + */ +#include +#include +#include + +/* The P array and the four S boxes, which are the digits of pi. */ +static const uint32_t initial_p[18] = { + 0x243f6a88U, 0x85a308d3U, 0x13198a2eU, 0x03707344U, 0xa4093822U, 0x299f31d0U, + 0x082efa98U, 0xec4e6c89U, 0x452821e6U, 0x38d01377U, 0xbe5466cfU, 0x34e90c6cU, + 0xc0ac29b7U, 0xc97c50ddU, 0x3f84d5b5U, 0xb5470917U, 0x9216d5d9U, 0x8979fb1bU +}; + +static const uint32_t initial_s[4][256] = { + { + 0xd1310ba6U, 0x98dfb5acU, 0x2ffd72dbU, 0xd01adfb7U, 0xb8e1afedU, 0x6a267e96U, + 0xba7c9045U, 0xf12c7f99U, 0x24a19947U, 0xb3916cf7U, 0x0801f2e2U, 0x858efc16U, + 0x636920d8U, 0x71574e69U, 0xa458fea3U, 0xf4933d7eU, 0x0d95748fU, 0x728eb658U, + 0x718bcd58U, 0x82154aeeU, 0x7b54a41dU, 0xc25a59b5U, 0x9c30d539U, 0x2af26013U, + 0xc5d1b023U, 0x286085f0U, 0xca417918U, 0xb8db38efU, 0x8e79dcb0U, 0x603a180eU, + 0x6c9e0e8bU, 0xb01e8a3eU, 0xd71577c1U, 0xbd314b27U, 0x78af2fdaU, 0x55605c60U, + 0xe65525f3U, 0xaa55ab94U, 0x57489862U, 0x63e81440U, 0x55ca396aU, 0x2aab10b6U, + 0xb4cc5c34U, 0x1141e8ceU, 0xa15486afU, 0x7c72e993U, 0xb3ee1411U, 0x636fbc2aU, + 0x2ba9c55dU, 0x741831f6U, 0xce5c3e16U, 0x9b87931eU, 0xafd6ba33U, 0x6c24cf5cU, + 0x7a325381U, 0x28958677U, 0x3b8f4898U, 0x6b4bb9afU, 0xc4bfe81bU, 0x66282193U, + 0x61d809ccU, 0xfb21a991U, 0x487cac60U, 0x5dec8032U, 0xef845d5dU, 0xe98575b1U, + 0xdc262302U, 0xeb651b88U, 0x23893e81U, 0xd396acc5U, 0x0f6d6ff3U, 0x83f44239U, + 0x2e0b4482U, 0xa4842004U, 0x69c8f04aU, 0x9e1f9b5eU, 0x21c66842U, 0xf6e96c9aU, + 0x670c9c61U, 0xabd388f0U, 0x6a51a0d2U, 0xd8542f68U, 0x960fa728U, 0xab5133a3U, + 0x6eef0b6cU, 0x137a3be4U, 0xba3bf050U, 0x7efb2a98U, 0xa1f1651dU, 0x39af0176U, + 0x66ca593eU, 0x82430e88U, 0x8cee8619U, 0x456f9fb4U, 0x7d84a5c3U, 0x3b8b5ebeU, + 0xe06f75d8U, 0x85c12073U, 0x401a449fU, 0x56c16aa6U, 0x4ed3aa62U, 0x363f7706U, + 0x1bfedf72U, 0x429b023dU, 0x37d0d724U, 0xd00a1248U, 0xdb0fead3U, 0x49f1c09bU, + 0x075372c9U, 0x80991b7bU, 0x25d479d8U, 0xf6e8def7U, 0xe3fe501aU, 0xb6794c3bU, + 0x976ce0bdU, 0x04c006baU, 0xc1a94fb6U, 0x409f60c4U, 0x5e5c9ec2U, 0x196a2463U, + 0x68fb6fafU, 0x3e6c53b5U, 0x1339b2ebU, 0x3b52ec6fU, 0x6dfc511fU, 0x9b30952cU, + 0xcc814544U, 0xaf5ebd09U, 0xbee3d004U, 0xde334afdU, 0x660f2807U, 0x192e4bb3U, + 0xc0cba857U, 0x45c8740fU, 0xd20b5f39U, 0xb9d3fbdbU, 0x5579c0bdU, 0x1a60320aU, + 0xd6a100c6U, 0x402c7279U, 0x679f25feU, 0xfb1fa3ccU, 0x8ea5e9f8U, 0xdb3222f8U, + 0x3c7516dfU, 0xfd616b15U, 0x2f501ec8U, 0xad0552abU, 0x323db5faU, 0xfd238760U, + 0x53317b48U, 0x3e00df82U, 0x9e5c57bbU, 0xca6f8ca0U, 0x1a87562eU, 0xdf1769dbU, + 0xd542a8f6U, 0x287effc3U, 0xac6732c6U, 0x8c4f5573U, 0x695b27b0U, 0xbbca58c8U, + 0xe1ffa35dU, 0xb8f011a0U, 0x10fa3d98U, 0xfd2183b8U, 0x4afcb56cU, 0x2dd1d35bU, + 0x9a53e479U, 0xb6f84565U, 0xd28e49bcU, 0x4bfb9790U, 0xe1ddf2daU, 0xa4cb7e33U, + 0x62fb1341U, 0xcee4c6e8U, 0xef20cadaU, 0x36774c01U, 0xd07e9efeU, 0x2bf11fb4U, + 0x95dbda4dU, 0xae909198U, 0xeaad8e71U, 0x6b93d5a0U, 0xd08ed1d0U, 0xafc725e0U, + 0x8e3c5b2fU, 0x8e7594b7U, 0x8ff6e2fbU, 0xf2122b64U, 0x8888b812U, 0x900df01cU, + 0x4fad5ea0U, 0x688fc31cU, 0xd1cff191U, 0xb3a8c1adU, 0x2f2f2218U, 0xbe0e1777U, + 0xea752dfeU, 0x8b021fa1U, 0xe5a0cc0fU, 0xb56f74e8U, 0x18acf3d6U, 0xce89e299U, + 0xb4a84fe0U, 0xfd13e0b7U, 0x7cc43b81U, 0xd2ada8d9U, 0x165fa266U, 0x80957705U, + 0x93cc7314U, 0x211a1477U, 0xe6ad2065U, 0x77b5fa86U, 0xc75442f5U, 0xfb9d35cfU, + 0xebcdaf0cU, 0x7b3e89a0U, 0xd6411bd3U, 0xae1e7e49U, 0x00250e2dU, 0x2071b35eU, + 0x226800bbU, 0x57b8e0afU, 0x2464369bU, 0xf009b91eU, 0x5563911dU, 0x59dfa6aaU, + 0x78c14389U, 0xd95a537fU, 0x207d5ba2U, 0x02e5b9c5U, 0x83260376U, 0x6295cfa9U, + 0x11c81968U, 0x4e734a41U, 0xb3472dcaU, 0x7b14a94aU, 0x1b510052U, 0x9a532915U, + 0xd60f573fU, 0xbc9bc6e4U, 0x2b60a476U, 0x81e67400U, 0x08ba6fb5U, 0x571be91fU, + 0xf296ec6bU, 0x2a0dd915U, 0xb6636521U, 0xe7b9f9b6U, 0xff34052eU, 0xc5855664U, + 0x53b02d5dU, 0xa99f8fa1U, 0x08ba4799U, 0x6e85076aU + }, + { + 0x4b7a70e9U, 0xb5b32944U, 0xdb75092eU, 0xc4192623U, 0xad6ea6b0U, 0x49a7df7dU, + 0x9cee60b8U, 0x8fedb266U, 0xecaa8c71U, 0x699a17ffU, 0x5664526cU, 0xc2b19ee1U, + 0x193602a5U, 0x75094c29U, 0xa0591340U, 0xe4183a3eU, 0x3f54989aU, 0x5b429d65U, + 0x6b8fe4d6U, 0x99f73fd6U, 0xa1d29c07U, 0xefe830f5U, 0x4d2d38e6U, 0xf0255dc1U, + 0x4cdd2086U, 0x8470eb26U, 0x6382e9c6U, 0x021ecc5eU, 0x09686b3fU, 0x3ebaefc9U, + 0x3c971814U, 0x6b6a70a1U, 0x687f3584U, 0x52a0e286U, 0xb79c5305U, 0xaa500737U, + 0x3e07841cU, 0x7fdeae5cU, 0x8e7d44ecU, 0x5716f2b8U, 0xb03ada37U, 0xf0500c0dU, + 0xf01c1f04U, 0x0200b3ffU, 0xae0cf51aU, 0x3cb574b2U, 0x25837a58U, 0xdc0921bdU, + 0xd19113f9U, 0x7ca92ff6U, 0x94324773U, 0x22f54701U, 0x3ae5e581U, 0x37c2dadcU, + 0xc8b57634U, 0x9af3dda7U, 0xa9446146U, 0x0fd0030eU, 0xecc8c73eU, 0xa4751e41U, + 0xe238cd99U, 0x3bea0e2fU, 0x3280bba1U, 0x183eb331U, 0x4e548b38U, 0x4f6db908U, + 0x6f420d03U, 0xf60a04bfU, 0x2cb81290U, 0x24977c79U, 0x5679b072U, 0xbcaf89afU, + 0xde9a771fU, 0xd9930810U, 0xb38bae12U, 0xdccf3f2eU, 0x5512721fU, 0x2e6b7124U, + 0x501adde6U, 0x9f84cd87U, 0x7a584718U, 0x7408da17U, 0xbc9f9abcU, 0xe94b7d8cU, + 0xec7aec3aU, 0xdb851dfaU, 0x63094366U, 0xc464c3d2U, 0xef1c1847U, 0x3215d908U, + 0xdd433b37U, 0x24c2ba16U, 0x12a14d43U, 0x2a65c451U, 0x50940002U, 0x133ae4ddU, + 0x71dff89eU, 0x10314e55U, 0x81ac77d6U, 0x5f11199bU, 0x043556f1U, 0xd7a3c76bU, + 0x3c11183bU, 0x5924a509U, 0xf28fe6edU, 0x97f1fbfaU, 0x9ebabf2cU, 0x1e153c6eU, + 0x86e34570U, 0xeae96fb1U, 0x860e5e0aU, 0x5a3e2ab3U, 0x771fe71cU, 0x4e3d06faU, + 0x2965dcb9U, 0x99e71d0fU, 0x803e89d6U, 0x5266c825U, 0x2e4cc978U, 0x9c10b36aU, + 0xc6150ebaU, 0x94e2ea78U, 0xa5fc3c53U, 0x1e0a2df4U, 0xf2f74ea7U, 0x361d2b3dU, + 0x1939260fU, 0x19c27960U, 0x5223a708U, 0xf71312b6U, 0xebadfe6eU, 0xeac31f66U, + 0xe3bc4595U, 0xa67bc883U, 0xb17f37d1U, 0x018cff28U, 0xc332ddefU, 0xbe6c5aa5U, + 0x65582185U, 0x68ab9802U, 0xeecea50fU, 0xdb2f953bU, 0x2aef7dadU, 0x5b6e2f84U, + 0x1521b628U, 0x29076170U, 0xecdd4775U, 0x619f1510U, 0x13cca830U, 0xeb61bd96U, + 0x0334fe1eU, 0xaa0363cfU, 0xb5735c90U, 0x4c70a239U, 0xd59e9e0bU, 0xcbaade14U, + 0xeecc86bcU, 0x60622ca7U, 0x9cab5cabU, 0xb2f3846eU, 0x648b1eafU, 0x19bdf0caU, + 0xa02369b9U, 0x655abb50U, 0x40685a32U, 0x3c2ab4b3U, 0x319ee9d5U, 0xc021b8f7U, + 0x9b540b19U, 0x875fa099U, 0x95f7997eU, 0x623d7da8U, 0xf837889aU, 0x97e32d77U, + 0x11ed935fU, 0x16681281U, 0x0e358829U, 0xc7e61fd6U, 0x96dedfa1U, 0x7858ba99U, + 0x57f584a5U, 0x1b227263U, 0x9b83c3ffU, 0x1ac24696U, 0xcdb30aebU, 0x532e3054U, + 0x8fd948e4U, 0x6dbc3128U, 0x58ebf2efU, 0x34c6ffeaU, 0xfe28ed61U, 0xee7c3c73U, + 0x5d4a14d9U, 0xe864b7e3U, 0x42105d14U, 0x203e13e0U, 0x45eee2b6U, 0xa3aaabeaU, + 0xdb6c4f15U, 0xfacb4fd0U, 0xc742f442U, 0xef6abbb5U, 0x654f3b1dU, 0x41cd2105U, + 0xd81e799eU, 0x86854dc7U, 0xe44b476aU, 0x3d816250U, 0xcf62a1f2U, 0x5b8d2646U, + 0xfc8883a0U, 0xc1c7b6a3U, 0x7f1524c3U, 0x69cb7492U, 0x47848a0bU, 0x5692b285U, + 0x095bbf00U, 0xad19489dU, 0x1462b174U, 0x23820e00U, 0x58428d2aU, 0x0c55f5eaU, + 0x1dadf43eU, 0x233f7061U, 0x3372f092U, 0x8d937e41U, 0xd65fecf1U, 0x6c223bdbU, + 0x7cde3759U, 0xcbee7460U, 0x4085f2a7U, 0xce77326eU, 0xa6078084U, 0x19f8509eU, + 0xe8efd855U, 0x61d99735U, 0xa969a7aaU, 0xc50c06c2U, 0x5a04abfcU, 0x800bcadcU, + 0x9e447a2eU, 0xc3453484U, 0xfdd56705U, 0x0e1e9ec9U, 0xdb73dbd3U, 0x105588cdU, + 0x675fda79U, 0xe3674340U, 0xc5c43465U, 0x713e38d8U, 0x3d28f89eU, 0xf16dff20U, + 0x153e21e7U, 0x8fb03d4aU, 0xe6e39f2bU, 0xdb83adf7U + }, + { + 0xe93d5a68U, 0x948140f7U, 0xf64c261cU, 0x94692934U, 0x411520f7U, 0x7602d4f7U, + 0xbcf46b2eU, 0xd4a20068U, 0xd4082471U, 0x3320f46aU, 0x43b7d4b7U, 0x500061afU, + 0x1e39f62eU, 0x97244546U, 0x14214f74U, 0xbf8b8840U, 0x4d95fc1dU, 0x96b591afU, + 0x70f4ddd3U, 0x66a02f45U, 0xbfbc09ecU, 0x03bd9785U, 0x7fac6dd0U, 0x31cb8504U, + 0x96eb27b3U, 0x55fd3941U, 0xda2547e6U, 0xabca0a9aU, 0x28507825U, 0x530429f4U, + 0x0a2c86daU, 0xe9b66dfbU, 0x68dc1462U, 0xd7486900U, 0x680ec0a4U, 0x27a18deeU, + 0x4f3ffea2U, 0xe887ad8cU, 0xb58ce006U, 0x7af4d6b6U, 0xaace1e7cU, 0xd3375fecU, + 0xce78a399U, 0x406b2a42U, 0x20fe9e35U, 0xd9f385b9U, 0xee39d7abU, 0x3b124e8bU, + 0x1dc9faf7U, 0x4b6d1856U, 0x26a36631U, 0xeae397b2U, 0x3a6efa74U, 0xdd5b4332U, + 0x6841e7f7U, 0xca7820fbU, 0xfb0af54eU, 0xd8feb397U, 0x454056acU, 0xba489527U, + 0x55533a3aU, 0x20838d87U, 0xfe6ba9b7U, 0xd096954bU, 0x55a867bcU, 0xa1159a58U, + 0xcca92963U, 0x99e1db33U, 0xa62a4a56U, 0x3f3125f9U, 0x5ef47e1cU, 0x9029317cU, + 0xfdf8e802U, 0x04272f70U, 0x80bb155cU, 0x05282ce3U, 0x95c11548U, 0xe4c66d22U, + 0x48c1133fU, 0xc70f86dcU, 0x07f9c9eeU, 0x41041f0fU, 0x404779a4U, 0x5d886e17U, + 0x325f51ebU, 0xd59bc0d1U, 0xf2bcc18fU, 0x41113564U, 0x257b7834U, 0x602a9c60U, + 0xdff8e8a3U, 0x1f636c1bU, 0x0e12b4c2U, 0x02e1329eU, 0xaf664fd1U, 0xcad18115U, + 0x6b2395e0U, 0x333e92e1U, 0x3b240b62U, 0xeebeb922U, 0x85b2a20eU, 0xe6ba0d99U, + 0xde720c8cU, 0x2da2f728U, 0xd0127845U, 0x95b794fdU, 0x647d0862U, 0xe7ccf5f0U, + 0x5449a36fU, 0x877d48faU, 0xc39dfd27U, 0xf33e8d1eU, 0x0a476341U, 0x992eff74U, + 0x3a6f6eabU, 0xf4f8fd37U, 0xa812dc60U, 0xa1ebddf8U, 0x991be14cU, 0xdb6e6b0dU, + 0xc67b5510U, 0x6d672c37U, 0x2765d43bU, 0xdcd0e804U, 0xf1290dc7U, 0xcc00ffa3U, + 0xb5390f92U, 0x690fed0bU, 0x667b9ffbU, 0xcedb7d9cU, 0xa091cf0bU, 0xd9155ea3U, + 0xbb132f88U, 0x515bad24U, 0x7b9479bfU, 0x763bd6ebU, 0x37392eb3U, 0xcc115979U, + 0x8026e297U, 0xf42e312dU, 0x6842ada7U, 0xc66a2b3bU, 0x12754cccU, 0x782ef11cU, + 0x6a124237U, 0xb79251e7U, 0x06a1bbe6U, 0x4bfb6350U, 0x1a6b1018U, 0x11caedfaU, + 0x3d25bdd8U, 0xe2e1c3c9U, 0x44421659U, 0x0a121386U, 0xd90cec6eU, 0xd5abea2aU, + 0x64af674eU, 0xda86a85fU, 0xbebfe988U, 0x64e4c3feU, 0x9dbc8057U, 0xf0f7c086U, + 0x60787bf8U, 0x6003604dU, 0xd1fd8346U, 0xf6381fb0U, 0x7745ae04U, 0xd736fcccU, + 0x83426b33U, 0xf01eab71U, 0xb0804187U, 0x3c005e5fU, 0x77a057beU, 0xbde8ae24U, + 0x55464299U, 0xbf582e61U, 0x4e58f48fU, 0xf2ddfda2U, 0xf474ef38U, 0x8789bdc2U, + 0x5366f9c3U, 0xc8b38e74U, 0xb475f255U, 0x46fcd9b9U, 0x7aeb2661U, 0x8b1ddf84U, + 0x846a0e79U, 0x915f95e2U, 0x466e598eU, 0x20b45770U, 0x8cd55591U, 0xc902de4cU, + 0xb90bace1U, 0xbb8205d0U, 0x11a86248U, 0x7574a99eU, 0xb77f19b6U, 0xe0a9dc09U, + 0x662d09a1U, 0xc4324633U, 0xe85a1f02U, 0x09f0be8cU, 0x4a99a025U, 0x1d6efe10U, + 0x1ab93d1dU, 0x0ba5a4dfU, 0xa186f20fU, 0x2868f169U, 0xdcb7da83U, 0x573906feU, + 0xa1e2ce9bU, 0x4fcd7f52U, 0x50115e01U, 0xa70683faU, 0xa002b5c4U, 0x0de6d027U, + 0x9af88c27U, 0x773f8641U, 0xc3604c06U, 0x61a806b5U, 0xf0177a28U, 0xc0f586e0U, + 0x006058aaU, 0x30dc7d62U, 0x11e69ed7U, 0x2338ea63U, 0x53c2dd94U, 0xc2c21634U, + 0xbbcbee56U, 0x90bcb6deU, 0xebfc7da1U, 0xce591d76U, 0x6f05e409U, 0x4b7c0188U, + 0x39720a3dU, 0x7c927c24U, 0x86e3725fU, 0x724d9db9U, 0x1ac15bb4U, 0xd39eb8fcU, + 0xed545578U, 0x08fca5b5U, 0xd83d7cd3U, 0x4dad0fc4U, 0x1e50ef5eU, 0xb161e6f8U, + 0xa28514d9U, 0x6c51133cU, 0x6fd5c7e7U, 0x56e14ec4U, 0x362abfceU, 0xddc6c837U, + 0xd79a3234U, 0x92638212U, 0x670efa8eU, 0x406000e0U + }, + { + 0x3a39ce37U, 0xd3faf5cfU, 0xabc27737U, 0x5ac52d1bU, 0x5cb0679eU, 0x4fa33742U, + 0xd3822740U, 0x99bc9bbeU, 0xd5118e9dU, 0xbf0f7315U, 0xd62d1c7eU, 0xc700c47bU, + 0xb78c1b6bU, 0x21a19045U, 0xb26eb1beU, 0x6a366eb4U, 0x5748ab2fU, 0xbc946e79U, + 0xc6a376d2U, 0x6549c2c8U, 0x530ff8eeU, 0x468dde7dU, 0xd5730a1dU, 0x4cd04dc6U, + 0x2939bbdbU, 0xa9ba4650U, 0xac9526e8U, 0xbe5ee304U, 0xa1fad5f0U, 0x6a2d519aU, + 0x63ef8ce2U, 0x9a86ee22U, 0xc089c2b8U, 0x43242ef6U, 0xa51e03aaU, 0x9cf2d0a4U, + 0x83c061baU, 0x9be96a4dU, 0x8fe51550U, 0xba645bd6U, 0x2826a2f9U, 0xa73a3ae1U, + 0x4ba99586U, 0xef5562e9U, 0xc72fefd3U, 0xf752f7daU, 0x3f046f69U, 0x77fa0a59U, + 0x80e4a915U, 0x87b08601U, 0x9b09e6adU, 0x3b3ee593U, 0xe990fd5aU, 0x9e34d797U, + 0x2cf0b7d9U, 0x022b8b51U, 0x96d5ac3aU, 0x017da67dU, 0xd1cf3ed6U, 0x7c7d2d28U, + 0x1f9f25cfU, 0xadf2b89bU, 0x5ad6b472U, 0x5a88f54cU, 0xe029ac71U, 0xe019a5e6U, + 0x47b0acfdU, 0xed93fa9bU, 0xe8d3c48dU, 0x283b57ccU, 0xf8d56629U, 0x79132e28U, + 0x785f0191U, 0xed756055U, 0xf7960e44U, 0xe3d35e8cU, 0x15056dd4U, 0x88f46dbaU, + 0x03a16125U, 0x0564f0bdU, 0xc3eb9e15U, 0x3c9057a2U, 0x97271aecU, 0xa93a072aU, + 0x1b3f6d9bU, 0x1e6321f5U, 0xf59c66fbU, 0x26dcf319U, 0x7533d928U, 0xb155fdf5U, + 0x03563482U, 0x8aba3cbbU, 0x28517711U, 0xc20ad9f8U, 0xabcc5167U, 0xccad925fU, + 0x4de81751U, 0x3830dc8eU, 0x379d5862U, 0x9320f991U, 0xea7a90c2U, 0xfb3e7bceU, + 0x5121ce64U, 0x774fbe32U, 0xa8b6e37eU, 0xc3293d46U, 0x48de5369U, 0x6413e680U, + 0xa2ae0810U, 0xdd6db224U, 0x69852dfdU, 0x09072166U, 0xb39a460aU, 0x6445c0ddU, + 0x586cdecfU, 0x1c20c8aeU, 0x5bbef7ddU, 0x1b588d40U, 0xccd2017fU, 0x6bb4e3bbU, + 0xdda26a7eU, 0x3a59ff45U, 0x3e350a44U, 0xbcb4cdd5U, 0x72eacea8U, 0xfa6484bbU, + 0x8d6612aeU, 0xbf3c6f47U, 0xd29be463U, 0x542f5d9eU, 0xaec2771bU, 0xf64e6370U, + 0x740e0d8dU, 0xe75b1357U, 0xf8721671U, 0xaf537d5dU, 0x4040cb08U, 0x4eb4e2ccU, + 0x34d2466aU, 0x0115af84U, 0xe1b00428U, 0x95983a1dU, 0x06b89fb4U, 0xce6ea048U, + 0x6f3f3b82U, 0x3520ab82U, 0x011a1d4bU, 0x277227f8U, 0x611560b1U, 0xe7933fdcU, + 0xbb3a792bU, 0x344525bdU, 0xa08839e1U, 0x51ce794bU, 0x2f32c9b7U, 0xa01fbac9U, + 0xe01cc87eU, 0xbcc7d1f6U, 0xcf0111c3U, 0xa1e8aac7U, 0x1a908749U, 0xd44fbd9aU, + 0xd0dadecbU, 0xd50ada38U, 0x0339c32aU, 0xc6913667U, 0x8df9317cU, 0xe0b12b4fU, + 0xf79e59b7U, 0x43f5bb3aU, 0xf2d519ffU, 0x27d9459cU, 0xbf97222cU, 0x15e6fc2aU, + 0x0f91fc71U, 0x9b941525U, 0xfae59361U, 0xceb69cebU, 0xc2a86459U, 0x12baa8d1U, + 0xb6c1075eU, 0xe3056a0cU, 0x10d25065U, 0xcb03a442U, 0xe0ec6e0eU, 0x1698db3bU, + 0x4c98a0beU, 0x3278e964U, 0x9f1f9532U, 0xe0d392dfU, 0xd3a0342bU, 0x8971f21eU, + 0x1b0a7441U, 0x4ba3348cU, 0xc5be7120U, 0xc37632d8U, 0xdf359f8dU, 0x9b992f2eU, + 0xe60b6f47U, 0x0fe3f11dU, 0xe54cda54U, 0x1edad891U, 0xce6279cfU, 0xcd3e7e6fU, + 0x1618b166U, 0xfd2c1d05U, 0x848fd2c5U, 0xf6fb2299U, 0xf523f357U, 0xa6327623U, + 0x93a83531U, 0x56cccd02U, 0xacf08162U, 0x5a75ebb5U, 0x6e163697U, 0x88d273ccU, + 0xde966292U, 0x81b949d0U, 0x4c50901bU, 0x71c65614U, 0xe6c6c7bdU, 0x327a140aU, + 0x45e1d006U, 0xc3f27b9aU, 0xc9aa53fdU, 0x62a80f00U, 0xbb25bfe2U, 0x35bdd2f6U, + 0x71126905U, 0xb2040222U, 0xb6cbcf7cU, 0xcd769c2bU, 0x53113ec0U, 0x1640e3d3U, + 0x38abbd60U, 0x2547adf0U, 0xba38209cU, 0xf746ce76U, 0x77afa1c5U, 0x20756060U, + 0x85cbfe4eU, 0x8ae88dd8U, 0x7aaaf9b0U, 0x4cf9aa7eU, 0x1948c25cU, 0x02fb8a8cU, + 0x01c36ae4U, 0xd6ebe1f9U, 0x90d4f869U, 0xa65cdea0U, 0x3f09252dU, 0xc208e69fU, + 0xb74e6132U, 0xce77e25bU, 0x578fdfe3U, 0x3ac372e6U + } +}; + +#define F(ctx, x) \ + ((((ctx)->s[0][(x) >> 24] + (ctx)->s[1][((x) >> 16) & 0xff]) \ + ^ (ctx)->s[2][((x) >> 8) & 0xff]) \ + + (ctx)->s[3][(x) & 0xff]) + +static void block_encrypt(const crypton_blowfish_ctx *ctx, uint32_t *xl, + uint32_t *xr) +{ + uint32_t l = *xl, r = *xr; + int i; + + for (i = 0; i < 16; i += 2) { + l ^= ctx->p[i]; + r ^= F(ctx, l); + r ^= ctx->p[i + 1]; + l ^= F(ctx, r); + } + l ^= ctx->p[16]; + r ^= ctx->p[17]; + *xl = r; + *xr = l; +} + +static void block_decrypt(const crypton_blowfish_ctx *ctx, uint32_t *xl, + uint32_t *xr) +{ + uint32_t l = *xl, r = *xr; + int i; + + for (i = 16; i > 0; i -= 2) { + l ^= ctx->p[i + 1]; + r ^= F(ctx, l); + r ^= ctx->p[i]; + l ^= F(ctx, r); + } + l ^= ctx->p[1]; + r ^= ctx->p[0]; + *xl = r; + *xr = l; +} + +/* the next four bytes of the key, taken round and round */ +static uint32_t key_word(const uint8_t *key, uint32_t keylen, uint32_t *pos) +{ + uint32_t w = 0, i; + + for (i = 0; i < 4; i++) { + w = (w << 8) | key[*pos]; + *pos = (*pos + 1) % keylen; + } + return w; +} + +/* the key into the P array, and then the whole schedule rewritten by + * encrypting its way through itself */ +static void expand_key(crypton_blowfish_ctx *ctx, const uint8_t *key, + uint32_t keylen) +{ + uint32_t pos = 0, l = 0, r = 0; + int i, j; + + if (keylen > 0) + for (i = 0; i < 18; i++) + ctx->p[i] ^= key_word(key, keylen, &pos); + for (i = 0; i < 18; i += 2) { + block_encrypt(ctx, &l, &r); + ctx->p[i] = l; + ctx->p[i + 1] = r; + } + for (i = 0; i < 4; i++) + for (j = 0; j < 256; j += 2) { + block_encrypt(ctx, &l, &r); + ctx->s[i][j] = l; + ctx->s[i][j + 1] = r; + } +} + +/* the same, with the salt exclusive-ored into what is encrypted at every + * step, which is what makes the schedule depend on it. The salt is taken + * round and round as the key is, so a salt of any length will do: bcrypt + * hands it sixteen bytes and bcrypt_pbkdf hands it sixty-four. */ +static void expand_key_with_salt(crypton_blowfish_ctx *ctx, const uint8_t *key, + uint32_t keylen, const uint8_t *salt, + uint32_t saltlen) +{ + uint32_t kpos = 0, spos = 0, l = 0, r = 0; + int i, j; + + if (keylen > 0) + for (i = 0; i < 18; i++) + ctx->p[i] ^= key_word(key, keylen, &kpos); + for (i = 0; i < 18; i += 2) { + l ^= key_word(salt, saltlen, &spos); + r ^= key_word(salt, saltlen, &spos); + block_encrypt(ctx, &l, &r); + ctx->p[i] = l; + ctx->p[i + 1] = r; + } + for (i = 0; i < 4; i++) + for (j = 0; j < 256; j += 2) { + l ^= key_word(salt, saltlen, &spos); + r ^= key_word(salt, saltlen, &spos); + block_encrypt(ctx, &l, &r); + ctx->s[i][j] = l; + ctx->s[i][j + 1] = r; + } +} + +void crypton_blowfish_init(crypton_blowfish_ctx *ctx, const uint8_t *key, + uint32_t keylen) +{ + memcpy(ctx->p, initial_p, sizeof(ctx->p)); + memcpy(ctx->s, initial_s, sizeof(ctx->s)); + expand_key(ctx, key, keylen); +} + +void crypton_blowfish_encrypt(const crypton_blowfish_ctx *ctx, uint8_t *out, + const uint8_t *in, uint32_t len) +{ + uint32_t i; + + for (i = 0; i + 8 <= len; i += 8) { + uint32_t l = ((uint32_t) in[i] << 24) | ((uint32_t) in[i + 1] << 16) + | ((uint32_t) in[i + 2] << 8) | (uint32_t) in[i + 3]; + uint32_t r = ((uint32_t) in[i + 4] << 24) + | ((uint32_t) in[i + 5] << 16) + | ((uint32_t) in[i + 6] << 8) | (uint32_t) in[i + 7]; + + block_encrypt(ctx, &l, &r); + out[i] = (uint8_t) (l >> 24); + out[i + 1] = (uint8_t) (l >> 16); + out[i + 2] = (uint8_t) (l >> 8); + out[i + 3] = (uint8_t) l; + out[i + 4] = (uint8_t) (r >> 24); + out[i + 5] = (uint8_t) (r >> 16); + out[i + 6] = (uint8_t) (r >> 8); + out[i + 7] = (uint8_t) r; + } +} + +void crypton_blowfish_decrypt(const crypton_blowfish_ctx *ctx, uint8_t *out, + const uint8_t *in, uint32_t len) +{ + uint32_t i; + + for (i = 0; i + 8 <= len; i += 8) { + uint32_t l = ((uint32_t) in[i] << 24) | ((uint32_t) in[i + 1] << 16) + | ((uint32_t) in[i + 2] << 8) | (uint32_t) in[i + 3]; + uint32_t r = ((uint32_t) in[i + 4] << 24) + | ((uint32_t) in[i + 5] << 16) + | ((uint32_t) in[i + 6] << 8) | (uint32_t) in[i + 7]; + + block_decrypt(ctx, &l, &r); + out[i] = (uint8_t) (l >> 24); + out[i + 1] = (uint8_t) (l >> 16); + out[i + 2] = (uint8_t) (l >> 8); + out[i + 3] = (uint8_t) l; + out[i + 4] = (uint8_t) (r >> 24); + out[i + 5] = (uint8_t) (r >> 16); + out[i + 6] = (uint8_t) (r >> 8); + out[i + 7] = (uint8_t) r; + } +} + +int crypton_bcrypt(uint8_t out[24], uint32_t cost, const uint8_t salt[16], + const uint8_t *key, uint32_t keylen) +{ + /* "OrpheanBeholderScryDoubt", which is what bcrypt encrypts */ + static const uint8_t magic[24] = { + 0x4f, 0x72, 0x70, 0x68, 0x65, 0x61, 0x6e, 0x42, + 0x65, 0x68, 0x6f, 0x6c, 0x64, 0x65, 0x72, 0x53, + 0x63, 0x72, 0x79, 0x44, 0x6f, 0x75, 0x62, 0x74 + }; + crypton_blowfish_ctx ctx; + uint32_t rounds, i; + + if (cost < 4 || cost > 31 || keylen == 0 || keylen > 73) + return -1; + + memcpy(ctx.p, initial_p, sizeof(ctx.p)); + memcpy(ctx.s, initial_s, sizeof(ctx.s)); + expand_key_with_salt(&ctx, key, keylen, salt, 16); + rounds = (uint32_t) 1 << cost; + for (i = 0; i < rounds; i++) { + expand_key(&ctx, key, keylen); + expand_key(&ctx, salt, 16); + } + + memcpy(out, magic, sizeof(magic)); + for (i = 0; i < 64; i++) + crypton_blowfish_encrypt(&ctx, out, out, 24); + + memset(&ctx, 0, sizeof(ctx)); + return 0; +} + +int crypton_bcrypt_pbkdf_hash(uint8_t out[32], const uint8_t *pass, + uint32_t passlen, const uint8_t *salt, + uint32_t saltlen) +{ + /* "OxychromaticBlowfishSwatDynamite", which is what this one encrypts */ + static const uint8_t magic[32] = { + 0x4f, 0x78, 0x79, 0x63, 0x68, 0x72, 0x6f, 0x6d, + 0x61, 0x74, 0x69, 0x63, 0x42, 0x6c, 0x6f, 0x77, + 0x66, 0x69, 0x73, 0x68, 0x53, 0x77, 0x61, 0x74, + 0x44, 0x79, 0x6e, 0x61, 0x6d, 0x69, 0x74, 0x65 + }; + crypton_blowfish_ctx ctx; + uint32_t i, j; + + if (passlen == 0 || saltlen == 0) + return -1; + + memcpy(ctx.p, initial_p, sizeof(ctx.p)); + memcpy(ctx.s, initial_s, sizeof(ctx.s)); + expand_key_with_salt(&ctx, pass, passlen, salt, saltlen); + for (i = 0; i < 64; i++) { + expand_key(&ctx, salt, saltlen); + expand_key(&ctx, pass, passlen); + } + + /* each block encrypted sixty-four times, and stored with each half the + * way round that the original implementation stores it */ + for (i = 0; i < 4; i++) { + uint32_t l = ((uint32_t) magic[8 * i] << 24) + | ((uint32_t) magic[8 * i + 1] << 16) + | ((uint32_t) magic[8 * i + 2] << 8) + | (uint32_t) magic[8 * i + 3]; + uint32_t r = ((uint32_t) magic[8 * i + 4] << 24) + | ((uint32_t) magic[8 * i + 5] << 16) + | ((uint32_t) magic[8 * i + 6] << 8) + | (uint32_t) magic[8 * i + 7]; + + for (j = 0; j < 64; j++) + block_encrypt(&ctx, &l, &r); + out[8 * i] = (uint8_t) l; + out[8 * i + 1] = (uint8_t) (l >> 8); + out[8 * i + 2] = (uint8_t) (l >> 16); + out[8 * i + 3] = (uint8_t) (l >> 24); + out[8 * i + 4] = (uint8_t) r; + out[8 * i + 5] = (uint8_t) (r >> 8); + out[8 * i + 6] = (uint8_t) (r >> 16); + out[8 * i + 7] = (uint8_t) (r >> 24); + } + + memset(&ctx, 0, sizeof(ctx)); + return 0; +} diff --git a/cbits/crypton_blowfish.h b/cbits/crypton_blowfish.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_blowfish.h @@ -0,0 +1,44 @@ +#ifndef CRYPTON_BLOWFISH_H +#define CRYPTON_BLOWFISH_H + +#include + +/* The key schedule: the P array and the four S boxes, which is all the state + * Blowfish has. The caller keeps it; nothing here allocates. */ +typedef struct { + uint32_t p[18]; + uint32_t s[4][256]; +} crypton_blowfish_ctx; + +/* Set a schedule up from a key of keylen bytes, which has to be 1 to 56. */ +void crypton_blowfish_init(crypton_blowfish_ctx *ctx, const uint8_t *key, + uint32_t keylen); + +/* Encrypt or decrypt whole blocks: len has to be a multiple of eight, and out + * may be in. */ +void crypton_blowfish_encrypt(const crypton_blowfish_ctx *ctx, uint8_t *out, + const uint8_t *in, uint32_t len); +void crypton_blowfish_decrypt(const crypton_blowfish_ctx *ctx, uint8_t *out, + const uint8_t *in, uint32_t len); + +/* The whole of what bcrypt does with Blowfish: the key setup that costs what + * the cost says, and then the sixty-four encryptions. Writes 24 bytes, of + * which bcrypt keeps 23. The salt is 16 bytes and the key is the password + * with its terminating zero, 1 to 72 bytes of it. + * + * Returns 0, or -1 for a cost or a length it will not take. + */ +int crypton_bcrypt(uint8_t out[24], uint32_t cost, const uint8_t salt[16], + const uint8_t *key, uint32_t keylen); + +/* What bcrypt_pbkdf does with Blowfish: the same key setup, sixty-four times + * over, and then the four blocks of its own magic. Writes 32 bytes. The two + * hashes it is given are 64 bytes each in the only caller there is. + * + * Returns 0, or -1 for a length it will not take. + */ +int crypton_bcrypt_pbkdf_hash(uint8_t out[32], const uint8_t *pass, + uint32_t passlen, const uint8_t *salt, + uint32_t saltlen); + +#endif diff --git a/cbits/crypton_camellia.c b/cbits/crypton_camellia.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_camellia.c @@ -0,0 +1,697 @@ +/* + * Camellia with a 128-bit key, as RFC 3713 defines it. + * + * SP[i] is generated from that standard: the S-box byte i goes through, spread + * into the positions the P layer sends it to. P is an exclusive-or of bytes, + * so the round function is the exclusive-or of eight lookups. + */ +#include +#include + +static const uint64_t SP[8][256] = { +{ + 0x7070700070000070ULL, 0x8282820082000082ULL, 0x2c2c2c002c00002cULL, 0xececec00ec0000ecULL, + 0xb3b3b300b30000b3ULL, 0x2727270027000027ULL, 0xc0c0c000c00000c0ULL, 0xe5e5e500e50000e5ULL, + 0xe4e4e400e40000e4ULL, 0x8585850085000085ULL, 0x5757570057000057ULL, 0x3535350035000035ULL, + 0xeaeaea00ea0000eaULL, 0x0c0c0c000c00000cULL, 0xaeaeae00ae0000aeULL, 0x4141410041000041ULL, + 0x2323230023000023ULL, 0xefefef00ef0000efULL, 0x6b6b6b006b00006bULL, 0x9393930093000093ULL, + 0x4545450045000045ULL, 0x1919190019000019ULL, 0xa5a5a500a50000a5ULL, 0x2121210021000021ULL, + 0xededed00ed0000edULL, 0x0e0e0e000e00000eULL, 0x4f4f4f004f00004fULL, 0x4e4e4e004e00004eULL, + 0x1d1d1d001d00001dULL, 0x6565650065000065ULL, 0x9292920092000092ULL, 0xbdbdbd00bd0000bdULL, + 0x8686860086000086ULL, 0xb8b8b800b80000b8ULL, 0xafafaf00af0000afULL, 0x8f8f8f008f00008fULL, + 0x7c7c7c007c00007cULL, 0xebebeb00eb0000ebULL, 0x1f1f1f001f00001fULL, 0xcecece00ce0000ceULL, + 0x3e3e3e003e00003eULL, 0x3030300030000030ULL, 0xdcdcdc00dc0000dcULL, 0x5f5f5f005f00005fULL, + 0x5e5e5e005e00005eULL, 0xc5c5c500c50000c5ULL, 0x0b0b0b000b00000bULL, 0x1a1a1a001a00001aULL, + 0xa6a6a600a60000a6ULL, 0xe1e1e100e10000e1ULL, 0x3939390039000039ULL, 0xcacaca00ca0000caULL, + 0xd5d5d500d50000d5ULL, 0x4747470047000047ULL, 0x5d5d5d005d00005dULL, 0x3d3d3d003d00003dULL, + 0xd9d9d900d90000d9ULL, 0x0101010001000001ULL, 0x5a5a5a005a00005aULL, 0xd6d6d600d60000d6ULL, + 0x5151510051000051ULL, 0x5656560056000056ULL, 0x6c6c6c006c00006cULL, 0x4d4d4d004d00004dULL, + 0x8b8b8b008b00008bULL, 0x0d0d0d000d00000dULL, 0x9a9a9a009a00009aULL, 0x6666660066000066ULL, + 0xfbfbfb00fb0000fbULL, 0xcccccc00cc0000ccULL, 0xb0b0b000b00000b0ULL, 0x2d2d2d002d00002dULL, + 0x7474740074000074ULL, 0x1212120012000012ULL, 0x2b2b2b002b00002bULL, 0x2020200020000020ULL, + 0xf0f0f000f00000f0ULL, 0xb1b1b100b10000b1ULL, 0x8484840084000084ULL, 0x9999990099000099ULL, + 0xdfdfdf00df0000dfULL, 0x4c4c4c004c00004cULL, 0xcbcbcb00cb0000cbULL, 0xc2c2c200c20000c2ULL, + 0x3434340034000034ULL, 0x7e7e7e007e00007eULL, 0x7676760076000076ULL, 0x0505050005000005ULL, + 0x6d6d6d006d00006dULL, 0xb7b7b700b70000b7ULL, 0xa9a9a900a90000a9ULL, 0x3131310031000031ULL, + 0xd1d1d100d10000d1ULL, 0x1717170017000017ULL, 0x0404040004000004ULL, 0xd7d7d700d70000d7ULL, + 0x1414140014000014ULL, 0x5858580058000058ULL, 0x3a3a3a003a00003aULL, 0x6161610061000061ULL, + 0xdedede00de0000deULL, 0x1b1b1b001b00001bULL, 0x1111110011000011ULL, 0x1c1c1c001c00001cULL, + 0x3232320032000032ULL, 0x0f0f0f000f00000fULL, 0x9c9c9c009c00009cULL, 0x1616160016000016ULL, + 0x5353530053000053ULL, 0x1818180018000018ULL, 0xf2f2f200f20000f2ULL, 0x2222220022000022ULL, + 0xfefefe00fe0000feULL, 0x4444440044000044ULL, 0xcfcfcf00cf0000cfULL, 0xb2b2b200b20000b2ULL, + 0xc3c3c300c30000c3ULL, 0xb5b5b500b50000b5ULL, 0x7a7a7a007a00007aULL, 0x9191910091000091ULL, + 0x2424240024000024ULL, 0x0808080008000008ULL, 0xe8e8e800e80000e8ULL, 0xa8a8a800a80000a8ULL, + 0x6060600060000060ULL, 0xfcfcfc00fc0000fcULL, 0x6969690069000069ULL, 0x5050500050000050ULL, + 0xaaaaaa00aa0000aaULL, 0xd0d0d000d00000d0ULL, 0xa0a0a000a00000a0ULL, 0x7d7d7d007d00007dULL, + 0xa1a1a100a10000a1ULL, 0x8989890089000089ULL, 0x6262620062000062ULL, 0x9797970097000097ULL, + 0x5454540054000054ULL, 0x5b5b5b005b00005bULL, 0x1e1e1e001e00001eULL, 0x9595950095000095ULL, + 0xe0e0e000e00000e0ULL, 0xffffff00ff0000ffULL, 0x6464640064000064ULL, 0xd2d2d200d20000d2ULL, + 0x1010100010000010ULL, 0xc4c4c400c40000c4ULL, 0x0000000000000000ULL, 0x4848480048000048ULL, + 0xa3a3a300a30000a3ULL, 0xf7f7f700f70000f7ULL, 0x7575750075000075ULL, 0xdbdbdb00db0000dbULL, + 0x8a8a8a008a00008aULL, 0x0303030003000003ULL, 0xe6e6e600e60000e6ULL, 0xdadada00da0000daULL, + 0x0909090009000009ULL, 0x3f3f3f003f00003fULL, 0xdddddd00dd0000ddULL, 0x9494940094000094ULL, + 0x8787870087000087ULL, 0x5c5c5c005c00005cULL, 0x8383830083000083ULL, 0x0202020002000002ULL, + 0xcdcdcd00cd0000cdULL, 0x4a4a4a004a00004aULL, 0x9090900090000090ULL, 0x3333330033000033ULL, + 0x7373730073000073ULL, 0x6767670067000067ULL, 0xf6f6f600f60000f6ULL, 0xf3f3f300f30000f3ULL, + 0x9d9d9d009d00009dULL, 0x7f7f7f007f00007fULL, 0xbfbfbf00bf0000bfULL, 0xe2e2e200e20000e2ULL, + 0x5252520052000052ULL, 0x9b9b9b009b00009bULL, 0xd8d8d800d80000d8ULL, 0x2626260026000026ULL, + 0xc8c8c800c80000c8ULL, 0x3737370037000037ULL, 0xc6c6c600c60000c6ULL, 0x3b3b3b003b00003bULL, + 0x8181810081000081ULL, 0x9696960096000096ULL, 0x6f6f6f006f00006fULL, 0x4b4b4b004b00004bULL, + 0x1313130013000013ULL, 0xbebebe00be0000beULL, 0x6363630063000063ULL, 0x2e2e2e002e00002eULL, + 0xe9e9e900e90000e9ULL, 0x7979790079000079ULL, 0xa7a7a700a70000a7ULL, 0x8c8c8c008c00008cULL, + 0x9f9f9f009f00009fULL, 0x6e6e6e006e00006eULL, 0xbcbcbc00bc0000bcULL, 0x8e8e8e008e00008eULL, + 0x2929290029000029ULL, 0xf5f5f500f50000f5ULL, 0xf9f9f900f90000f9ULL, 0xb6b6b600b60000b6ULL, + 0x2f2f2f002f00002fULL, 0xfdfdfd00fd0000fdULL, 0xb4b4b400b40000b4ULL, 0x5959590059000059ULL, + 0x7878780078000078ULL, 0x9898980098000098ULL, 0x0606060006000006ULL, 0x6a6a6a006a00006aULL, + 0xe7e7e700e70000e7ULL, 0x4646460046000046ULL, 0x7171710071000071ULL, 0xbababa00ba0000baULL, + 0xd4d4d400d40000d4ULL, 0x2525250025000025ULL, 0xababab00ab0000abULL, 0x4242420042000042ULL, + 0x8888880088000088ULL, 0xa2a2a200a20000a2ULL, 0x8d8d8d008d00008dULL, 0xfafafa00fa0000faULL, + 0x7272720072000072ULL, 0x0707070007000007ULL, 0xb9b9b900b90000b9ULL, 0x5555550055000055ULL, + 0xf8f8f800f80000f8ULL, 0xeeeeee00ee0000eeULL, 0xacacac00ac0000acULL, 0x0a0a0a000a00000aULL, + 0x3636360036000036ULL, 0x4949490049000049ULL, 0x2a2a2a002a00002aULL, 0x6868680068000068ULL, + 0x3c3c3c003c00003cULL, 0x3838380038000038ULL, 0xf1f1f100f10000f1ULL, 0xa4a4a400a40000a4ULL, + 0x4040400040000040ULL, 0x2828280028000028ULL, 0xd3d3d300d30000d3ULL, 0x7b7b7b007b00007bULL, + 0xbbbbbb00bb0000bbULL, 0xc9c9c900c90000c9ULL, 0x4343430043000043ULL, 0xc1c1c100c10000c1ULL, + 0x1515150015000015ULL, 0xe3e3e300e30000e3ULL, 0xadadad00ad0000adULL, 0xf4f4f400f40000f4ULL, + 0x7777770077000077ULL, 0xc7c7c700c70000c7ULL, 0x8080800080000080ULL, 0x9e9e9e009e00009eULL, +}, +{ + 0x00e0e0e0e0e00000ULL, 0x0005050505050000ULL, 0x0058585858580000ULL, 0x00d9d9d9d9d90000ULL, + 0x0067676767670000ULL, 0x004e4e4e4e4e0000ULL, 0x0081818181810000ULL, 0x00cbcbcbcbcb0000ULL, + 0x00c9c9c9c9c90000ULL, 0x000b0b0b0b0b0000ULL, 0x00aeaeaeaeae0000ULL, 0x006a6a6a6a6a0000ULL, + 0x00d5d5d5d5d50000ULL, 0x0018181818180000ULL, 0x005d5d5d5d5d0000ULL, 0x0082828282820000ULL, + 0x0046464646460000ULL, 0x00dfdfdfdfdf0000ULL, 0x00d6d6d6d6d60000ULL, 0x0027272727270000ULL, + 0x008a8a8a8a8a0000ULL, 0x0032323232320000ULL, 0x004b4b4b4b4b0000ULL, 0x0042424242420000ULL, + 0x00dbdbdbdbdb0000ULL, 0x001c1c1c1c1c0000ULL, 0x009e9e9e9e9e0000ULL, 0x009c9c9c9c9c0000ULL, + 0x003a3a3a3a3a0000ULL, 0x00cacacacaca0000ULL, 0x0025252525250000ULL, 0x007b7b7b7b7b0000ULL, + 0x000d0d0d0d0d0000ULL, 0x0071717171710000ULL, 0x005f5f5f5f5f0000ULL, 0x001f1f1f1f1f0000ULL, + 0x00f8f8f8f8f80000ULL, 0x00d7d7d7d7d70000ULL, 0x003e3e3e3e3e0000ULL, 0x009d9d9d9d9d0000ULL, + 0x007c7c7c7c7c0000ULL, 0x0060606060600000ULL, 0x00b9b9b9b9b90000ULL, 0x00bebebebebe0000ULL, + 0x00bcbcbcbcbc0000ULL, 0x008b8b8b8b8b0000ULL, 0x0016161616160000ULL, 0x0034343434340000ULL, + 0x004d4d4d4d4d0000ULL, 0x00c3c3c3c3c30000ULL, 0x0072727272720000ULL, 0x0095959595950000ULL, + 0x00ababababab0000ULL, 0x008e8e8e8e8e0000ULL, 0x00bababababa0000ULL, 0x007a7a7a7a7a0000ULL, + 0x00b3b3b3b3b30000ULL, 0x0002020202020000ULL, 0x00b4b4b4b4b40000ULL, 0x00adadadadad0000ULL, + 0x00a2a2a2a2a20000ULL, 0x00acacacacac0000ULL, 0x00d8d8d8d8d80000ULL, 0x009a9a9a9a9a0000ULL, + 0x0017171717170000ULL, 0x001a1a1a1a1a0000ULL, 0x0035353535350000ULL, 0x00cccccccccc0000ULL, + 0x00f7f7f7f7f70000ULL, 0x0099999999990000ULL, 0x0061616161610000ULL, 0x005a5a5a5a5a0000ULL, + 0x00e8e8e8e8e80000ULL, 0x0024242424240000ULL, 0x0056565656560000ULL, 0x0040404040400000ULL, + 0x00e1e1e1e1e10000ULL, 0x0063636363630000ULL, 0x0009090909090000ULL, 0x0033333333330000ULL, + 0x00bfbfbfbfbf0000ULL, 0x0098989898980000ULL, 0x0097979797970000ULL, 0x0085858585850000ULL, + 0x0068686868680000ULL, 0x00fcfcfcfcfc0000ULL, 0x00ececececec0000ULL, 0x000a0a0a0a0a0000ULL, + 0x00dadadadada0000ULL, 0x006f6f6f6f6f0000ULL, 0x0053535353530000ULL, 0x0062626262620000ULL, + 0x00a3a3a3a3a30000ULL, 0x002e2e2e2e2e0000ULL, 0x0008080808080000ULL, 0x00afafafafaf0000ULL, + 0x0028282828280000ULL, 0x00b0b0b0b0b00000ULL, 0x0074747474740000ULL, 0x00c2c2c2c2c20000ULL, + 0x00bdbdbdbdbd0000ULL, 0x0036363636360000ULL, 0x0022222222220000ULL, 0x0038383838380000ULL, + 0x0064646464640000ULL, 0x001e1e1e1e1e0000ULL, 0x0039393939390000ULL, 0x002c2c2c2c2c0000ULL, + 0x00a6a6a6a6a60000ULL, 0x0030303030300000ULL, 0x00e5e5e5e5e50000ULL, 0x0044444444440000ULL, + 0x00fdfdfdfdfd0000ULL, 0x0088888888880000ULL, 0x009f9f9f9f9f0000ULL, 0x0065656565650000ULL, + 0x0087878787870000ULL, 0x006b6b6b6b6b0000ULL, 0x00f4f4f4f4f40000ULL, 0x0023232323230000ULL, + 0x0048484848480000ULL, 0x0010101010100000ULL, 0x00d1d1d1d1d10000ULL, 0x0051515151510000ULL, + 0x00c0c0c0c0c00000ULL, 0x00f9f9f9f9f90000ULL, 0x00d2d2d2d2d20000ULL, 0x00a0a0a0a0a00000ULL, + 0x0055555555550000ULL, 0x00a1a1a1a1a10000ULL, 0x0041414141410000ULL, 0x00fafafafafa0000ULL, + 0x0043434343430000ULL, 0x0013131313130000ULL, 0x00c4c4c4c4c40000ULL, 0x002f2f2f2f2f0000ULL, + 0x00a8a8a8a8a80000ULL, 0x00b6b6b6b6b60000ULL, 0x003c3c3c3c3c0000ULL, 0x002b2b2b2b2b0000ULL, + 0x00c1c1c1c1c10000ULL, 0x00ffffffffff0000ULL, 0x00c8c8c8c8c80000ULL, 0x00a5a5a5a5a50000ULL, + 0x0020202020200000ULL, 0x0089898989890000ULL, 0x0000000000000000ULL, 0x0090909090900000ULL, + 0x0047474747470000ULL, 0x00efefefefef0000ULL, 0x00eaeaeaeaea0000ULL, 0x00b7b7b7b7b70000ULL, + 0x0015151515150000ULL, 0x0006060606060000ULL, 0x00cdcdcdcdcd0000ULL, 0x00b5b5b5b5b50000ULL, + 0x0012121212120000ULL, 0x007e7e7e7e7e0000ULL, 0x00bbbbbbbbbb0000ULL, 0x0029292929290000ULL, + 0x000f0f0f0f0f0000ULL, 0x00b8b8b8b8b80000ULL, 0x0007070707070000ULL, 0x0004040404040000ULL, + 0x009b9b9b9b9b0000ULL, 0x0094949494940000ULL, 0x0021212121210000ULL, 0x0066666666660000ULL, + 0x00e6e6e6e6e60000ULL, 0x00cecececece0000ULL, 0x00ededededed0000ULL, 0x00e7e7e7e7e70000ULL, + 0x003b3b3b3b3b0000ULL, 0x00fefefefefe0000ULL, 0x007f7f7f7f7f0000ULL, 0x00c5c5c5c5c50000ULL, + 0x00a4a4a4a4a40000ULL, 0x0037373737370000ULL, 0x00b1b1b1b1b10000ULL, 0x004c4c4c4c4c0000ULL, + 0x0091919191910000ULL, 0x006e6e6e6e6e0000ULL, 0x008d8d8d8d8d0000ULL, 0x0076767676760000ULL, + 0x0003030303030000ULL, 0x002d2d2d2d2d0000ULL, 0x00dedededede0000ULL, 0x0096969696960000ULL, + 0x0026262626260000ULL, 0x007d7d7d7d7d0000ULL, 0x00c6c6c6c6c60000ULL, 0x005c5c5c5c5c0000ULL, + 0x00d3d3d3d3d30000ULL, 0x00f2f2f2f2f20000ULL, 0x004f4f4f4f4f0000ULL, 0x0019191919190000ULL, + 0x003f3f3f3f3f0000ULL, 0x00dcdcdcdcdc0000ULL, 0x0079797979790000ULL, 0x001d1d1d1d1d0000ULL, + 0x0052525252520000ULL, 0x00ebebebebeb0000ULL, 0x00f3f3f3f3f30000ULL, 0x006d6d6d6d6d0000ULL, + 0x005e5e5e5e5e0000ULL, 0x00fbfbfbfbfb0000ULL, 0x0069696969690000ULL, 0x00b2b2b2b2b20000ULL, + 0x00f0f0f0f0f00000ULL, 0x0031313131310000ULL, 0x000c0c0c0c0c0000ULL, 0x00d4d4d4d4d40000ULL, + 0x00cfcfcfcfcf0000ULL, 0x008c8c8c8c8c0000ULL, 0x00e2e2e2e2e20000ULL, 0x0075757575750000ULL, + 0x00a9a9a9a9a90000ULL, 0x004a4a4a4a4a0000ULL, 0x0057575757570000ULL, 0x0084848484840000ULL, + 0x0011111111110000ULL, 0x0045454545450000ULL, 0x001b1b1b1b1b0000ULL, 0x00f5f5f5f5f50000ULL, + 0x00e4e4e4e4e40000ULL, 0x000e0e0e0e0e0000ULL, 0x0073737373730000ULL, 0x00aaaaaaaaaa0000ULL, + 0x00f1f1f1f1f10000ULL, 0x00dddddddddd0000ULL, 0x0059595959590000ULL, 0x0014141414140000ULL, + 0x006c6c6c6c6c0000ULL, 0x0092929292920000ULL, 0x0054545454540000ULL, 0x00d0d0d0d0d00000ULL, + 0x0078787878780000ULL, 0x0070707070700000ULL, 0x00e3e3e3e3e30000ULL, 0x0049494949490000ULL, + 0x0080808080800000ULL, 0x0050505050500000ULL, 0x00a7a7a7a7a70000ULL, 0x00f6f6f6f6f60000ULL, + 0x0077777777770000ULL, 0x0093939393930000ULL, 0x0086868686860000ULL, 0x0083838383830000ULL, + 0x002a2a2a2a2a0000ULL, 0x00c7c7c7c7c70000ULL, 0x005b5b5b5b5b0000ULL, 0x00e9e9e9e9e90000ULL, + 0x00eeeeeeeeee0000ULL, 0x008f8f8f8f8f0000ULL, 0x0001010101010000ULL, 0x003d3d3d3d3d0000ULL, +}, +{ + 0x3800383800383800ULL, 0x4100414100414100ULL, 0x1600161600161600ULL, 0x7600767600767600ULL, + 0xd900d9d900d9d900ULL, 0x9300939300939300ULL, 0x6000606000606000ULL, 0xf200f2f200f2f200ULL, + 0x7200727200727200ULL, 0xc200c2c200c2c200ULL, 0xab00abab00abab00ULL, 0x9a009a9a009a9a00ULL, + 0x7500757500757500ULL, 0x0600060600060600ULL, 0x5700575700575700ULL, 0xa000a0a000a0a000ULL, + 0x9100919100919100ULL, 0xf700f7f700f7f700ULL, 0xb500b5b500b5b500ULL, 0xc900c9c900c9c900ULL, + 0xa200a2a200a2a200ULL, 0x8c008c8c008c8c00ULL, 0xd200d2d200d2d200ULL, 0x9000909000909000ULL, + 0xf600f6f600f6f600ULL, 0x0700070700070700ULL, 0xa700a7a700a7a700ULL, 0x2700272700272700ULL, + 0x8e008e8e008e8e00ULL, 0xb200b2b200b2b200ULL, 0x4900494900494900ULL, 0xde00dede00dede00ULL, + 0x4300434300434300ULL, 0x5c005c5c005c5c00ULL, 0xd700d7d700d7d700ULL, 0xc700c7c700c7c700ULL, + 0x3e003e3e003e3e00ULL, 0xf500f5f500f5f500ULL, 0x8f008f8f008f8f00ULL, 0x6700676700676700ULL, + 0x1f001f1f001f1f00ULL, 0x1800181800181800ULL, 0x6e006e6e006e6e00ULL, 0xaf00afaf00afaf00ULL, + 0x2f002f2f002f2f00ULL, 0xe200e2e200e2e200ULL, 0x8500858500858500ULL, 0x0d000d0d000d0d00ULL, + 0x5300535300535300ULL, 0xf000f0f000f0f000ULL, 0x9c009c9c009c9c00ULL, 0x6500656500656500ULL, + 0xea00eaea00eaea00ULL, 0xa300a3a300a3a300ULL, 0xae00aeae00aeae00ULL, 0x9e009e9e009e9e00ULL, + 0xec00ecec00ecec00ULL, 0x8000808000808000ULL, 0x2d002d2d002d2d00ULL, 0x6b006b6b006b6b00ULL, + 0xa800a8a800a8a800ULL, 0x2b002b2b002b2b00ULL, 0x3600363600363600ULL, 0xa600a6a600a6a600ULL, + 0xc500c5c500c5c500ULL, 0x8600868600868600ULL, 0x4d004d4d004d4d00ULL, 0x3300333300333300ULL, + 0xfd00fdfd00fdfd00ULL, 0x6600666600666600ULL, 0x5800585800585800ULL, 0x9600969600969600ULL, + 0x3a003a3a003a3a00ULL, 0x0900090900090900ULL, 0x9500959500959500ULL, 0x1000101000101000ULL, + 0x7800787800787800ULL, 0xd800d8d800d8d800ULL, 0x4200424200424200ULL, 0xcc00cccc00cccc00ULL, + 0xef00efef00efef00ULL, 0x2600262600262600ULL, 0xe500e5e500e5e500ULL, 0x6100616100616100ULL, + 0x1a001a1a001a1a00ULL, 0x3f003f3f003f3f00ULL, 0x3b003b3b003b3b00ULL, 0x8200828200828200ULL, + 0xb600b6b600b6b600ULL, 0xdb00dbdb00dbdb00ULL, 0xd400d4d400d4d400ULL, 0x9800989800989800ULL, + 0xe800e8e800e8e800ULL, 0x8b008b8b008b8b00ULL, 0x0200020200020200ULL, 0xeb00ebeb00ebeb00ULL, + 0x0a000a0a000a0a00ULL, 0x2c002c2c002c2c00ULL, 0x1d001d1d001d1d00ULL, 0xb000b0b000b0b000ULL, + 0x6f006f6f006f6f00ULL, 0x8d008d8d008d8d00ULL, 0x8800888800888800ULL, 0x0e000e0e000e0e00ULL, + 0x1900191900191900ULL, 0x8700878700878700ULL, 0x4e004e4e004e4e00ULL, 0x0b000b0b000b0b00ULL, + 0xa900a9a900a9a900ULL, 0x0c000c0c000c0c00ULL, 0x7900797900797900ULL, 0x1100111100111100ULL, + 0x7f007f7f007f7f00ULL, 0x2200222200222200ULL, 0xe700e7e700e7e700ULL, 0x5900595900595900ULL, + 0xe100e1e100e1e100ULL, 0xda00dada00dada00ULL, 0x3d003d3d003d3d00ULL, 0xc800c8c800c8c800ULL, + 0x1200121200121200ULL, 0x0400040400040400ULL, 0x7400747400747400ULL, 0x5400545400545400ULL, + 0x3000303000303000ULL, 0x7e007e7e007e7e00ULL, 0xb400b4b400b4b400ULL, 0x2800282800282800ULL, + 0x5500555500555500ULL, 0x6800686800686800ULL, 0x5000505000505000ULL, 0xbe00bebe00bebe00ULL, + 0xd000d0d000d0d000ULL, 0xc400c4c400c4c400ULL, 0x3100313100313100ULL, 0xcb00cbcb00cbcb00ULL, + 0x2a002a2a002a2a00ULL, 0xad00adad00adad00ULL, 0x0f000f0f000f0f00ULL, 0xca00caca00caca00ULL, + 0x7000707000707000ULL, 0xff00ffff00ffff00ULL, 0x3200323200323200ULL, 0x6900696900696900ULL, + 0x0800080800080800ULL, 0x6200626200626200ULL, 0x0000000000000000ULL, 0x2400242400242400ULL, + 0xd100d1d100d1d100ULL, 0xfb00fbfb00fbfb00ULL, 0xba00baba00baba00ULL, 0xed00eded00eded00ULL, + 0x4500454500454500ULL, 0x8100818100818100ULL, 0x7300737300737300ULL, 0x6d006d6d006d6d00ULL, + 0x8400848400848400ULL, 0x9f009f9f009f9f00ULL, 0xee00eeee00eeee00ULL, 0x4a004a4a004a4a00ULL, + 0xc300c3c300c3c300ULL, 0x2e002e2e002e2e00ULL, 0xc100c1c100c1c100ULL, 0x0100010100010100ULL, + 0xe600e6e600e6e600ULL, 0x2500252500252500ULL, 0x4800484800484800ULL, 0x9900999900999900ULL, + 0xb900b9b900b9b900ULL, 0xb300b3b300b3b300ULL, 0x7b007b7b007b7b00ULL, 0xf900f9f900f9f900ULL, + 0xce00cece00cece00ULL, 0xbf00bfbf00bfbf00ULL, 0xdf00dfdf00dfdf00ULL, 0x7100717100717100ULL, + 0x2900292900292900ULL, 0xcd00cdcd00cdcd00ULL, 0x6c006c6c006c6c00ULL, 0x1300131300131300ULL, + 0x6400646400646400ULL, 0x9b009b9b009b9b00ULL, 0x6300636300636300ULL, 0x9d009d9d009d9d00ULL, + 0xc000c0c000c0c000ULL, 0x4b004b4b004b4b00ULL, 0xb700b7b700b7b700ULL, 0xa500a5a500a5a500ULL, + 0x8900898900898900ULL, 0x5f005f5f005f5f00ULL, 0xb100b1b100b1b100ULL, 0x1700171700171700ULL, + 0xf400f4f400f4f400ULL, 0xbc00bcbc00bcbc00ULL, 0xd300d3d300d3d300ULL, 0x4600464600464600ULL, + 0xcf00cfcf00cfcf00ULL, 0x3700373700373700ULL, 0x5e005e5e005e5e00ULL, 0x4700474700474700ULL, + 0x9400949400949400ULL, 0xfa00fafa00fafa00ULL, 0xfc00fcfc00fcfc00ULL, 0x5b005b5b005b5b00ULL, + 0x9700979700979700ULL, 0xfe00fefe00fefe00ULL, 0x5a005a5a005a5a00ULL, 0xac00acac00acac00ULL, + 0x3c003c3c003c3c00ULL, 0x4c004c4c004c4c00ULL, 0x0300030300030300ULL, 0x3500353500353500ULL, + 0xf300f3f300f3f300ULL, 0x2300232300232300ULL, 0xb800b8b800b8b800ULL, 0x5d005d5d005d5d00ULL, + 0x6a006a6a006a6a00ULL, 0x9200929200929200ULL, 0xd500d5d500d5d500ULL, 0x2100212100212100ULL, + 0x4400444400444400ULL, 0x5100515100515100ULL, 0xc600c6c600c6c600ULL, 0x7d007d7d007d7d00ULL, + 0x3900393900393900ULL, 0x8300838300838300ULL, 0xdc00dcdc00dcdc00ULL, 0xaa00aaaa00aaaa00ULL, + 0x7c007c7c007c7c00ULL, 0x7700777700777700ULL, 0x5600565600565600ULL, 0x0500050500050500ULL, + 0x1b001b1b001b1b00ULL, 0xa400a4a400a4a400ULL, 0x1500151500151500ULL, 0x3400343400343400ULL, + 0x1e001e1e001e1e00ULL, 0x1c001c1c001c1c00ULL, 0xf800f8f800f8f800ULL, 0x5200525200525200ULL, + 0x2000202000202000ULL, 0x1400141400141400ULL, 0xe900e9e900e9e900ULL, 0xbd00bdbd00bdbd00ULL, + 0xdd00dddd00dddd00ULL, 0xe400e4e400e4e400ULL, 0xa100a1a100a1a100ULL, 0xe000e0e000e0e000ULL, + 0x8a008a8a008a8a00ULL, 0xf100f1f100f1f100ULL, 0xd600d6d600d6d600ULL, 0x7a007a7a007a7a00ULL, + 0xbb00bbbb00bbbb00ULL, 0xe300e3e300e3e300ULL, 0x4000404000404000ULL, 0x4f004f4f004f4f00ULL, +}, +{ + 0x7070007000007070ULL, 0x2c2c002c00002c2cULL, 0xb3b300b30000b3b3ULL, 0xc0c000c00000c0c0ULL, + 0xe4e400e40000e4e4ULL, 0x5757005700005757ULL, 0xeaea00ea0000eaeaULL, 0xaeae00ae0000aeaeULL, + 0x2323002300002323ULL, 0x6b6b006b00006b6bULL, 0x4545004500004545ULL, 0xa5a500a50000a5a5ULL, + 0xeded00ed0000ededULL, 0x4f4f004f00004f4fULL, 0x1d1d001d00001d1dULL, 0x9292009200009292ULL, + 0x8686008600008686ULL, 0xafaf00af0000afafULL, 0x7c7c007c00007c7cULL, 0x1f1f001f00001f1fULL, + 0x3e3e003e00003e3eULL, 0xdcdc00dc0000dcdcULL, 0x5e5e005e00005e5eULL, 0x0b0b000b00000b0bULL, + 0xa6a600a60000a6a6ULL, 0x3939003900003939ULL, 0xd5d500d50000d5d5ULL, 0x5d5d005d00005d5dULL, + 0xd9d900d90000d9d9ULL, 0x5a5a005a00005a5aULL, 0x5151005100005151ULL, 0x6c6c006c00006c6cULL, + 0x8b8b008b00008b8bULL, 0x9a9a009a00009a9aULL, 0xfbfb00fb0000fbfbULL, 0xb0b000b00000b0b0ULL, + 0x7474007400007474ULL, 0x2b2b002b00002b2bULL, 0xf0f000f00000f0f0ULL, 0x8484008400008484ULL, + 0xdfdf00df0000dfdfULL, 0xcbcb00cb0000cbcbULL, 0x3434003400003434ULL, 0x7676007600007676ULL, + 0x6d6d006d00006d6dULL, 0xa9a900a90000a9a9ULL, 0xd1d100d10000d1d1ULL, 0x0404000400000404ULL, + 0x1414001400001414ULL, 0x3a3a003a00003a3aULL, 0xdede00de0000dedeULL, 0x1111001100001111ULL, + 0x3232003200003232ULL, 0x9c9c009c00009c9cULL, 0x5353005300005353ULL, 0xf2f200f20000f2f2ULL, + 0xfefe00fe0000fefeULL, 0xcfcf00cf0000cfcfULL, 0xc3c300c30000c3c3ULL, 0x7a7a007a00007a7aULL, + 0x2424002400002424ULL, 0xe8e800e80000e8e8ULL, 0x6060006000006060ULL, 0x6969006900006969ULL, + 0xaaaa00aa0000aaaaULL, 0xa0a000a00000a0a0ULL, 0xa1a100a10000a1a1ULL, 0x6262006200006262ULL, + 0x5454005400005454ULL, 0x1e1e001e00001e1eULL, 0xe0e000e00000e0e0ULL, 0x6464006400006464ULL, + 0x1010001000001010ULL, 0x0000000000000000ULL, 0xa3a300a30000a3a3ULL, 0x7575007500007575ULL, + 0x8a8a008a00008a8aULL, 0xe6e600e60000e6e6ULL, 0x0909000900000909ULL, 0xdddd00dd0000ddddULL, + 0x8787008700008787ULL, 0x8383008300008383ULL, 0xcdcd00cd0000cdcdULL, 0x9090009000009090ULL, + 0x7373007300007373ULL, 0xf6f600f60000f6f6ULL, 0x9d9d009d00009d9dULL, 0xbfbf00bf0000bfbfULL, + 0x5252005200005252ULL, 0xd8d800d80000d8d8ULL, 0xc8c800c80000c8c8ULL, 0xc6c600c60000c6c6ULL, + 0x8181008100008181ULL, 0x6f6f006f00006f6fULL, 0x1313001300001313ULL, 0x6363006300006363ULL, + 0xe9e900e90000e9e9ULL, 0xa7a700a70000a7a7ULL, 0x9f9f009f00009f9fULL, 0xbcbc00bc0000bcbcULL, + 0x2929002900002929ULL, 0xf9f900f90000f9f9ULL, 0x2f2f002f00002f2fULL, 0xb4b400b40000b4b4ULL, + 0x7878007800007878ULL, 0x0606000600000606ULL, 0xe7e700e70000e7e7ULL, 0x7171007100007171ULL, + 0xd4d400d40000d4d4ULL, 0xabab00ab0000ababULL, 0x8888008800008888ULL, 0x8d8d008d00008d8dULL, + 0x7272007200007272ULL, 0xb9b900b90000b9b9ULL, 0xf8f800f80000f8f8ULL, 0xacac00ac0000acacULL, + 0x3636003600003636ULL, 0x2a2a002a00002a2aULL, 0x3c3c003c00003c3cULL, 0xf1f100f10000f1f1ULL, + 0x4040004000004040ULL, 0xd3d300d30000d3d3ULL, 0xbbbb00bb0000bbbbULL, 0x4343004300004343ULL, + 0x1515001500001515ULL, 0xadad00ad0000adadULL, 0x7777007700007777ULL, 0x8080008000008080ULL, + 0x8282008200008282ULL, 0xecec00ec0000ececULL, 0x2727002700002727ULL, 0xe5e500e50000e5e5ULL, + 0x8585008500008585ULL, 0x3535003500003535ULL, 0x0c0c000c00000c0cULL, 0x4141004100004141ULL, + 0xefef00ef0000efefULL, 0x9393009300009393ULL, 0x1919001900001919ULL, 0x2121002100002121ULL, + 0x0e0e000e00000e0eULL, 0x4e4e004e00004e4eULL, 0x6565006500006565ULL, 0xbdbd00bd0000bdbdULL, + 0xb8b800b80000b8b8ULL, 0x8f8f008f00008f8fULL, 0xebeb00eb0000ebebULL, 0xcece00ce0000ceceULL, + 0x3030003000003030ULL, 0x5f5f005f00005f5fULL, 0xc5c500c50000c5c5ULL, 0x1a1a001a00001a1aULL, + 0xe1e100e10000e1e1ULL, 0xcaca00ca0000cacaULL, 0x4747004700004747ULL, 0x3d3d003d00003d3dULL, + 0x0101000100000101ULL, 0xd6d600d60000d6d6ULL, 0x5656005600005656ULL, 0x4d4d004d00004d4dULL, + 0x0d0d000d00000d0dULL, 0x6666006600006666ULL, 0xcccc00cc0000ccccULL, 0x2d2d002d00002d2dULL, + 0x1212001200001212ULL, 0x2020002000002020ULL, 0xb1b100b10000b1b1ULL, 0x9999009900009999ULL, + 0x4c4c004c00004c4cULL, 0xc2c200c20000c2c2ULL, 0x7e7e007e00007e7eULL, 0x0505000500000505ULL, + 0xb7b700b70000b7b7ULL, 0x3131003100003131ULL, 0x1717001700001717ULL, 0xd7d700d70000d7d7ULL, + 0x5858005800005858ULL, 0x6161006100006161ULL, 0x1b1b001b00001b1bULL, 0x1c1c001c00001c1cULL, + 0x0f0f000f00000f0fULL, 0x1616001600001616ULL, 0x1818001800001818ULL, 0x2222002200002222ULL, + 0x4444004400004444ULL, 0xb2b200b20000b2b2ULL, 0xb5b500b50000b5b5ULL, 0x9191009100009191ULL, + 0x0808000800000808ULL, 0xa8a800a80000a8a8ULL, 0xfcfc00fc0000fcfcULL, 0x5050005000005050ULL, + 0xd0d000d00000d0d0ULL, 0x7d7d007d00007d7dULL, 0x8989008900008989ULL, 0x9797009700009797ULL, + 0x5b5b005b00005b5bULL, 0x9595009500009595ULL, 0xffff00ff0000ffffULL, 0xd2d200d20000d2d2ULL, + 0xc4c400c40000c4c4ULL, 0x4848004800004848ULL, 0xf7f700f70000f7f7ULL, 0xdbdb00db0000dbdbULL, + 0x0303000300000303ULL, 0xdada00da0000dadaULL, 0x3f3f003f00003f3fULL, 0x9494009400009494ULL, + 0x5c5c005c00005c5cULL, 0x0202000200000202ULL, 0x4a4a004a00004a4aULL, 0x3333003300003333ULL, + 0x6767006700006767ULL, 0xf3f300f30000f3f3ULL, 0x7f7f007f00007f7fULL, 0xe2e200e20000e2e2ULL, + 0x9b9b009b00009b9bULL, 0x2626002600002626ULL, 0x3737003700003737ULL, 0x3b3b003b00003b3bULL, + 0x9696009600009696ULL, 0x4b4b004b00004b4bULL, 0xbebe00be0000bebeULL, 0x2e2e002e00002e2eULL, + 0x7979007900007979ULL, 0x8c8c008c00008c8cULL, 0x6e6e006e00006e6eULL, 0x8e8e008e00008e8eULL, + 0xf5f500f50000f5f5ULL, 0xb6b600b60000b6b6ULL, 0xfdfd00fd0000fdfdULL, 0x5959005900005959ULL, + 0x9898009800009898ULL, 0x6a6a006a00006a6aULL, 0x4646004600004646ULL, 0xbaba00ba0000babaULL, + 0x2525002500002525ULL, 0x4242004200004242ULL, 0xa2a200a20000a2a2ULL, 0xfafa00fa0000fafaULL, + 0x0707000700000707ULL, 0x5555005500005555ULL, 0xeeee00ee0000eeeeULL, 0x0a0a000a00000a0aULL, + 0x4949004900004949ULL, 0x6868006800006868ULL, 0x3838003800003838ULL, 0xa4a400a40000a4a4ULL, + 0x2828002800002828ULL, 0x7b7b007b00007b7bULL, 0xc9c900c90000c9c9ULL, 0xc1c100c10000c1c1ULL, + 0xe3e300e30000e3e3ULL, 0xf4f400f40000f4f4ULL, 0xc7c700c70000c7c7ULL, 0x9e9e009e00009e9eULL, +}, +{ + 0x00e0e0e000e0e0e0ULL, 0x0005050500050505ULL, 0x0058585800585858ULL, 0x00d9d9d900d9d9d9ULL, + 0x0067676700676767ULL, 0x004e4e4e004e4e4eULL, 0x0081818100818181ULL, 0x00cbcbcb00cbcbcbULL, + 0x00c9c9c900c9c9c9ULL, 0x000b0b0b000b0b0bULL, 0x00aeaeae00aeaeaeULL, 0x006a6a6a006a6a6aULL, + 0x00d5d5d500d5d5d5ULL, 0x0018181800181818ULL, 0x005d5d5d005d5d5dULL, 0x0082828200828282ULL, + 0x0046464600464646ULL, 0x00dfdfdf00dfdfdfULL, 0x00d6d6d600d6d6d6ULL, 0x0027272700272727ULL, + 0x008a8a8a008a8a8aULL, 0x0032323200323232ULL, 0x004b4b4b004b4b4bULL, 0x0042424200424242ULL, + 0x00dbdbdb00dbdbdbULL, 0x001c1c1c001c1c1cULL, 0x009e9e9e009e9e9eULL, 0x009c9c9c009c9c9cULL, + 0x003a3a3a003a3a3aULL, 0x00cacaca00cacacaULL, 0x0025252500252525ULL, 0x007b7b7b007b7b7bULL, + 0x000d0d0d000d0d0dULL, 0x0071717100717171ULL, 0x005f5f5f005f5f5fULL, 0x001f1f1f001f1f1fULL, + 0x00f8f8f800f8f8f8ULL, 0x00d7d7d700d7d7d7ULL, 0x003e3e3e003e3e3eULL, 0x009d9d9d009d9d9dULL, + 0x007c7c7c007c7c7cULL, 0x0060606000606060ULL, 0x00b9b9b900b9b9b9ULL, 0x00bebebe00bebebeULL, + 0x00bcbcbc00bcbcbcULL, 0x008b8b8b008b8b8bULL, 0x0016161600161616ULL, 0x0034343400343434ULL, + 0x004d4d4d004d4d4dULL, 0x00c3c3c300c3c3c3ULL, 0x0072727200727272ULL, 0x0095959500959595ULL, + 0x00ababab00abababULL, 0x008e8e8e008e8e8eULL, 0x00bababa00bababaULL, 0x007a7a7a007a7a7aULL, + 0x00b3b3b300b3b3b3ULL, 0x0002020200020202ULL, 0x00b4b4b400b4b4b4ULL, 0x00adadad00adadadULL, + 0x00a2a2a200a2a2a2ULL, 0x00acacac00acacacULL, 0x00d8d8d800d8d8d8ULL, 0x009a9a9a009a9a9aULL, + 0x0017171700171717ULL, 0x001a1a1a001a1a1aULL, 0x0035353500353535ULL, 0x00cccccc00ccccccULL, + 0x00f7f7f700f7f7f7ULL, 0x0099999900999999ULL, 0x0061616100616161ULL, 0x005a5a5a005a5a5aULL, + 0x00e8e8e800e8e8e8ULL, 0x0024242400242424ULL, 0x0056565600565656ULL, 0x0040404000404040ULL, + 0x00e1e1e100e1e1e1ULL, 0x0063636300636363ULL, 0x0009090900090909ULL, 0x0033333300333333ULL, + 0x00bfbfbf00bfbfbfULL, 0x0098989800989898ULL, 0x0097979700979797ULL, 0x0085858500858585ULL, + 0x0068686800686868ULL, 0x00fcfcfc00fcfcfcULL, 0x00ececec00ecececULL, 0x000a0a0a000a0a0aULL, + 0x00dadada00dadadaULL, 0x006f6f6f006f6f6fULL, 0x0053535300535353ULL, 0x0062626200626262ULL, + 0x00a3a3a300a3a3a3ULL, 0x002e2e2e002e2e2eULL, 0x0008080800080808ULL, 0x00afafaf00afafafULL, + 0x0028282800282828ULL, 0x00b0b0b000b0b0b0ULL, 0x0074747400747474ULL, 0x00c2c2c200c2c2c2ULL, + 0x00bdbdbd00bdbdbdULL, 0x0036363600363636ULL, 0x0022222200222222ULL, 0x0038383800383838ULL, + 0x0064646400646464ULL, 0x001e1e1e001e1e1eULL, 0x0039393900393939ULL, 0x002c2c2c002c2c2cULL, + 0x00a6a6a600a6a6a6ULL, 0x0030303000303030ULL, 0x00e5e5e500e5e5e5ULL, 0x0044444400444444ULL, + 0x00fdfdfd00fdfdfdULL, 0x0088888800888888ULL, 0x009f9f9f009f9f9fULL, 0x0065656500656565ULL, + 0x0087878700878787ULL, 0x006b6b6b006b6b6bULL, 0x00f4f4f400f4f4f4ULL, 0x0023232300232323ULL, + 0x0048484800484848ULL, 0x0010101000101010ULL, 0x00d1d1d100d1d1d1ULL, 0x0051515100515151ULL, + 0x00c0c0c000c0c0c0ULL, 0x00f9f9f900f9f9f9ULL, 0x00d2d2d200d2d2d2ULL, 0x00a0a0a000a0a0a0ULL, + 0x0055555500555555ULL, 0x00a1a1a100a1a1a1ULL, 0x0041414100414141ULL, 0x00fafafa00fafafaULL, + 0x0043434300434343ULL, 0x0013131300131313ULL, 0x00c4c4c400c4c4c4ULL, 0x002f2f2f002f2f2fULL, + 0x00a8a8a800a8a8a8ULL, 0x00b6b6b600b6b6b6ULL, 0x003c3c3c003c3c3cULL, 0x002b2b2b002b2b2bULL, + 0x00c1c1c100c1c1c1ULL, 0x00ffffff00ffffffULL, 0x00c8c8c800c8c8c8ULL, 0x00a5a5a500a5a5a5ULL, + 0x0020202000202020ULL, 0x0089898900898989ULL, 0x0000000000000000ULL, 0x0090909000909090ULL, + 0x0047474700474747ULL, 0x00efefef00efefefULL, 0x00eaeaea00eaeaeaULL, 0x00b7b7b700b7b7b7ULL, + 0x0015151500151515ULL, 0x0006060600060606ULL, 0x00cdcdcd00cdcdcdULL, 0x00b5b5b500b5b5b5ULL, + 0x0012121200121212ULL, 0x007e7e7e007e7e7eULL, 0x00bbbbbb00bbbbbbULL, 0x0029292900292929ULL, + 0x000f0f0f000f0f0fULL, 0x00b8b8b800b8b8b8ULL, 0x0007070700070707ULL, 0x0004040400040404ULL, + 0x009b9b9b009b9b9bULL, 0x0094949400949494ULL, 0x0021212100212121ULL, 0x0066666600666666ULL, + 0x00e6e6e600e6e6e6ULL, 0x00cecece00cececeULL, 0x00ededed00edededULL, 0x00e7e7e700e7e7e7ULL, + 0x003b3b3b003b3b3bULL, 0x00fefefe00fefefeULL, 0x007f7f7f007f7f7fULL, 0x00c5c5c500c5c5c5ULL, + 0x00a4a4a400a4a4a4ULL, 0x0037373700373737ULL, 0x00b1b1b100b1b1b1ULL, 0x004c4c4c004c4c4cULL, + 0x0091919100919191ULL, 0x006e6e6e006e6e6eULL, 0x008d8d8d008d8d8dULL, 0x0076767600767676ULL, + 0x0003030300030303ULL, 0x002d2d2d002d2d2dULL, 0x00dedede00dededeULL, 0x0096969600969696ULL, + 0x0026262600262626ULL, 0x007d7d7d007d7d7dULL, 0x00c6c6c600c6c6c6ULL, 0x005c5c5c005c5c5cULL, + 0x00d3d3d300d3d3d3ULL, 0x00f2f2f200f2f2f2ULL, 0x004f4f4f004f4f4fULL, 0x0019191900191919ULL, + 0x003f3f3f003f3f3fULL, 0x00dcdcdc00dcdcdcULL, 0x0079797900797979ULL, 0x001d1d1d001d1d1dULL, + 0x0052525200525252ULL, 0x00ebebeb00ebebebULL, 0x00f3f3f300f3f3f3ULL, 0x006d6d6d006d6d6dULL, + 0x005e5e5e005e5e5eULL, 0x00fbfbfb00fbfbfbULL, 0x0069696900696969ULL, 0x00b2b2b200b2b2b2ULL, + 0x00f0f0f000f0f0f0ULL, 0x0031313100313131ULL, 0x000c0c0c000c0c0cULL, 0x00d4d4d400d4d4d4ULL, + 0x00cfcfcf00cfcfcfULL, 0x008c8c8c008c8c8cULL, 0x00e2e2e200e2e2e2ULL, 0x0075757500757575ULL, + 0x00a9a9a900a9a9a9ULL, 0x004a4a4a004a4a4aULL, 0x0057575700575757ULL, 0x0084848400848484ULL, + 0x0011111100111111ULL, 0x0045454500454545ULL, 0x001b1b1b001b1b1bULL, 0x00f5f5f500f5f5f5ULL, + 0x00e4e4e400e4e4e4ULL, 0x000e0e0e000e0e0eULL, 0x0073737300737373ULL, 0x00aaaaaa00aaaaaaULL, + 0x00f1f1f100f1f1f1ULL, 0x00dddddd00ddddddULL, 0x0059595900595959ULL, 0x0014141400141414ULL, + 0x006c6c6c006c6c6cULL, 0x0092929200929292ULL, 0x0054545400545454ULL, 0x00d0d0d000d0d0d0ULL, + 0x0078787800787878ULL, 0x0070707000707070ULL, 0x00e3e3e300e3e3e3ULL, 0x0049494900494949ULL, + 0x0080808000808080ULL, 0x0050505000505050ULL, 0x00a7a7a700a7a7a7ULL, 0x00f6f6f600f6f6f6ULL, + 0x0077777700777777ULL, 0x0093939300939393ULL, 0x0086868600868686ULL, 0x0083838300838383ULL, + 0x002a2a2a002a2a2aULL, 0x00c7c7c700c7c7c7ULL, 0x005b5b5b005b5b5bULL, 0x00e9e9e900e9e9e9ULL, + 0x00eeeeee00eeeeeeULL, 0x008f8f8f008f8f8fULL, 0x0001010100010101ULL, 0x003d3d3d003d3d3dULL, +}, +{ + 0x3800383838003838ULL, 0x4100414141004141ULL, 0x1600161616001616ULL, 0x7600767676007676ULL, + 0xd900d9d9d900d9d9ULL, 0x9300939393009393ULL, 0x6000606060006060ULL, 0xf200f2f2f200f2f2ULL, + 0x7200727272007272ULL, 0xc200c2c2c200c2c2ULL, 0xab00ababab00ababULL, 0x9a009a9a9a009a9aULL, + 0x7500757575007575ULL, 0x0600060606000606ULL, 0x5700575757005757ULL, 0xa000a0a0a000a0a0ULL, + 0x9100919191009191ULL, 0xf700f7f7f700f7f7ULL, 0xb500b5b5b500b5b5ULL, 0xc900c9c9c900c9c9ULL, + 0xa200a2a2a200a2a2ULL, 0x8c008c8c8c008c8cULL, 0xd200d2d2d200d2d2ULL, 0x9000909090009090ULL, + 0xf600f6f6f600f6f6ULL, 0x0700070707000707ULL, 0xa700a7a7a700a7a7ULL, 0x2700272727002727ULL, + 0x8e008e8e8e008e8eULL, 0xb200b2b2b200b2b2ULL, 0x4900494949004949ULL, 0xde00dedede00dedeULL, + 0x4300434343004343ULL, 0x5c005c5c5c005c5cULL, 0xd700d7d7d700d7d7ULL, 0xc700c7c7c700c7c7ULL, + 0x3e003e3e3e003e3eULL, 0xf500f5f5f500f5f5ULL, 0x8f008f8f8f008f8fULL, 0x6700676767006767ULL, + 0x1f001f1f1f001f1fULL, 0x1800181818001818ULL, 0x6e006e6e6e006e6eULL, 0xaf00afafaf00afafULL, + 0x2f002f2f2f002f2fULL, 0xe200e2e2e200e2e2ULL, 0x8500858585008585ULL, 0x0d000d0d0d000d0dULL, + 0x5300535353005353ULL, 0xf000f0f0f000f0f0ULL, 0x9c009c9c9c009c9cULL, 0x6500656565006565ULL, + 0xea00eaeaea00eaeaULL, 0xa300a3a3a300a3a3ULL, 0xae00aeaeae00aeaeULL, 0x9e009e9e9e009e9eULL, + 0xec00ececec00ececULL, 0x8000808080008080ULL, 0x2d002d2d2d002d2dULL, 0x6b006b6b6b006b6bULL, + 0xa800a8a8a800a8a8ULL, 0x2b002b2b2b002b2bULL, 0x3600363636003636ULL, 0xa600a6a6a600a6a6ULL, + 0xc500c5c5c500c5c5ULL, 0x8600868686008686ULL, 0x4d004d4d4d004d4dULL, 0x3300333333003333ULL, + 0xfd00fdfdfd00fdfdULL, 0x6600666666006666ULL, 0x5800585858005858ULL, 0x9600969696009696ULL, + 0x3a003a3a3a003a3aULL, 0x0900090909000909ULL, 0x9500959595009595ULL, 0x1000101010001010ULL, + 0x7800787878007878ULL, 0xd800d8d8d800d8d8ULL, 0x4200424242004242ULL, 0xcc00cccccc00ccccULL, + 0xef00efefef00efefULL, 0x2600262626002626ULL, 0xe500e5e5e500e5e5ULL, 0x6100616161006161ULL, + 0x1a001a1a1a001a1aULL, 0x3f003f3f3f003f3fULL, 0x3b003b3b3b003b3bULL, 0x8200828282008282ULL, + 0xb600b6b6b600b6b6ULL, 0xdb00dbdbdb00dbdbULL, 0xd400d4d4d400d4d4ULL, 0x9800989898009898ULL, + 0xe800e8e8e800e8e8ULL, 0x8b008b8b8b008b8bULL, 0x0200020202000202ULL, 0xeb00ebebeb00ebebULL, + 0x0a000a0a0a000a0aULL, 0x2c002c2c2c002c2cULL, 0x1d001d1d1d001d1dULL, 0xb000b0b0b000b0b0ULL, + 0x6f006f6f6f006f6fULL, 0x8d008d8d8d008d8dULL, 0x8800888888008888ULL, 0x0e000e0e0e000e0eULL, + 0x1900191919001919ULL, 0x8700878787008787ULL, 0x4e004e4e4e004e4eULL, 0x0b000b0b0b000b0bULL, + 0xa900a9a9a900a9a9ULL, 0x0c000c0c0c000c0cULL, 0x7900797979007979ULL, 0x1100111111001111ULL, + 0x7f007f7f7f007f7fULL, 0x2200222222002222ULL, 0xe700e7e7e700e7e7ULL, 0x5900595959005959ULL, + 0xe100e1e1e100e1e1ULL, 0xda00dadada00dadaULL, 0x3d003d3d3d003d3dULL, 0xc800c8c8c800c8c8ULL, + 0x1200121212001212ULL, 0x0400040404000404ULL, 0x7400747474007474ULL, 0x5400545454005454ULL, + 0x3000303030003030ULL, 0x7e007e7e7e007e7eULL, 0xb400b4b4b400b4b4ULL, 0x2800282828002828ULL, + 0x5500555555005555ULL, 0x6800686868006868ULL, 0x5000505050005050ULL, 0xbe00bebebe00bebeULL, + 0xd000d0d0d000d0d0ULL, 0xc400c4c4c400c4c4ULL, 0x3100313131003131ULL, 0xcb00cbcbcb00cbcbULL, + 0x2a002a2a2a002a2aULL, 0xad00adadad00adadULL, 0x0f000f0f0f000f0fULL, 0xca00cacaca00cacaULL, + 0x7000707070007070ULL, 0xff00ffffff00ffffULL, 0x3200323232003232ULL, 0x6900696969006969ULL, + 0x0800080808000808ULL, 0x6200626262006262ULL, 0x0000000000000000ULL, 0x2400242424002424ULL, + 0xd100d1d1d100d1d1ULL, 0xfb00fbfbfb00fbfbULL, 0xba00bababa00babaULL, 0xed00ededed00ededULL, + 0x4500454545004545ULL, 0x8100818181008181ULL, 0x7300737373007373ULL, 0x6d006d6d6d006d6dULL, + 0x8400848484008484ULL, 0x9f009f9f9f009f9fULL, 0xee00eeeeee00eeeeULL, 0x4a004a4a4a004a4aULL, + 0xc300c3c3c300c3c3ULL, 0x2e002e2e2e002e2eULL, 0xc100c1c1c100c1c1ULL, 0x0100010101000101ULL, + 0xe600e6e6e600e6e6ULL, 0x2500252525002525ULL, 0x4800484848004848ULL, 0x9900999999009999ULL, + 0xb900b9b9b900b9b9ULL, 0xb300b3b3b300b3b3ULL, 0x7b007b7b7b007b7bULL, 0xf900f9f9f900f9f9ULL, + 0xce00cecece00ceceULL, 0xbf00bfbfbf00bfbfULL, 0xdf00dfdfdf00dfdfULL, 0x7100717171007171ULL, + 0x2900292929002929ULL, 0xcd00cdcdcd00cdcdULL, 0x6c006c6c6c006c6cULL, 0x1300131313001313ULL, + 0x6400646464006464ULL, 0x9b009b9b9b009b9bULL, 0x6300636363006363ULL, 0x9d009d9d9d009d9dULL, + 0xc000c0c0c000c0c0ULL, 0x4b004b4b4b004b4bULL, 0xb700b7b7b700b7b7ULL, 0xa500a5a5a500a5a5ULL, + 0x8900898989008989ULL, 0x5f005f5f5f005f5fULL, 0xb100b1b1b100b1b1ULL, 0x1700171717001717ULL, + 0xf400f4f4f400f4f4ULL, 0xbc00bcbcbc00bcbcULL, 0xd300d3d3d300d3d3ULL, 0x4600464646004646ULL, + 0xcf00cfcfcf00cfcfULL, 0x3700373737003737ULL, 0x5e005e5e5e005e5eULL, 0x4700474747004747ULL, + 0x9400949494009494ULL, 0xfa00fafafa00fafaULL, 0xfc00fcfcfc00fcfcULL, 0x5b005b5b5b005b5bULL, + 0x9700979797009797ULL, 0xfe00fefefe00fefeULL, 0x5a005a5a5a005a5aULL, 0xac00acacac00acacULL, + 0x3c003c3c3c003c3cULL, 0x4c004c4c4c004c4cULL, 0x0300030303000303ULL, 0x3500353535003535ULL, + 0xf300f3f3f300f3f3ULL, 0x2300232323002323ULL, 0xb800b8b8b800b8b8ULL, 0x5d005d5d5d005d5dULL, + 0x6a006a6a6a006a6aULL, 0x9200929292009292ULL, 0xd500d5d5d500d5d5ULL, 0x2100212121002121ULL, + 0x4400444444004444ULL, 0x5100515151005151ULL, 0xc600c6c6c600c6c6ULL, 0x7d007d7d7d007d7dULL, + 0x3900393939003939ULL, 0x8300838383008383ULL, 0xdc00dcdcdc00dcdcULL, 0xaa00aaaaaa00aaaaULL, + 0x7c007c7c7c007c7cULL, 0x7700777777007777ULL, 0x5600565656005656ULL, 0x0500050505000505ULL, + 0x1b001b1b1b001b1bULL, 0xa400a4a4a400a4a4ULL, 0x1500151515001515ULL, 0x3400343434003434ULL, + 0x1e001e1e1e001e1eULL, 0x1c001c1c1c001c1cULL, 0xf800f8f8f800f8f8ULL, 0x5200525252005252ULL, + 0x2000202020002020ULL, 0x1400141414001414ULL, 0xe900e9e9e900e9e9ULL, 0xbd00bdbdbd00bdbdULL, + 0xdd00dddddd00ddddULL, 0xe400e4e4e400e4e4ULL, 0xa100a1a1a100a1a1ULL, 0xe000e0e0e000e0e0ULL, + 0x8a008a8a8a008a8aULL, 0xf100f1f1f100f1f1ULL, 0xd600d6d6d600d6d6ULL, 0x7a007a7a7a007a7aULL, + 0xbb00bbbbbb00bbbbULL, 0xe300e3e3e300e3e3ULL, 0x4000404040004040ULL, 0x4f004f4f4f004f4fULL, +}, +{ + 0x7070007070700070ULL, 0x2c2c002c2c2c002cULL, 0xb3b300b3b3b300b3ULL, 0xc0c000c0c0c000c0ULL, + 0xe4e400e4e4e400e4ULL, 0x5757005757570057ULL, 0xeaea00eaeaea00eaULL, 0xaeae00aeaeae00aeULL, + 0x2323002323230023ULL, 0x6b6b006b6b6b006bULL, 0x4545004545450045ULL, 0xa5a500a5a5a500a5ULL, + 0xeded00ededed00edULL, 0x4f4f004f4f4f004fULL, 0x1d1d001d1d1d001dULL, 0x9292009292920092ULL, + 0x8686008686860086ULL, 0xafaf00afafaf00afULL, 0x7c7c007c7c7c007cULL, 0x1f1f001f1f1f001fULL, + 0x3e3e003e3e3e003eULL, 0xdcdc00dcdcdc00dcULL, 0x5e5e005e5e5e005eULL, 0x0b0b000b0b0b000bULL, + 0xa6a600a6a6a600a6ULL, 0x3939003939390039ULL, 0xd5d500d5d5d500d5ULL, 0x5d5d005d5d5d005dULL, + 0xd9d900d9d9d900d9ULL, 0x5a5a005a5a5a005aULL, 0x5151005151510051ULL, 0x6c6c006c6c6c006cULL, + 0x8b8b008b8b8b008bULL, 0x9a9a009a9a9a009aULL, 0xfbfb00fbfbfb00fbULL, 0xb0b000b0b0b000b0ULL, + 0x7474007474740074ULL, 0x2b2b002b2b2b002bULL, 0xf0f000f0f0f000f0ULL, 0x8484008484840084ULL, + 0xdfdf00dfdfdf00dfULL, 0xcbcb00cbcbcb00cbULL, 0x3434003434340034ULL, 0x7676007676760076ULL, + 0x6d6d006d6d6d006dULL, 0xa9a900a9a9a900a9ULL, 0xd1d100d1d1d100d1ULL, 0x0404000404040004ULL, + 0x1414001414140014ULL, 0x3a3a003a3a3a003aULL, 0xdede00dedede00deULL, 0x1111001111110011ULL, + 0x3232003232320032ULL, 0x9c9c009c9c9c009cULL, 0x5353005353530053ULL, 0xf2f200f2f2f200f2ULL, + 0xfefe00fefefe00feULL, 0xcfcf00cfcfcf00cfULL, 0xc3c300c3c3c300c3ULL, 0x7a7a007a7a7a007aULL, + 0x2424002424240024ULL, 0xe8e800e8e8e800e8ULL, 0x6060006060600060ULL, 0x6969006969690069ULL, + 0xaaaa00aaaaaa00aaULL, 0xa0a000a0a0a000a0ULL, 0xa1a100a1a1a100a1ULL, 0x6262006262620062ULL, + 0x5454005454540054ULL, 0x1e1e001e1e1e001eULL, 0xe0e000e0e0e000e0ULL, 0x6464006464640064ULL, + 0x1010001010100010ULL, 0x0000000000000000ULL, 0xa3a300a3a3a300a3ULL, 0x7575007575750075ULL, + 0x8a8a008a8a8a008aULL, 0xe6e600e6e6e600e6ULL, 0x0909000909090009ULL, 0xdddd00dddddd00ddULL, + 0x8787008787870087ULL, 0x8383008383830083ULL, 0xcdcd00cdcdcd00cdULL, 0x9090009090900090ULL, + 0x7373007373730073ULL, 0xf6f600f6f6f600f6ULL, 0x9d9d009d9d9d009dULL, 0xbfbf00bfbfbf00bfULL, + 0x5252005252520052ULL, 0xd8d800d8d8d800d8ULL, 0xc8c800c8c8c800c8ULL, 0xc6c600c6c6c600c6ULL, + 0x8181008181810081ULL, 0x6f6f006f6f6f006fULL, 0x1313001313130013ULL, 0x6363006363630063ULL, + 0xe9e900e9e9e900e9ULL, 0xa7a700a7a7a700a7ULL, 0x9f9f009f9f9f009fULL, 0xbcbc00bcbcbc00bcULL, + 0x2929002929290029ULL, 0xf9f900f9f9f900f9ULL, 0x2f2f002f2f2f002fULL, 0xb4b400b4b4b400b4ULL, + 0x7878007878780078ULL, 0x0606000606060006ULL, 0xe7e700e7e7e700e7ULL, 0x7171007171710071ULL, + 0xd4d400d4d4d400d4ULL, 0xabab00ababab00abULL, 0x8888008888880088ULL, 0x8d8d008d8d8d008dULL, + 0x7272007272720072ULL, 0xb9b900b9b9b900b9ULL, 0xf8f800f8f8f800f8ULL, 0xacac00acacac00acULL, + 0x3636003636360036ULL, 0x2a2a002a2a2a002aULL, 0x3c3c003c3c3c003cULL, 0xf1f100f1f1f100f1ULL, + 0x4040004040400040ULL, 0xd3d300d3d3d300d3ULL, 0xbbbb00bbbbbb00bbULL, 0x4343004343430043ULL, + 0x1515001515150015ULL, 0xadad00adadad00adULL, 0x7777007777770077ULL, 0x8080008080800080ULL, + 0x8282008282820082ULL, 0xecec00ececec00ecULL, 0x2727002727270027ULL, 0xe5e500e5e5e500e5ULL, + 0x8585008585850085ULL, 0x3535003535350035ULL, 0x0c0c000c0c0c000cULL, 0x4141004141410041ULL, + 0xefef00efefef00efULL, 0x9393009393930093ULL, 0x1919001919190019ULL, 0x2121002121210021ULL, + 0x0e0e000e0e0e000eULL, 0x4e4e004e4e4e004eULL, 0x6565006565650065ULL, 0xbdbd00bdbdbd00bdULL, + 0xb8b800b8b8b800b8ULL, 0x8f8f008f8f8f008fULL, 0xebeb00ebebeb00ebULL, 0xcece00cecece00ceULL, + 0x3030003030300030ULL, 0x5f5f005f5f5f005fULL, 0xc5c500c5c5c500c5ULL, 0x1a1a001a1a1a001aULL, + 0xe1e100e1e1e100e1ULL, 0xcaca00cacaca00caULL, 0x4747004747470047ULL, 0x3d3d003d3d3d003dULL, + 0x0101000101010001ULL, 0xd6d600d6d6d600d6ULL, 0x5656005656560056ULL, 0x4d4d004d4d4d004dULL, + 0x0d0d000d0d0d000dULL, 0x6666006666660066ULL, 0xcccc00cccccc00ccULL, 0x2d2d002d2d2d002dULL, + 0x1212001212120012ULL, 0x2020002020200020ULL, 0xb1b100b1b1b100b1ULL, 0x9999009999990099ULL, + 0x4c4c004c4c4c004cULL, 0xc2c200c2c2c200c2ULL, 0x7e7e007e7e7e007eULL, 0x0505000505050005ULL, + 0xb7b700b7b7b700b7ULL, 0x3131003131310031ULL, 0x1717001717170017ULL, 0xd7d700d7d7d700d7ULL, + 0x5858005858580058ULL, 0x6161006161610061ULL, 0x1b1b001b1b1b001bULL, 0x1c1c001c1c1c001cULL, + 0x0f0f000f0f0f000fULL, 0x1616001616160016ULL, 0x1818001818180018ULL, 0x2222002222220022ULL, + 0x4444004444440044ULL, 0xb2b200b2b2b200b2ULL, 0xb5b500b5b5b500b5ULL, 0x9191009191910091ULL, + 0x0808000808080008ULL, 0xa8a800a8a8a800a8ULL, 0xfcfc00fcfcfc00fcULL, 0x5050005050500050ULL, + 0xd0d000d0d0d000d0ULL, 0x7d7d007d7d7d007dULL, 0x8989008989890089ULL, 0x9797009797970097ULL, + 0x5b5b005b5b5b005bULL, 0x9595009595950095ULL, 0xffff00ffffff00ffULL, 0xd2d200d2d2d200d2ULL, + 0xc4c400c4c4c400c4ULL, 0x4848004848480048ULL, 0xf7f700f7f7f700f7ULL, 0xdbdb00dbdbdb00dbULL, + 0x0303000303030003ULL, 0xdada00dadada00daULL, 0x3f3f003f3f3f003fULL, 0x9494009494940094ULL, + 0x5c5c005c5c5c005cULL, 0x0202000202020002ULL, 0x4a4a004a4a4a004aULL, 0x3333003333330033ULL, + 0x6767006767670067ULL, 0xf3f300f3f3f300f3ULL, 0x7f7f007f7f7f007fULL, 0xe2e200e2e2e200e2ULL, + 0x9b9b009b9b9b009bULL, 0x2626002626260026ULL, 0x3737003737370037ULL, 0x3b3b003b3b3b003bULL, + 0x9696009696960096ULL, 0x4b4b004b4b4b004bULL, 0xbebe00bebebe00beULL, 0x2e2e002e2e2e002eULL, + 0x7979007979790079ULL, 0x8c8c008c8c8c008cULL, 0x6e6e006e6e6e006eULL, 0x8e8e008e8e8e008eULL, + 0xf5f500f5f5f500f5ULL, 0xb6b600b6b6b600b6ULL, 0xfdfd00fdfdfd00fdULL, 0x5959005959590059ULL, + 0x9898009898980098ULL, 0x6a6a006a6a6a006aULL, 0x4646004646460046ULL, 0xbaba00bababa00baULL, + 0x2525002525250025ULL, 0x4242004242420042ULL, 0xa2a200a2a2a200a2ULL, 0xfafa00fafafa00faULL, + 0x0707000707070007ULL, 0x5555005555550055ULL, 0xeeee00eeeeee00eeULL, 0x0a0a000a0a0a000aULL, + 0x4949004949490049ULL, 0x6868006868680068ULL, 0x3838003838380038ULL, 0xa4a400a4a4a400a4ULL, + 0x2828002828280028ULL, 0x7b7b007b7b7b007bULL, 0xc9c900c9c9c900c9ULL, 0xc1c100c1c1c100c1ULL, + 0xe3e300e3e3e300e3ULL, 0xf4f400f4f4f400f4ULL, 0xc7c700c7c7c700c7ULL, 0x9e9e009e9e9e009eULL, +}, +{ + 0x7070700070707000ULL, 0x8282820082828200ULL, 0x2c2c2c002c2c2c00ULL, 0xececec00ececec00ULL, + 0xb3b3b300b3b3b300ULL, 0x2727270027272700ULL, 0xc0c0c000c0c0c000ULL, 0xe5e5e500e5e5e500ULL, + 0xe4e4e400e4e4e400ULL, 0x8585850085858500ULL, 0x5757570057575700ULL, 0x3535350035353500ULL, + 0xeaeaea00eaeaea00ULL, 0x0c0c0c000c0c0c00ULL, 0xaeaeae00aeaeae00ULL, 0x4141410041414100ULL, + 0x2323230023232300ULL, 0xefefef00efefef00ULL, 0x6b6b6b006b6b6b00ULL, 0x9393930093939300ULL, + 0x4545450045454500ULL, 0x1919190019191900ULL, 0xa5a5a500a5a5a500ULL, 0x2121210021212100ULL, + 0xededed00ededed00ULL, 0x0e0e0e000e0e0e00ULL, 0x4f4f4f004f4f4f00ULL, 0x4e4e4e004e4e4e00ULL, + 0x1d1d1d001d1d1d00ULL, 0x6565650065656500ULL, 0x9292920092929200ULL, 0xbdbdbd00bdbdbd00ULL, + 0x8686860086868600ULL, 0xb8b8b800b8b8b800ULL, 0xafafaf00afafaf00ULL, 0x8f8f8f008f8f8f00ULL, + 0x7c7c7c007c7c7c00ULL, 0xebebeb00ebebeb00ULL, 0x1f1f1f001f1f1f00ULL, 0xcecece00cecece00ULL, + 0x3e3e3e003e3e3e00ULL, 0x3030300030303000ULL, 0xdcdcdc00dcdcdc00ULL, 0x5f5f5f005f5f5f00ULL, + 0x5e5e5e005e5e5e00ULL, 0xc5c5c500c5c5c500ULL, 0x0b0b0b000b0b0b00ULL, 0x1a1a1a001a1a1a00ULL, + 0xa6a6a600a6a6a600ULL, 0xe1e1e100e1e1e100ULL, 0x3939390039393900ULL, 0xcacaca00cacaca00ULL, + 0xd5d5d500d5d5d500ULL, 0x4747470047474700ULL, 0x5d5d5d005d5d5d00ULL, 0x3d3d3d003d3d3d00ULL, + 0xd9d9d900d9d9d900ULL, 0x0101010001010100ULL, 0x5a5a5a005a5a5a00ULL, 0xd6d6d600d6d6d600ULL, + 0x5151510051515100ULL, 0x5656560056565600ULL, 0x6c6c6c006c6c6c00ULL, 0x4d4d4d004d4d4d00ULL, + 0x8b8b8b008b8b8b00ULL, 0x0d0d0d000d0d0d00ULL, 0x9a9a9a009a9a9a00ULL, 0x6666660066666600ULL, + 0xfbfbfb00fbfbfb00ULL, 0xcccccc00cccccc00ULL, 0xb0b0b000b0b0b000ULL, 0x2d2d2d002d2d2d00ULL, + 0x7474740074747400ULL, 0x1212120012121200ULL, 0x2b2b2b002b2b2b00ULL, 0x2020200020202000ULL, + 0xf0f0f000f0f0f000ULL, 0xb1b1b100b1b1b100ULL, 0x8484840084848400ULL, 0x9999990099999900ULL, + 0xdfdfdf00dfdfdf00ULL, 0x4c4c4c004c4c4c00ULL, 0xcbcbcb00cbcbcb00ULL, 0xc2c2c200c2c2c200ULL, + 0x3434340034343400ULL, 0x7e7e7e007e7e7e00ULL, 0x7676760076767600ULL, 0x0505050005050500ULL, + 0x6d6d6d006d6d6d00ULL, 0xb7b7b700b7b7b700ULL, 0xa9a9a900a9a9a900ULL, 0x3131310031313100ULL, + 0xd1d1d100d1d1d100ULL, 0x1717170017171700ULL, 0x0404040004040400ULL, 0xd7d7d700d7d7d700ULL, + 0x1414140014141400ULL, 0x5858580058585800ULL, 0x3a3a3a003a3a3a00ULL, 0x6161610061616100ULL, + 0xdedede00dedede00ULL, 0x1b1b1b001b1b1b00ULL, 0x1111110011111100ULL, 0x1c1c1c001c1c1c00ULL, + 0x3232320032323200ULL, 0x0f0f0f000f0f0f00ULL, 0x9c9c9c009c9c9c00ULL, 0x1616160016161600ULL, + 0x5353530053535300ULL, 0x1818180018181800ULL, 0xf2f2f200f2f2f200ULL, 0x2222220022222200ULL, + 0xfefefe00fefefe00ULL, 0x4444440044444400ULL, 0xcfcfcf00cfcfcf00ULL, 0xb2b2b200b2b2b200ULL, + 0xc3c3c300c3c3c300ULL, 0xb5b5b500b5b5b500ULL, 0x7a7a7a007a7a7a00ULL, 0x9191910091919100ULL, + 0x2424240024242400ULL, 0x0808080008080800ULL, 0xe8e8e800e8e8e800ULL, 0xa8a8a800a8a8a800ULL, + 0x6060600060606000ULL, 0xfcfcfc00fcfcfc00ULL, 0x6969690069696900ULL, 0x5050500050505000ULL, + 0xaaaaaa00aaaaaa00ULL, 0xd0d0d000d0d0d000ULL, 0xa0a0a000a0a0a000ULL, 0x7d7d7d007d7d7d00ULL, + 0xa1a1a100a1a1a100ULL, 0x8989890089898900ULL, 0x6262620062626200ULL, 0x9797970097979700ULL, + 0x5454540054545400ULL, 0x5b5b5b005b5b5b00ULL, 0x1e1e1e001e1e1e00ULL, 0x9595950095959500ULL, + 0xe0e0e000e0e0e000ULL, 0xffffff00ffffff00ULL, 0x6464640064646400ULL, 0xd2d2d200d2d2d200ULL, + 0x1010100010101000ULL, 0xc4c4c400c4c4c400ULL, 0x0000000000000000ULL, 0x4848480048484800ULL, + 0xa3a3a300a3a3a300ULL, 0xf7f7f700f7f7f700ULL, 0x7575750075757500ULL, 0xdbdbdb00dbdbdb00ULL, + 0x8a8a8a008a8a8a00ULL, 0x0303030003030300ULL, 0xe6e6e600e6e6e600ULL, 0xdadada00dadada00ULL, + 0x0909090009090900ULL, 0x3f3f3f003f3f3f00ULL, 0xdddddd00dddddd00ULL, 0x9494940094949400ULL, + 0x8787870087878700ULL, 0x5c5c5c005c5c5c00ULL, 0x8383830083838300ULL, 0x0202020002020200ULL, + 0xcdcdcd00cdcdcd00ULL, 0x4a4a4a004a4a4a00ULL, 0x9090900090909000ULL, 0x3333330033333300ULL, + 0x7373730073737300ULL, 0x6767670067676700ULL, 0xf6f6f600f6f6f600ULL, 0xf3f3f300f3f3f300ULL, + 0x9d9d9d009d9d9d00ULL, 0x7f7f7f007f7f7f00ULL, 0xbfbfbf00bfbfbf00ULL, 0xe2e2e200e2e2e200ULL, + 0x5252520052525200ULL, 0x9b9b9b009b9b9b00ULL, 0xd8d8d800d8d8d800ULL, 0x2626260026262600ULL, + 0xc8c8c800c8c8c800ULL, 0x3737370037373700ULL, 0xc6c6c600c6c6c600ULL, 0x3b3b3b003b3b3b00ULL, + 0x8181810081818100ULL, 0x9696960096969600ULL, 0x6f6f6f006f6f6f00ULL, 0x4b4b4b004b4b4b00ULL, + 0x1313130013131300ULL, 0xbebebe00bebebe00ULL, 0x6363630063636300ULL, 0x2e2e2e002e2e2e00ULL, + 0xe9e9e900e9e9e900ULL, 0x7979790079797900ULL, 0xa7a7a700a7a7a700ULL, 0x8c8c8c008c8c8c00ULL, + 0x9f9f9f009f9f9f00ULL, 0x6e6e6e006e6e6e00ULL, 0xbcbcbc00bcbcbc00ULL, 0x8e8e8e008e8e8e00ULL, + 0x2929290029292900ULL, 0xf5f5f500f5f5f500ULL, 0xf9f9f900f9f9f900ULL, 0xb6b6b600b6b6b600ULL, + 0x2f2f2f002f2f2f00ULL, 0xfdfdfd00fdfdfd00ULL, 0xb4b4b400b4b4b400ULL, 0x5959590059595900ULL, + 0x7878780078787800ULL, 0x9898980098989800ULL, 0x0606060006060600ULL, 0x6a6a6a006a6a6a00ULL, + 0xe7e7e700e7e7e700ULL, 0x4646460046464600ULL, 0x7171710071717100ULL, 0xbababa00bababa00ULL, + 0xd4d4d400d4d4d400ULL, 0x2525250025252500ULL, 0xababab00ababab00ULL, 0x4242420042424200ULL, + 0x8888880088888800ULL, 0xa2a2a200a2a2a200ULL, 0x8d8d8d008d8d8d00ULL, 0xfafafa00fafafa00ULL, + 0x7272720072727200ULL, 0x0707070007070700ULL, 0xb9b9b900b9b9b900ULL, 0x5555550055555500ULL, + 0xf8f8f800f8f8f800ULL, 0xeeeeee00eeeeee00ULL, 0xacacac00acacac00ULL, 0x0a0a0a000a0a0a00ULL, + 0x3636360036363600ULL, 0x4949490049494900ULL, 0x2a2a2a002a2a2a00ULL, 0x6868680068686800ULL, + 0x3c3c3c003c3c3c00ULL, 0x3838380038383800ULL, 0xf1f1f100f1f1f100ULL, 0xa4a4a400a4a4a400ULL, + 0x4040400040404000ULL, 0x2828280028282800ULL, 0xd3d3d300d3d3d300ULL, 0x7b7b7b007b7b7b00ULL, + 0xbbbbbb00bbbbbb00ULL, 0xc9c9c900c9c9c900ULL, 0x4343430043434300ULL, 0xc1c1c100c1c1c100ULL, + 0x1515150015151500ULL, 0xe3e3e300e3e3e300ULL, 0xadadad00adadad00ULL, 0xf4f4f400f4f4f400ULL, + 0x7777770077777700ULL, 0xc7c7c700c7c7c700ULL, 0x8080800080808000ULL, 0x9e9e9e009e9e9e00ULL, +}, +}; + +static const uint64_t SIGMA[6] = { + 0xA09E667F3BCC908BULL, 0xB67AE8584CAA73B2ULL, 0xC6EF372FE94F82BEULL, + 0x54FF53A5F1D36F1CULL, 0x10E527FADE682D1DULL, 0xB05688C2B3E6C1FDULL +}; + +static inline uint64_t load_be64(const uint8_t *p) +{ + return ((uint64_t) p[0] << 56) | ((uint64_t) p[1] << 48) + | ((uint64_t) p[2] << 40) | ((uint64_t) p[3] << 32) + | ((uint64_t) p[4] << 24) | ((uint64_t) p[5] << 16) + | ((uint64_t) p[6] << 8) | ((uint64_t) p[7]); +} + +static inline void store_be64(uint8_t *p, uint64_t v) +{ + p[0] = (uint8_t) (v >> 56); p[1] = (uint8_t) (v >> 48); + p[2] = (uint8_t) (v >> 40); p[3] = (uint8_t) (v >> 32); + p[4] = (uint8_t) (v >> 24); p[5] = (uint8_t) (v >> 16); + p[6] = (uint8_t) (v >> 8); p[7] = (uint8_t) v; +} + +static inline uint64_t camellia_f(uint64_t fin, uint64_t ke) +{ + uint64_t x = fin ^ ke; + return SP[0][(x >> 56) & 0xff] ^ SP[1][(x >> 48) & 0xff] + ^ SP[2][(x >> 40) & 0xff] ^ SP[3][(x >> 32) & 0xff] + ^ SP[4][(x >> 24) & 0xff] ^ SP[5][(x >> 16) & 0xff] + ^ SP[6][(x >> 8) & 0xff] ^ SP[7][ x & 0xff]; +} + +static inline uint32_t rotl32(uint32_t v, int n) +{ + return (v << n) | (v >> (32 - n)); +} + +static inline uint64_t camellia_fl(uint64_t fin, uint64_t ke) +{ + uint32_t x1 = (uint32_t) (fin >> 32), x2 = (uint32_t) fin; + uint32_t k1 = (uint32_t) (ke >> 32), k2 = (uint32_t) ke; + + x2 ^= rotl32(x1 & k1, 1); + x1 ^= (x2 | k2); + return ((uint64_t) x1 << 32) | x2; +} + +static inline uint64_t camellia_flinv(uint64_t fin, uint64_t ke) +{ + uint32_t y1 = (uint32_t) (fin >> 32), y2 = (uint32_t) fin; + uint32_t k1 = (uint32_t) (ke >> 32), k2 = (uint32_t) ke; + + y1 ^= (y2 | k2); + y2 ^= rotl32(y1 & k1, 1); + return ((uint64_t) y1 << 32) | y2; +} + +/* the halves of a 128-bit value rotated left by n, 0 < n < 128 */ +static void rotl128(uint64_t hi, uint64_t lo, int n, uint64_t *rhi, uint64_t *rlo) +{ + if (n >= 64) { + uint64_t t = hi; + hi = lo; + lo = t; + n -= 64; + } + if (n == 0) { + *rhi = hi; + *rlo = lo; + } else { + *rhi = (hi << n) | (lo >> (64 - n)); + *rlo = (lo << n) | (hi >> (64 - n)); + } +} + +void crypton_camellia_init(crypton_camellia_key *ks, const uint8_t *key) +{ + uint64_t klhi = load_be64(key), kllo = load_be64(key + 8); + uint64_t d1 = klhi, d2 = kllo, kahi, kalo, hi, lo; + + d2 ^= camellia_f(d1, SIGMA[0]); + d1 ^= camellia_f(d2, SIGMA[1]); + d1 ^= klhi; + d2 ^= kllo; + d2 ^= camellia_f(d1, SIGMA[2]); + d1 ^= camellia_f(d2, SIGMA[3]); + kahi = d1; + kalo = d2; + + ks->kw[0] = klhi; + ks->kw[1] = kllo; + ks->k[0] = kahi; + ks->k[1] = kalo; + rotl128(klhi, kllo, 15, &hi, &lo); ks->k[2] = hi; ks->k[3] = lo; + rotl128(kahi, kalo, 15, &hi, &lo); ks->k[4] = hi; ks->k[5] = lo; + rotl128(kahi, kalo, 30, &hi, &lo); ks->ke[0] = hi; ks->ke[1] = lo; + rotl128(klhi, kllo, 45, &hi, &lo); ks->k[6] = hi; ks->k[7] = lo; + rotl128(kahi, kalo, 45, &hi, &lo); ks->k[8] = hi; + rotl128(klhi, kllo, 60, &hi, &lo); ks->k[9] = lo; + rotl128(kahi, kalo, 60, &hi, &lo); ks->k[10] = hi; ks->k[11] = lo; + rotl128(klhi, kllo, 77, &hi, &lo); ks->ke[2] = hi; ks->ke[3] = lo; + rotl128(klhi, kllo, 94, &hi, &lo); ks->k[12] = hi; ks->k[13] = lo; + rotl128(kahi, kalo, 94, &hi, &lo); ks->k[14] = hi; ks->k[15] = lo; + rotl128(klhi, kllo, 111, &hi, &lo); ks->k[16] = hi; ks->k[17] = lo; + rotl128(kahi, kalo, 111, &hi, &lo); ks->kw[2] = hi; ks->kw[3] = lo; +} + +static void camellia_crypt(uint8_t *out, const uint64_t kw[4], const uint64_t k[18], + const uint64_t ke[4], const uint8_t *in, uint32_t nblocks) +{ + uint32_t i; + + for (i = 0; i < nblocks; i++) { + uint64_t d1 = load_be64(in + 16 * i) ^ kw[0]; + uint64_t d2 = load_be64(in + 16 * i + 8) ^ kw[1]; + int base; + + for (base = 0; base <= 12; base += 6) { + d2 ^= camellia_f(d1, k[base + 0]); + d1 ^= camellia_f(d2, k[base + 1]); + d2 ^= camellia_f(d1, k[base + 2]); + d1 ^= camellia_f(d2, k[base + 3]); + d2 ^= camellia_f(d1, k[base + 4]); + d1 ^= camellia_f(d2, k[base + 5]); + if (base == 0) { + d1 = camellia_fl(d1, ke[0]); + d2 = camellia_flinv(d2, ke[1]); + } else if (base == 6) { + d1 = camellia_fl(d1, ke[2]); + d2 = camellia_flinv(d2, ke[3]); + } + } + + store_be64(out + 16 * i, d2 ^ kw[2]); + store_be64(out + 16 * i + 8, d1 ^ kw[3]); + } +} + +void crypton_camellia_encrypt(uint8_t *out, const crypton_camellia_key *ks, + const uint8_t *in, uint32_t nblocks) +{ + camellia_crypt(out, ks->kw, ks->k, ks->ke, in, nblocks); +} + +/* Decryption is the same rounds with the subkeys the other way round. */ +void crypton_camellia_decrypt(uint8_t *out, const crypton_camellia_key *ks, + const uint8_t *in, uint32_t nblocks) +{ + uint64_t kw[4], k[18], ke[4]; + int i; + + kw[0] = ks->kw[2]; kw[1] = ks->kw[3]; kw[2] = ks->kw[0]; kw[3] = ks->kw[1]; + for (i = 0; i < 18; i++) + k[i] = ks->k[17 - i]; + for (i = 0; i < 4; i++) + ke[i] = ks->ke[3 - i]; + camellia_crypt(out, kw, k, ke, in, nblocks); +} diff --git a/cbits/crypton_camellia.h b/cbits/crypton_camellia.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_camellia.h @@ -0,0 +1,21 @@ +#ifndef CRYPTON_CAMELLIA_H +#define CRYPTON_CAMELLIA_H + +#include + +/* the subkeys of RFC 3713 section 2.2, for a 128-bit key */ +typedef struct { + uint64_t kw[4]; + uint64_t k[18]; + uint64_t ke[4]; +} crypton_camellia_key; + +void crypton_camellia_init(crypton_camellia_key *ks, const uint8_t *key); + +void crypton_camellia_encrypt(uint8_t *out, const crypton_camellia_key *ks, + const uint8_t *in, uint32_t nblocks); + +void crypton_camellia_decrypt(uint8_t *out, const crypton_camellia_key *ks, + const uint8_t *in, uint32_t nblocks); + +#endif diff --git a/cbits/crypton_chacha.c b/cbits/crypton_chacha.c --- a/cbits/crypton_chacha.c +++ b/cbits/crypton_chacha.c @@ -35,6 +35,71 @@ #include "crypton_align.h" #include +/* + * Four blocks at a time with whichever vector unit the target has: NEON in + * chacha_neon.c, SSE2 in chacha_sse2.c. Both present the same two entry + * points, so there is one path here. + * + * The state words are held little-endian -- the core below reads them + * without converting -- so the vector versions, which also do not convert, + * are left out on a big-endian machine. + */ +#if (defined(WITH_ARMV8_NEON) && !defined(__AARCH64EB__)) || defined(WITH_X86_SSE2) +#define CHACHA_SIMD 1 +int crypton_chacha_simd_width(void); +void crypton_chacha_simd_combine(int rounds, uint8_t *dst, const uint8_t *src, + const crypton_chacha_state *in); +void crypton_chacha_simd_generate(int rounds, uint8_t *dst, + const crypton_chacha_state *in); +/* The counters in a group must not carry into d[13], which the block loop + * below handles and the vector one does not; that is one run in 2^29. */ +#define CHACHA_SIMD_OK(st, n) ((st)->d[12] <= 0xffffffffU - (uint32_t) (n)) +#endif + +/* + * ChaCha20 from CRYPTOGAMS, in cbits/asm/chacha-armv8-*.S. It keeps four + * vector blocks and a fifth in the general registers in flight at once, or + * six and two above 512 bytes, which is more than the intrinsics above can + * be made to do: the vector registers hold four states and there is no room + * for another, so the extra parallelism has to come from the integer side, + * and that means saying which register holds what. + * + * Twenty rounds and the 256-bit constants are built into it, and it takes + * the counter as 32 bits wide, so it is given only the states it fits. + */ +#if (defined(WITH_ARMV8_CHACHA_ASM) && !defined(__AARCH64EB__)) \ + || defined(WITH_X86_CHACHA_ASM) +#define CHACHA_ASM 1 +#include "crypton_cpu.h" +void crypton_chacha20_asm_ctr32(uint8_t *out, const uint8_t *in, size_t len, + const uint32_t key[8], const uint32_t counter[4]); + +/* crypton_cpu.c defines the crypton_armcap_P that the assembly reads to + * find out whether the processor has NEON. */ + +/* The four words at the head of the state are the constants that go with a + * 256-bit key, and the assembly has only those. */ +static int chacha_asm_state(const crypton_chacha_state *st) +{ + return st->d[0] == 0x61707865 && st->d[1] == 0x3320646e + && st->d[2] == 0x79622d32 && st->d[3] == 0x6b206574; +} + +/* + * How much is worth handing over. On AArch64 the module's vector path + * starts at three blocks and below that its scalar path measures level with + * the C here, so there is nothing to gain; on x86-64 it is ahead from one + * block, the C there having no vector path until eight. + */ +#ifndef CHACHA_ASM_MIN_BLOCKS +#ifdef WITH_X86_CHACHA_ASM +#define CHACHA_ASM_MIN_BLOCKS 1 +#else +#define CHACHA_ASM_MIN_BLOCKS 3 +#endif +#endif +#endif + #define QR(a,b,c,d) \ a += b; d = rol32(d ^ a,16); \ c += d; b = rol32(b ^ c,12); \ @@ -256,6 +321,43 @@ st = &ctx->st; +#ifdef CHACHA_ASM + if (ctx->nb_rounds == 20 && chacha_asm_state(st)) { + uint32_t blocks = bytes / 64; + + /* the counter is the caller's to advance, and the assembly + * carries it no further than its own 32 bits */ + if (blocks > 0xffffffffU - st->d[12]) + blocks = 0xffffffffU - st->d[12]; + if (blocks >= CHACHA_ASM_MIN_BLOCKS) { + const uint32_t done = blocks * 64; + +#ifdef CRYPTON_X86_ASM + /* what the module dispatches on, which it reads + * directly; resolved once */ + crypton_x86_ia32cap_resolve(); +#endif + crypton_chacha20_asm_ctr32(dst, src, done, &st->d[4], + &st->d[12]); + st->d[12] += blocks; + bytes -= done; src += done; dst += done; + } + } +#endif + +#ifdef CHACHA_SIMD + { + const uint32_t nb = (uint32_t) crypton_chacha_simd_width(); + const uint32_t step = 64 * nb; + + while (bytes >= step && CHACHA_SIMD_OK(st, nb)) { + crypton_chacha_simd_combine(ctx->nb_rounds, dst, src, st); + st->d[12] += nb; + bytes -= step; src += step; dst += step; + } + } +#endif + /* xor new 64-bytes chunks and store the left over if any */ for (; bytes >= 64; bytes -= 64, src += 64, dst += 64) { /* generate new chunk and update state */ @@ -354,6 +456,19 @@ return; st = &ctx->st; + +#ifdef CHACHA_SIMD + { + const uint32_t nb = (uint32_t) crypton_chacha_simd_width(); + const uint32_t step = 64 * nb; + + while (bytes >= step && CHACHA_SIMD_OK(st, nb)) { + crypton_chacha_simd_generate(ctx->nb_rounds, dst, st); + st->d[12] += nb; + bytes -= step; dst += step; + } + } +#endif if (ALIGNED64(dst)) { /* xor new 64-bytes chunks and store the left over if any */ diff --git a/cbits/crypton_cpu.c b/cbits/crypton_cpu.c --- a/cbits/crypton_cpu.c +++ b/cbits/crypton_cpu.c @@ -31,6 +31,18 @@ #include "crypton_cpu.h" #include +/* + * The word the assembly reads; crypton_cpu.h says what is in it. Hidden, + * so that the reference to it from the assembly resolves at link time in a + * shared object as well as a static one. The SHA-256 bit is set by + * cbits/crypton_sha256.c once it has asked whether the processor has those + * instructions. + */ +#ifdef CRYPTON_ARM_ASM +__attribute__((visibility("hidden"))) unsigned int crypton_armcap_P = + CRYPTON_ARMCAP_NEON; +#endif + #ifdef ARCH_X86 static void cpuid(uint32_t info, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) { @@ -51,6 +63,157 @@ #endif :"+a" (*eax), "=S" (*ebx), "=c" (*ecx), "=d" (*edx) : :"edi"); +} + +/* + * What the machine will let us use beyond the x86-64 baseline. XGETBV is + * spelled out in bytes because it predates some assemblers that are still + * in use. + */ +static void cpuid_count(uint32_t info, uint32_t sub, uint32_t *eax, uint32_t *ebx, uint32_t *ecx, uint32_t *edx) +{ + *eax = info; + *ecx = sub; + __asm__ volatile + ( +#ifdef __x86_64__ + "mov %%rbx, %%rdi;" +#else + "mov %%ebx, %%edi;" +#endif + "cpuid;" + "mov %%ebx, %%esi;" +#ifdef __x86_64__ + "mov %%rdi, %%rbx;" +#else + "mov %%edi, %%ebx;" +#endif + :"+a" (*eax), "=S" (*ebx), "+c" (*ecx), "=d" (*edx) + : :"edi"); +} + +static uint64_t xcr0(void) +{ + uint32_t lo, hi; + + __asm__ volatile(".byte 0x0f, 0x01, 0xd0" : "=a" (lo), "=d" (hi) : "c" (0)); + return ((uint64_t) hi << 32) | lo; +} + +#ifdef CRYPTON_X86_ASM +__attribute__((visibility("hidden"))) unsigned int crypton_ia32cap_P[4]; + +/* + * The AVX-512 bits of leaf 7 EBX -- F, DQ, IFMA, PF, ER, CD, BW and VL, + * which is every bit from 16 up except 21's neighbours and 29, the SHA + * extensions, which are not AVX-512 and are wanted. They are cleared + * whatever the processor says: the code they would select in the vendored + * assembly cannot be run, let alone measured, on any machine here, and + * shipping a path nothing has executed is not worth the few per cent it + * might be worth. Turning them on is a one-line change for whoever has + * the hardware. + */ +#define IA32CAP_AVX512 \ + ((1u << 16) | (1u << 17) | (1u << 21) | (1u << 26) | (1u << 27) \ + | (1u << 28) | (1u << 30) | (1u << 31)) + +/* + * cpuid as the assembly reads it, with the two bits it dispatches on -- AVX + * in leaf 1 and AVX2 in leaf 7 -- left set only where the answer already + * agreed that the operating system saves the registers. Two threads racing + * here write the same values. + */ +void crypton_x86_ia32cap_resolve(void) +{ + static int resolved = 0; + + if (!resolved) { + uint32_t eax, ebx, ecx, edx, maxleaf; + uint32_t f = crypton_x86_simd_features(); + uint32_t leaf1_ecx, leaf7_ebx = 0; + int intel; + + cpuid(0, &eax, &ebx, &ecx, &edx); + maxleaf = eax; + /* "GenuineIntel", which OpenSSL records in a bit of leaf 1 + * EDX that cpuid leaves reserved: some of the assembly asks, + * having found a path worth taking on one make and not the + * other */ + intel = (ebx == 0x756e6547 && edx == 0x49656e69 + && ecx == 0x6c65746e); + + cpuid(1, &eax, &ebx, &ecx, &edx); + crypton_ia32cap_P[0] = intel ? (edx | (1u << 30)) : edx; + leaf1_ecx = ecx; + if (!(f & CRYPTON_X86_AVX)) + leaf1_ecx &= ~(1u << 28); + crypton_ia32cap_P[1] = leaf1_ecx; + + if (maxleaf >= 7) { + cpuid_count(7, 0, &eax, &ebx, &ecx, &edx); + leaf7_ebx = ebx; + } + if (!(f & CRYPTON_X86_AVX2)) + leaf7_ebx &= ~(1u << 5); + crypton_ia32cap_P[2] = leaf7_ebx & ~IA32CAP_AVX512; + + resolved = 1; + } +} +#endif + +uint32_t crypton_x86_simd_features(void) +{ + static int resolved = 0; + static uint32_t features = 0; + + if (!resolved) { + uint32_t eax, ebx, ecx, edx, leaf1, maxleaf, f = 0; + + cpuid(0, &eax, &ebx, &ecx, &edx); + maxleaf = eax; + + cpuid(1, &eax, &ebx, &ecx, &edx); + leaf1 = ecx; + if (leaf1 & (1 << 9)) + f |= CRYPTON_X86_SSSE3; + if (leaf1 & (1 << 1)) + f |= CRYPTON_X86_PCLMUL; + if (leaf1 & (1 << 22)) + f |= CRYPTON_X86_MOVBE; + /* AVX asks the same three things as AVX2 below: the + * processor has it, OSXSAVE is on, and the operating system + * says it saves the registers */ + if ((leaf1 & (1 << 28)) && (leaf1 & (1 << 27)) + && ((xcr0() & 6) == 6)) + f |= CRYPTON_X86_AVX; + + /* leaf 7 answers for both of the rest, and a processor that + * does not have it answers for the highest leaf it does have + * instead, so ask what that is first */ + if (maxleaf >= 7) { + cpuid_count(7, 0, &eax, &ebx, &ecx, &edx); + /* the SHA extensions work in registers the SSE state + * already covers, so they need nothing of the + * operating system. The code that uses them also + * wants SSSE3 and SSE4.1, which every processor that + * has them has, but ask rather than assume */ + if ((ebx & (1 << 29)) && (leaf1 & (1 << 9)) + && (leaf1 & (1 << 19))) + f |= CRYPTON_X86_SHA_NI; + /* AVX2 has the wider registers, which takes three + * things agreeing: the CPU has it, OSXSAVE is on, and + * XCR0 says the operating system saves them -- + * without that last one the upper halves are lost + * across a context switch */ + if ((ebx & (1 << 5)) && (leaf1 & (1 << 27)) + && (leaf1 & (1 << 28)) && ((xcr0() & 6) == 6)) + f |= CRYPTON_X86_AVX2; + } + features = f; + resolved = 1; + } + return features; } #ifdef USE_AESNI diff --git a/cbits/crypton_cpu.h b/cbits/crypton_cpu.h --- a/cbits/crypton_cpu.h +++ b/cbits/crypton_cpu.h @@ -31,9 +31,55 @@ #ifndef CPU_H #define CPU_H +#include + #if defined(__i386__) || defined(__x86_64__) #define ARCH_X86 #define USE_AESNI +#endif + +/* vector extensions beyond the x86-64 baseline, as cpuid reports them and + * the OS allows them */ +#define CRYPTON_X86_SSSE3 1 +#define CRYPTON_X86_AVX2 2 +#define CRYPTON_X86_PCLMUL 4 +/* the SHA extensions, and the SSSE3 and SSE4.1 the code around them uses */ +#define CRYPTON_X86_SHA_NI 8 +/* the 128-bit half of AVX, which is what the vendored assembly is written + * in, and the byte-swapping load it reads the message with */ +#define CRYPTON_X86_AVX 16 +#define CRYPTON_X86_MOVBE 32 +#ifdef ARCH_X86 +uint32_t crypton_x86_simd_features(void); +#endif + +/* + * What the vendored AArch64 assembly asks about the processor, in the way + * OpenSSL asks it and with OpenSSL's bit numbering. NEON is not optional + * on AArch64 and is set from the start; the SHA-256 instructions are, so + * the bit for them is set once the runtime check has answered. See + * cbits/crypton_cpu.c and cbits/asm/README.md. + */ +/* + * And what the vendored x86-64 assembly asks, which is cpuid's own words in + * the order OpenSSL keeps them: [0] is leaf 1 EDX, [1] leaf 1 ECX and [2] + * leaf 7 EBX, with the bits for what the operating system will not preserve + * cleared. Filled on first use; see cbits/crypton_cpu.c. + */ +#if defined(WITH_X86_POLY1305_ASM) || defined(WITH_X86_CHACHA_ASM) \ + || defined(WITH_X86_SHA256_ASM) || defined(WITH_X86_SHA512_ASM) +#define CRYPTON_X86_ASM 1 +extern unsigned int crypton_ia32cap_P[4]; +void crypton_x86_ia32cap_resolve(void); +#endif + +#if defined(WITH_ARMV8_CHACHA_ASM) || defined(WITH_ARMV8_POLY1305_ASM) \ + || defined(WITH_ARMV8_SHA1_ASM) || defined(WITH_ARMV8_SHA256_ASM) +#define CRYPTON_ARM_ASM 1 +#define CRYPTON_ARMCAP_NEON 1 +#define CRYPTON_ARMCAP_SHA1 (1 << 3) +#define CRYPTON_ARMCAP_SHA256 (1 << 4) +extern unsigned int crypton_armcap_P; #endif #ifdef USE_AESNI diff --git a/cbits/crypton_des.c b/cbits/crypton_des.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_des.c @@ -0,0 +1,1325 @@ +/* + * DES, as FIPS 46-3 defines it. + * + * The tables below are generated from the permutations and S-boxes of that + * standard: SP[i] combines S-box i with the P permutation, IPL/IPR and FPH/FPL + * apply the initial and final permutations one input byte at a time, and the + * 48-bit round key is kept as eight six-bit values so that the E expansion is + * a rotate and a shift rather than a table. + * + * DES is here because callers still meet it, not because it should be chosen: + * its 56-bit key is exhaustible, and this implementation indexes tables with + * key-dependent values, so it is not constant time. + */ +#include +#include +#include + +static const uint32_t SP[8][64] = { +{ + 0x00808200U, 0x00000000U, 0x00008000U, 0x00808202U, 0x00808002U, 0x00008202U, 0x00000002U, 0x00008000U, + 0x00000200U, 0x00808200U, 0x00808202U, 0x00000200U, 0x00800202U, 0x00808002U, 0x00800000U, 0x00000002U, + 0x00000202U, 0x00800200U, 0x00800200U, 0x00008200U, 0x00008200U, 0x00808000U, 0x00808000U, 0x00800202U, + 0x00008002U, 0x00800002U, 0x00800002U, 0x00008002U, 0x00000000U, 0x00000202U, 0x00008202U, 0x00800000U, + 0x00008000U, 0x00808202U, 0x00000002U, 0x00808000U, 0x00808200U, 0x00800000U, 0x00800000U, 0x00000200U, + 0x00808002U, 0x00008000U, 0x00008200U, 0x00800002U, 0x00000200U, 0x00000002U, 0x00800202U, 0x00008202U, + 0x00808202U, 0x00008002U, 0x00808000U, 0x00800202U, 0x00800002U, 0x00000202U, 0x00008202U, 0x00808200U, + 0x00000202U, 0x00800200U, 0x00800200U, 0x00000000U, 0x00008002U, 0x00008200U, 0x00000000U, 0x00808002U, +}, +{ + 0x40084010U, 0x40004000U, 0x00004000U, 0x00084010U, 0x00080000U, 0x00000010U, 0x40080010U, 0x40004010U, + 0x40000010U, 0x40084010U, 0x40084000U, 0x40000000U, 0x40004000U, 0x00080000U, 0x00000010U, 0x40080010U, + 0x00084000U, 0x00080010U, 0x40004010U, 0x00000000U, 0x40000000U, 0x00004000U, 0x00084010U, 0x40080000U, + 0x00080010U, 0x40000010U, 0x00000000U, 0x00084000U, 0x00004010U, 0x40084000U, 0x40080000U, 0x00004010U, + 0x00000000U, 0x00084010U, 0x40080010U, 0x00080000U, 0x40004010U, 0x40080000U, 0x40084000U, 0x00004000U, + 0x40080000U, 0x40004000U, 0x00000010U, 0x40084010U, 0x00084010U, 0x00000010U, 0x00004000U, 0x40000000U, + 0x00004010U, 0x40084000U, 0x00080000U, 0x40000010U, 0x00080010U, 0x40004010U, 0x40000010U, 0x00080010U, + 0x00084000U, 0x00000000U, 0x40004000U, 0x00004010U, 0x40000000U, 0x40080010U, 0x40084010U, 0x00084000U, +}, +{ + 0x00000104U, 0x04010100U, 0x00000000U, 0x04010004U, 0x04000100U, 0x00000000U, 0x00010104U, 0x04000100U, + 0x00010004U, 0x04000004U, 0x04000004U, 0x00010000U, 0x04010104U, 0x00010004U, 0x04010000U, 0x00000104U, + 0x04000000U, 0x00000004U, 0x04010100U, 0x00000100U, 0x00010100U, 0x04010000U, 0x04010004U, 0x00010104U, + 0x04000104U, 0x00010100U, 0x00010000U, 0x04000104U, 0x00000004U, 0x04010104U, 0x00000100U, 0x04000000U, + 0x04010100U, 0x04000000U, 0x00010004U, 0x00000104U, 0x00010000U, 0x04010100U, 0x04000100U, 0x00000000U, + 0x00000100U, 0x00010004U, 0x04010104U, 0x04000100U, 0x04000004U, 0x00000100U, 0x00000000U, 0x04010004U, + 0x04000104U, 0x00010000U, 0x04000000U, 0x04010104U, 0x00000004U, 0x00010104U, 0x00010100U, 0x04000004U, + 0x04010000U, 0x04000104U, 0x00000104U, 0x04010000U, 0x00010104U, 0x00000004U, 0x04010004U, 0x00010100U, +}, +{ + 0x80401000U, 0x80001040U, 0x80001040U, 0x00000040U, 0x00401040U, 0x80400040U, 0x80400000U, 0x80001000U, + 0x00000000U, 0x00401000U, 0x00401000U, 0x80401040U, 0x80000040U, 0x00000000U, 0x00400040U, 0x80400000U, + 0x80000000U, 0x00001000U, 0x00400000U, 0x80401000U, 0x00000040U, 0x00400000U, 0x80001000U, 0x00001040U, + 0x80400040U, 0x80000000U, 0x00001040U, 0x00400040U, 0x00001000U, 0x00401040U, 0x80401040U, 0x80000040U, + 0x00400040U, 0x80400000U, 0x00401000U, 0x80401040U, 0x80000040U, 0x00000000U, 0x00000000U, 0x00401000U, + 0x00001040U, 0x00400040U, 0x80400040U, 0x80000000U, 0x80401000U, 0x80001040U, 0x80001040U, 0x00000040U, + 0x80401040U, 0x80000040U, 0x80000000U, 0x00001000U, 0x80400000U, 0x80001000U, 0x00401040U, 0x80400040U, + 0x80001000U, 0x00001040U, 0x00400000U, 0x80401000U, 0x00000040U, 0x00400000U, 0x00001000U, 0x00401040U, +}, +{ + 0x00000080U, 0x01040080U, 0x01040000U, 0x21000080U, 0x00040000U, 0x00000080U, 0x20000000U, 0x01040000U, + 0x20040080U, 0x00040000U, 0x01000080U, 0x20040080U, 0x21000080U, 0x21040000U, 0x00040080U, 0x20000000U, + 0x01000000U, 0x20040000U, 0x20040000U, 0x00000000U, 0x20000080U, 0x21040080U, 0x21040080U, 0x01000080U, + 0x21040000U, 0x20000080U, 0x00000000U, 0x21000000U, 0x01040080U, 0x01000000U, 0x21000000U, 0x00040080U, + 0x00040000U, 0x21000080U, 0x00000080U, 0x01000000U, 0x20000000U, 0x01040000U, 0x21000080U, 0x20040080U, + 0x01000080U, 0x20000000U, 0x21040000U, 0x01040080U, 0x20040080U, 0x00000080U, 0x01000000U, 0x21040000U, + 0x21040080U, 0x00040080U, 0x21000000U, 0x21040080U, 0x01040000U, 0x00000000U, 0x20040000U, 0x21000000U, + 0x00040080U, 0x01000080U, 0x20000080U, 0x00040000U, 0x00000000U, 0x20040000U, 0x01040080U, 0x20000080U, +}, +{ + 0x10000008U, 0x10200000U, 0x00002000U, 0x10202008U, 0x10200000U, 0x00000008U, 0x10202008U, 0x00200000U, + 0x10002000U, 0x00202008U, 0x00200000U, 0x10000008U, 0x00200008U, 0x10002000U, 0x10000000U, 0x00002008U, + 0x00000000U, 0x00200008U, 0x10002008U, 0x00002000U, 0x00202000U, 0x10002008U, 0x00000008U, 0x10200008U, + 0x10200008U, 0x00000000U, 0x00202008U, 0x10202000U, 0x00002008U, 0x00202000U, 0x10202000U, 0x10000000U, + 0x10002000U, 0x00000008U, 0x10200008U, 0x00202000U, 0x10202008U, 0x00200000U, 0x00002008U, 0x10000008U, + 0x00200000U, 0x10002000U, 0x10000000U, 0x00002008U, 0x10000008U, 0x10202008U, 0x00202000U, 0x10200000U, + 0x00202008U, 0x10202000U, 0x00000000U, 0x10200008U, 0x00000008U, 0x00002000U, 0x10200000U, 0x00202008U, + 0x00002000U, 0x00200008U, 0x10002008U, 0x00000000U, 0x10202000U, 0x10000000U, 0x00200008U, 0x10002008U, +}, +{ + 0x00100000U, 0x02100001U, 0x02000401U, 0x00000000U, 0x00000400U, 0x02000401U, 0x00100401U, 0x02100400U, + 0x02100401U, 0x00100000U, 0x00000000U, 0x02000001U, 0x00000001U, 0x02000000U, 0x02100001U, 0x00000401U, + 0x02000400U, 0x00100401U, 0x00100001U, 0x02000400U, 0x02000001U, 0x02100000U, 0x02100400U, 0x00100001U, + 0x02100000U, 0x00000400U, 0x00000401U, 0x02100401U, 0x00100400U, 0x00000001U, 0x02000000U, 0x00100400U, + 0x02000000U, 0x00100400U, 0x00100000U, 0x02000401U, 0x02000401U, 0x02100001U, 0x02100001U, 0x00000001U, + 0x00100001U, 0x02000000U, 0x02000400U, 0x00100000U, 0x02100400U, 0x00000401U, 0x00100401U, 0x02100400U, + 0x00000401U, 0x02000001U, 0x02100401U, 0x02100000U, 0x00100400U, 0x00000000U, 0x00000001U, 0x02100401U, + 0x00000000U, 0x00100401U, 0x02100000U, 0x00000400U, 0x02000001U, 0x02000400U, 0x00000400U, 0x00100001U, +}, +{ + 0x08000820U, 0x00000800U, 0x00020000U, 0x08020820U, 0x08000000U, 0x08000820U, 0x00000020U, 0x08000000U, + 0x00020020U, 0x08020000U, 0x08020820U, 0x00020800U, 0x08020800U, 0x00020820U, 0x00000800U, 0x00000020U, + 0x08020000U, 0x08000020U, 0x08000800U, 0x00000820U, 0x00020800U, 0x00020020U, 0x08020020U, 0x08020800U, + 0x00000820U, 0x00000000U, 0x00000000U, 0x08020020U, 0x08000020U, 0x08000800U, 0x00020820U, 0x00020000U, + 0x00020820U, 0x00020000U, 0x08020800U, 0x00000800U, 0x00000020U, 0x08020020U, 0x00000800U, 0x00020820U, + 0x08000800U, 0x00000020U, 0x08000020U, 0x08020000U, 0x08020020U, 0x08000000U, 0x00020000U, 0x08000820U, + 0x00000000U, 0x08020820U, 0x00020020U, 0x08000020U, 0x08020000U, 0x08000800U, 0x08000820U, 0x00000000U, + 0x08020820U, 0x00020800U, 0x00020800U, 0x00000820U, 0x00000820U, 0x00020020U, 0x08000000U, 0x08020800U, +}, +}; + +static const uint32_t IPL[8][256] = { +{ + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00000000U, 0x00000001U, 0x00000000U, 0x00000001U, 0x00000100U, 0x00000101U, 0x00000100U, 0x00000101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x00010000U, 0x00010001U, 0x00010000U, 0x00010001U, 0x00010100U, 0x00010101U, 0x00010100U, 0x00010101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01000000U, 0x01000001U, 0x01000000U, 0x01000001U, 0x01000100U, 0x01000101U, 0x01000100U, 0x01000101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, + 0x01010000U, 0x01010001U, 0x01010000U, 0x01010001U, 0x01010100U, 0x01010101U, 0x01010100U, 0x01010101U, +}, +{ + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00000000U, 0x00000002U, 0x00000000U, 0x00000002U, 0x00000200U, 0x00000202U, 0x00000200U, 0x00000202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x00020000U, 0x00020002U, 0x00020000U, 0x00020002U, 0x00020200U, 0x00020202U, 0x00020200U, 0x00020202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02000000U, 0x02000002U, 0x02000000U, 0x02000002U, 0x02000200U, 0x02000202U, 0x02000200U, 0x02000202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, + 0x02020000U, 0x02020002U, 0x02020000U, 0x02020002U, 0x02020200U, 0x02020202U, 0x02020200U, 0x02020202U, +}, +{ + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00000000U, 0x00000004U, 0x00000000U, 0x00000004U, 0x00000400U, 0x00000404U, 0x00000400U, 0x00000404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x00040000U, 0x00040004U, 0x00040000U, 0x00040004U, 0x00040400U, 0x00040404U, 0x00040400U, 0x00040404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04000000U, 0x04000004U, 0x04000000U, 0x04000004U, 0x04000400U, 0x04000404U, 0x04000400U, 0x04000404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, + 0x04040000U, 0x04040004U, 0x04040000U, 0x04040004U, 0x04040400U, 0x04040404U, 0x04040400U, 0x04040404U, +}, +{ + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00000000U, 0x00000008U, 0x00000000U, 0x00000008U, 0x00000800U, 0x00000808U, 0x00000800U, 0x00000808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x00080000U, 0x00080008U, 0x00080000U, 0x00080008U, 0x00080800U, 0x00080808U, 0x00080800U, 0x00080808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08000000U, 0x08000008U, 0x08000000U, 0x08000008U, 0x08000800U, 0x08000808U, 0x08000800U, 0x08000808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, + 0x08080000U, 0x08080008U, 0x08080000U, 0x08080008U, 0x08080800U, 0x08080808U, 0x08080800U, 0x08080808U, +}, +{ + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00000000U, 0x00000010U, 0x00000000U, 0x00000010U, 0x00001000U, 0x00001010U, 0x00001000U, 0x00001010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x00100000U, 0x00100010U, 0x00100000U, 0x00100010U, 0x00101000U, 0x00101010U, 0x00101000U, 0x00101010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10000000U, 0x10000010U, 0x10000000U, 0x10000010U, 0x10001000U, 0x10001010U, 0x10001000U, 0x10001010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, + 0x10100000U, 0x10100010U, 0x10100000U, 0x10100010U, 0x10101000U, 0x10101010U, 0x10101000U, 0x10101010U, +}, +{ + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00000000U, 0x00000020U, 0x00000000U, 0x00000020U, 0x00002000U, 0x00002020U, 0x00002000U, 0x00002020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x00200000U, 0x00200020U, 0x00200000U, 0x00200020U, 0x00202000U, 0x00202020U, 0x00202000U, 0x00202020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20000000U, 0x20000020U, 0x20000000U, 0x20000020U, 0x20002000U, 0x20002020U, 0x20002000U, 0x20002020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, + 0x20200000U, 0x20200020U, 0x20200000U, 0x20200020U, 0x20202000U, 0x20202020U, 0x20202000U, 0x20202020U, +}, +{ + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00000000U, 0x00000040U, 0x00000000U, 0x00000040U, 0x00004000U, 0x00004040U, 0x00004000U, 0x00004040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x00400000U, 0x00400040U, 0x00400000U, 0x00400040U, 0x00404000U, 0x00404040U, 0x00404000U, 0x00404040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40000000U, 0x40000040U, 0x40000000U, 0x40000040U, 0x40004000U, 0x40004040U, 0x40004000U, 0x40004040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, + 0x40400000U, 0x40400040U, 0x40400000U, 0x40400040U, 0x40404000U, 0x40404040U, 0x40404000U, 0x40404040U, +}, +{ + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00000000U, 0x00000080U, 0x00000000U, 0x00000080U, 0x00008000U, 0x00008080U, 0x00008000U, 0x00008080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x00800000U, 0x00800080U, 0x00800000U, 0x00800080U, 0x00808000U, 0x00808080U, 0x00808000U, 0x00808080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80000000U, 0x80000080U, 0x80000000U, 0x80000080U, 0x80008000U, 0x80008080U, 0x80008000U, 0x80008080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, + 0x80800000U, 0x80800080U, 0x80800000U, 0x80800080U, 0x80808000U, 0x80808080U, 0x80808000U, 0x80808080U, +}, +}; + +static const uint32_t IPR[8][256] = { +{ + 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, + 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, + 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, + 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, + 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, + 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, + 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, + 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, + 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, + 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, + 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, 0x00000000U, 0x00000000U, 0x00000001U, 0x00000001U, + 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, 0x00000100U, 0x00000100U, 0x00000101U, 0x00000101U, + 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, + 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, + 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, 0x00010000U, 0x00010000U, 0x00010001U, 0x00010001U, + 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, 0x00010100U, 0x00010100U, 0x00010101U, 0x00010101U, + 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, + 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, + 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, + 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, + 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, + 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, + 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, + 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, + 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, + 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, + 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, 0x01000000U, 0x01000000U, 0x01000001U, 0x01000001U, + 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, 0x01000100U, 0x01000100U, 0x01000101U, 0x01000101U, + 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, + 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, + 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, 0x01010000U, 0x01010000U, 0x01010001U, 0x01010001U, + 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, 0x01010100U, 0x01010100U, 0x01010101U, 0x01010101U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, + 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, + 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, + 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, + 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, + 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, + 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, + 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, + 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, + 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, + 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, 0x00000000U, 0x00000000U, 0x00000002U, 0x00000002U, + 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, 0x00000200U, 0x00000200U, 0x00000202U, 0x00000202U, + 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, + 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, + 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, 0x00020000U, 0x00020000U, 0x00020002U, 0x00020002U, + 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, 0x00020200U, 0x00020200U, 0x00020202U, 0x00020202U, + 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, + 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, + 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, + 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, + 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, + 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, + 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, + 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, + 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, + 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, + 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, 0x02000000U, 0x02000000U, 0x02000002U, 0x02000002U, + 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, 0x02000200U, 0x02000200U, 0x02000202U, 0x02000202U, + 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, + 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, + 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, 0x02020000U, 0x02020000U, 0x02020002U, 0x02020002U, + 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, 0x02020200U, 0x02020200U, 0x02020202U, 0x02020202U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, + 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, + 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, + 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, + 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, + 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, + 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, + 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, + 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, + 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, + 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, 0x00000000U, 0x00000000U, 0x00000004U, 0x00000004U, + 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, 0x00000400U, 0x00000400U, 0x00000404U, 0x00000404U, + 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, + 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, + 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, 0x00040000U, 0x00040000U, 0x00040004U, 0x00040004U, + 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, 0x00040400U, 0x00040400U, 0x00040404U, 0x00040404U, + 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, + 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, + 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, + 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, + 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, + 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, + 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, + 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, + 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, + 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, + 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, 0x04000000U, 0x04000000U, 0x04000004U, 0x04000004U, + 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, 0x04000400U, 0x04000400U, 0x04000404U, 0x04000404U, + 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, + 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, + 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, 0x04040000U, 0x04040000U, 0x04040004U, 0x04040004U, + 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, 0x04040400U, 0x04040400U, 0x04040404U, 0x04040404U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, + 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, + 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, + 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, + 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, + 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, + 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, + 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, + 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, + 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, + 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, 0x00000000U, 0x00000000U, 0x00000008U, 0x00000008U, + 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, 0x00000800U, 0x00000800U, 0x00000808U, 0x00000808U, + 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, + 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, + 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, 0x00080000U, 0x00080000U, 0x00080008U, 0x00080008U, + 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, 0x00080800U, 0x00080800U, 0x00080808U, 0x00080808U, + 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, + 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, + 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, + 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, + 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, + 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, + 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, + 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, + 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, + 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, + 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, 0x08000000U, 0x08000000U, 0x08000008U, 0x08000008U, + 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, 0x08000800U, 0x08000800U, 0x08000808U, 0x08000808U, + 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, + 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, + 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, 0x08080000U, 0x08080000U, 0x08080008U, 0x08080008U, + 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, 0x08080800U, 0x08080800U, 0x08080808U, 0x08080808U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, + 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, + 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, + 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, + 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, + 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, + 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, + 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, + 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, + 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, + 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, 0x00000000U, 0x00000000U, 0x00000010U, 0x00000010U, + 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, 0x00001000U, 0x00001000U, 0x00001010U, 0x00001010U, + 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, + 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, + 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, 0x00100000U, 0x00100000U, 0x00100010U, 0x00100010U, + 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, 0x00101000U, 0x00101000U, 0x00101010U, 0x00101010U, + 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, + 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, + 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, + 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, + 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, + 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, + 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, + 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, + 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, + 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, + 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, 0x10000000U, 0x10000000U, 0x10000010U, 0x10000010U, + 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, 0x10001000U, 0x10001000U, 0x10001010U, 0x10001010U, + 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, + 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, + 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, 0x10100000U, 0x10100000U, 0x10100010U, 0x10100010U, + 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, 0x10101000U, 0x10101000U, 0x10101010U, 0x10101010U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, + 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, + 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, + 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, + 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, + 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, + 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, + 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, + 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, + 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, + 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, 0x00000000U, 0x00000000U, 0x00000020U, 0x00000020U, + 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, 0x00002000U, 0x00002000U, 0x00002020U, 0x00002020U, + 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, + 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, + 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, 0x00200000U, 0x00200000U, 0x00200020U, 0x00200020U, + 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, 0x00202000U, 0x00202000U, 0x00202020U, 0x00202020U, + 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, + 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, + 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, + 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, + 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, + 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, + 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, + 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, + 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, + 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, + 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, 0x20000000U, 0x20000000U, 0x20000020U, 0x20000020U, + 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, 0x20002000U, 0x20002000U, 0x20002020U, 0x20002020U, + 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, + 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, + 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, 0x20200000U, 0x20200000U, 0x20200020U, 0x20200020U, + 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, 0x20202000U, 0x20202000U, 0x20202020U, 0x20202020U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, + 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, + 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, + 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, + 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, + 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, + 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, + 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, + 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, + 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, + 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, 0x00000000U, 0x00000000U, 0x00000040U, 0x00000040U, + 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, 0x00004000U, 0x00004000U, 0x00004040U, 0x00004040U, + 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, + 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, + 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, 0x00400000U, 0x00400000U, 0x00400040U, 0x00400040U, + 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, 0x00404000U, 0x00404000U, 0x00404040U, 0x00404040U, + 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, + 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, + 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, + 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, + 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, + 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, + 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, + 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, + 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, + 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, + 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, 0x40000000U, 0x40000000U, 0x40000040U, 0x40000040U, + 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, 0x40004000U, 0x40004000U, 0x40004040U, 0x40004040U, + 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, + 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, + 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, 0x40400000U, 0x40400000U, 0x40400040U, 0x40400040U, + 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, 0x40404000U, 0x40404000U, 0x40404040U, 0x40404040U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, + 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, + 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, + 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, + 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, + 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, + 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, + 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, + 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, + 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, + 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, 0x00000000U, 0x00000000U, 0x00000080U, 0x00000080U, + 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, 0x00008000U, 0x00008000U, 0x00008080U, 0x00008080U, + 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, + 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, + 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, 0x00800000U, 0x00800000U, 0x00800080U, 0x00800080U, + 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, 0x00808000U, 0x00808000U, 0x00808080U, 0x00808080U, + 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, + 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, + 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, + 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, + 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, + 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, + 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, + 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, + 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, + 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, + 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, 0x80000000U, 0x80000000U, 0x80000080U, 0x80000080U, + 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, 0x80008000U, 0x80008000U, 0x80008080U, 0x80008080U, + 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, + 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, + 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, 0x80800000U, 0x80800000U, 0x80800080U, 0x80800080U, + 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, 0x80808000U, 0x80808000U, 0x80808080U, 0x80808080U, +}, +}; + +static const uint32_t FPH[8][256] = { +{ + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, + 0x00000000U, 0x40000000U, 0x00400000U, 0x40400000U, 0x00004000U, 0x40004000U, 0x00404000U, 0x40404000U, + 0x00000040U, 0x40000040U, 0x00400040U, 0x40400040U, 0x00004040U, 0x40004040U, 0x00404040U, 0x40404040U, +}, +{ + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, + 0x00000000U, 0x10000000U, 0x00100000U, 0x10100000U, 0x00001000U, 0x10001000U, 0x00101000U, 0x10101000U, + 0x00000010U, 0x10000010U, 0x00100010U, 0x10100010U, 0x00001010U, 0x10001010U, 0x00101010U, 0x10101010U, +}, +{ + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, + 0x00000000U, 0x04000000U, 0x00040000U, 0x04040000U, 0x00000400U, 0x04000400U, 0x00040400U, 0x04040400U, + 0x00000004U, 0x04000004U, 0x00040004U, 0x04040004U, 0x00000404U, 0x04000404U, 0x00040404U, 0x04040404U, +}, +{ + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, + 0x00000000U, 0x01000000U, 0x00010000U, 0x01010000U, 0x00000100U, 0x01000100U, 0x00010100U, 0x01010100U, + 0x00000001U, 0x01000001U, 0x00010001U, 0x01010001U, 0x00000101U, 0x01000101U, 0x00010101U, 0x01010101U, +}, +{ + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, + 0x00000000U, 0x80000000U, 0x00800000U, 0x80800000U, 0x00008000U, 0x80008000U, 0x00808000U, 0x80808000U, + 0x00000080U, 0x80000080U, 0x00800080U, 0x80800080U, 0x00008080U, 0x80008080U, 0x00808080U, 0x80808080U, +}, +{ + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, + 0x00000000U, 0x20000000U, 0x00200000U, 0x20200000U, 0x00002000U, 0x20002000U, 0x00202000U, 0x20202000U, + 0x00000020U, 0x20000020U, 0x00200020U, 0x20200020U, 0x00002020U, 0x20002020U, 0x00202020U, 0x20202020U, +}, +{ + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, + 0x00000000U, 0x08000000U, 0x00080000U, 0x08080000U, 0x00000800U, 0x08000800U, 0x00080800U, 0x08080800U, + 0x00000008U, 0x08000008U, 0x00080008U, 0x08080008U, 0x00000808U, 0x08000808U, 0x00080808U, 0x08080808U, +}, +{ + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, + 0x00000000U, 0x02000000U, 0x00020000U, 0x02020000U, 0x00000200U, 0x02000200U, 0x00020200U, 0x02020200U, + 0x00000002U, 0x02000002U, 0x00020002U, 0x02020002U, 0x00000202U, 0x02000202U, 0x00020202U, 0x02020202U, +}, +}; + +static const uint32_t FPL[8][256] = { +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, + 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, 0x40000000U, + 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, + 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, 0x00400000U, + 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, + 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, 0x40400000U, + 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, + 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, 0x00004000U, + 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, + 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, 0x40004000U, + 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, + 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, 0x00404000U, + 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, + 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, 0x40404000U, + 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, + 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, 0x00000040U, + 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, + 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, 0x40000040U, + 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, + 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, 0x00400040U, + 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, + 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, 0x40400040U, + 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, + 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, 0x00004040U, + 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, + 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, 0x40004040U, + 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, + 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, 0x00404040U, + 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, + 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, 0x40404040U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, + 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, 0x10000000U, + 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, + 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, 0x00100000U, + 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, + 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, 0x10100000U, + 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, + 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, 0x00001000U, + 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, + 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, 0x10001000U, + 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, + 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, 0x00101000U, + 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, + 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, 0x10101000U, + 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, + 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, 0x00000010U, + 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, + 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, 0x10000010U, + 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, + 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, 0x00100010U, + 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, + 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, 0x10100010U, + 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, + 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, 0x00001010U, + 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, + 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, 0x10001010U, + 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, + 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, 0x00101010U, + 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, + 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, 0x10101010U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, + 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, 0x04000000U, + 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, + 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, 0x00040000U, + 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, + 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, 0x04040000U, + 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, + 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, 0x00000400U, + 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, + 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, 0x04000400U, + 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, + 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, 0x00040400U, + 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, + 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, 0x04040400U, + 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, + 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, 0x00000004U, + 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, + 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, 0x04000004U, + 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, + 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, 0x00040004U, + 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, + 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, 0x04040004U, + 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, + 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, 0x00000404U, + 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, + 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, 0x04000404U, + 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, + 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, 0x00040404U, + 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, + 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, 0x04040404U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, + 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, 0x01000000U, + 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, + 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, 0x00010000U, + 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, + 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, 0x01010000U, + 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, + 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, 0x00000100U, + 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, + 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, 0x01000100U, + 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, + 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, 0x00010100U, + 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, + 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, 0x01010100U, + 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, + 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, 0x00000001U, + 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, + 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, 0x01000001U, + 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, + 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, 0x00010001U, + 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, + 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, 0x01010001U, + 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, + 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, 0x00000101U, + 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, + 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, 0x01000101U, + 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, + 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, 0x00010101U, + 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, + 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, 0x01010101U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, + 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, 0x80000000U, + 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, + 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, 0x00800000U, + 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, + 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, 0x80800000U, + 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, + 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, 0x00008000U, + 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, + 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, 0x80008000U, + 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, + 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, 0x00808000U, + 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, + 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, 0x80808000U, + 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, + 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, 0x00000080U, + 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, + 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, 0x80000080U, + 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, + 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, 0x00800080U, + 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, + 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, 0x80800080U, + 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, + 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, 0x00008080U, + 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, + 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, 0x80008080U, + 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, + 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, 0x00808080U, + 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, + 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, 0x80808080U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, + 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, 0x20000000U, + 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, + 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, 0x00200000U, + 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, + 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, 0x20200000U, + 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, + 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, 0x00002000U, + 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, + 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, 0x20002000U, + 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, + 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, 0x00202000U, + 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, + 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, 0x20202000U, + 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, + 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, 0x00000020U, + 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, + 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, 0x20000020U, + 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, + 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, 0x00200020U, + 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, + 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, 0x20200020U, + 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, + 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, 0x00002020U, + 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, + 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, 0x20002020U, + 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, + 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, 0x00202020U, + 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, + 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, 0x20202020U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, + 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, 0x08000000U, + 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, + 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, 0x00080000U, + 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, + 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, 0x08080000U, + 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, + 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, 0x00000800U, + 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, + 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, 0x08000800U, + 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, + 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, 0x00080800U, + 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, + 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, 0x08080800U, + 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, + 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, 0x00000008U, + 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, + 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, 0x08000008U, + 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, + 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, 0x00080008U, + 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, + 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, 0x08080008U, + 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, + 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, 0x00000808U, + 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, + 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, 0x08000808U, + 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, + 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, 0x00080808U, + 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, + 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, 0x08080808U, +}, +{ + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, 0x00000000U, + 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, + 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, 0x02000000U, + 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, + 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, 0x00020000U, + 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, + 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, 0x02020000U, + 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, + 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, 0x00000200U, + 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, + 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, 0x02000200U, + 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, + 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, 0x00020200U, + 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, + 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, 0x02020200U, + 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, + 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, 0x00000002U, + 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, + 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, 0x02000002U, + 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, + 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, 0x00020002U, + 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, + 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, 0x02020002U, + 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, + 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, 0x00000202U, + 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, + 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, 0x02000202U, + 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, + 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, 0x00020202U, + 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, + 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, 0x02020202U, +}, +}; + +#define ROTL32(v, k) (((v) << (k)) | ((v) >> (32 - (k)))) + +static inline uint64_t load_be64(const uint8_t *p) +{ + return ((uint64_t) p[0] << 56) | ((uint64_t) p[1] << 48) + | ((uint64_t) p[2] << 40) | ((uint64_t) p[3] << 32) + | ((uint64_t) p[4] << 24) | ((uint64_t) p[5] << 16) + | ((uint64_t) p[6] << 8) | ((uint64_t) p[7]); +} + +static inline void store_be64(uint8_t *p, uint64_t v) +{ + p[0] = (uint8_t) (v >> 56); p[1] = (uint8_t) (v >> 48); + p[2] = (uint8_t) (v >> 40); p[3] = (uint8_t) (v >> 32); + p[4] = (uint8_t) (v >> 24); p[5] = (uint8_t) (v >> 16); + p[6] = (uint8_t) (v >> 8); p[7] = (uint8_t) v; +} + +/* bit i of v, numbered from 1 at the most significant end, as FIPS 46-3 does */ +static inline uint32_t bit_of(uint64_t v, int i, int width) +{ + return (uint32_t) ((v >> (width - i)) & 1); +} + +static const uint8_t PC1[56] = { + 57,49,41,33,25,17,9,1,58,50,42,34,26,18,10,2,59,51,43,35,27, + 19,11,3,60,52,44,36,63,55,47,39,31,23,15,7,62,54,46,38,30,22, + 14,6,61,53,45,37,29,21,13,5,28,20,12,4 +}; + +static const uint8_t PC2[48] = { + 14,17,11,24,1,5,3,28,15,6,21,10,23,19,12,4,26,8,16,7,27,20,13,2, + 41,52,31,37,47,55,30,40,51,45,33,48,44,49,39,56,34,53,46,42,50,36,29,32 +}; + +static const uint8_t SHIFTS[16] = { 1,1,2,2,2,2,2,2,1,2,2,2,2,2,2,1 }; + +void crypton_des_init(crypton_des_key *ks, const uint8_t *key, int reverse) +{ + uint64_t k = load_be64(key); + uint64_t cd = 0; + uint32_t c, d; + int round, i; + + for (i = 0; i < 56; i++) + cd = (cd << 1) | bit_of(k, PC1[i], 64); + c = (uint32_t) (cd >> 28); + d = (uint32_t) (cd & 0x0fffffffU); + + for (round = 0; round < 16; round++) { + uint64_t merged; + uint8_t *sk; + int s = SHIFTS[round]; + + c = ((c << s) | (c >> (28 - s))) & 0x0fffffffU; + d = ((d << s) | (d >> (28 - s))) & 0x0fffffffU; + merged = ((uint64_t) c << 28) | d; + + sk = ks->sk + (reverse ? (15 - round) : round) * 8; + memset(sk, 0, 8); + for (i = 0; i < 48; i++) + sk[i / 6] = (uint8_t) ((sk[i / 6] << 1) | bit_of(merged, PC2[i], 56)); + } +} + +static inline uint32_t des_f(uint32_t r, const uint8_t *sk) +{ + return SP[0][((ROTL32(r, 31) >> 26) & 0x3f) ^ sk[0]] + | SP[1][((ROTL32(r, 3) >> 26) & 0x3f) ^ sk[1]] + | SP[2][((ROTL32(r, 7) >> 26) & 0x3f) ^ sk[2]] + | SP[3][((ROTL32(r, 11) >> 26) & 0x3f) ^ sk[3]] + | SP[4][((ROTL32(r, 15) >> 26) & 0x3f) ^ sk[4]] + | SP[5][((ROTL32(r, 19) >> 26) & 0x3f) ^ sk[5]] + | SP[6][((ROTL32(r, 23) >> 26) & 0x3f) ^ sk[6]] + | SP[7][((ROTL32(r, 27) >> 26) & 0x3f) ^ sk[7]]; +} + +static inline uint64_t des_block(uint64_t b, const uint8_t *sk) +{ + uint32_t l, r; + int round; + + l = IPL[0][(b >> 56) & 0xff] | IPL[1][(b >> 48) & 0xff] + | IPL[2][(b >> 40) & 0xff] | IPL[3][(b >> 32) & 0xff] + | IPL[4][(b >> 24) & 0xff] | IPL[5][(b >> 16) & 0xff] + | IPL[6][(b >> 8) & 0xff] | IPL[7][ b & 0xff]; + r = IPR[0][(b >> 56) & 0xff] | IPR[1][(b >> 48) & 0xff] + | IPR[2][(b >> 40) & 0xff] | IPR[3][(b >> 32) & 0xff] + | IPR[4][(b >> 24) & 0xff] | IPR[5][(b >> 16) & 0xff] + | IPR[6][(b >> 8) & 0xff] | IPR[7][ b & 0xff]; + + for (round = 0; round < 16; round++) { + uint32_t t = r; + r = l ^ des_f(r, sk + round * 8); + l = t; + } + + { + /* the preoutput is the halves the other way round */ + uint64_t p = ((uint64_t) r << 32) | l; + return ((uint64_t) (FPH[0][(p >> 56) & 0xff] | FPH[1][(p >> 48) & 0xff] + | FPH[2][(p >> 40) & 0xff] | FPH[3][(p >> 32) & 0xff] + | FPH[4][(p >> 24) & 0xff] | FPH[5][(p >> 16) & 0xff] + | FPH[6][(p >> 8) & 0xff] | FPH[7][ p & 0xff]) << 32) + | (FPL[0][(p >> 56) & 0xff] | FPL[1][(p >> 48) & 0xff] + | FPL[2][(p >> 40) & 0xff] | FPL[3][(p >> 32) & 0xff] + | FPL[4][(p >> 24) & 0xff] | FPL[5][(p >> 16) & 0xff] + | FPL[6][(p >> 8) & 0xff] | FPL[7][ p & 0xff]); + } +} + +/* Run every stage of the schedule over each block: one stage is DES, three are + * EDE or EEE depending on the directions the schedules were built for. */ +void crypton_des_ecb(uint8_t *out, const crypton_des_key *ks, uint32_t nkeys, + const uint8_t *in, uint32_t nblocks) +{ + uint32_t i, s; + + for (i = 0; i < nblocks; i++) { + uint64_t b = load_be64(in + 8 * i); + for (s = 0; s < nkeys; s++) + b = des_block(b, ks[s].sk); + store_be64(out + 8 * i, b); + } +} diff --git a/cbits/crypton_des.h b/cbits/crypton_des.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_des.h @@ -0,0 +1,20 @@ +#ifndef CRYPTON_DES_H +#define CRYPTON_DES_H + +#include + +/* the sixteen round keys, as eight six-bit values each */ +typedef struct { + uint8_t sk[16 * 8]; +} crypton_des_key; + +/* Build a schedule from an eight byte key. The parity bits are ignored, as + * FIPS 46-3 says. With reverse set, the rounds come out in the order that + * decrypts. */ +void crypton_des_init(crypton_des_key *ks, const uint8_t *key, int reverse); + +/* Apply nkeys schedules in order to each of nblocks eight byte blocks. */ +void crypton_des_ecb(uint8_t *out, const crypton_des_key *ks, uint32_t nkeys, + const uint8_t *in, uint32_t nblocks); + +#endif diff --git a/cbits/crypton_ecc.c b/cbits/crypton_ecc.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_ecc.c @@ -0,0 +1,521 @@ +/* + * Scalar multiplication on a curve over a prime field, doing the same work + * whatever the scalar is. + * + * The scalar is walked four bits at a time: four doublings and one addition + * of a small multiple of the point, taken from a table of sixteen that is + * read by touching every entry and keeping one of them with a mask. So a + * window costs the same five operations and the same sixteen reads whatever + * its bits are, and nothing branches on, or indexes memory with, the scalar. + * + * The addition and the doubling are the complete formulas of Renes, Costello + * and Batina (eprint 2015/1060, algorithms 1 and 3), which answer for every + * pair of points there is -- the same point twice, a point and its negation, + * the point at infinity -- without a case to choose between. A formula with + * cases would need the choice to be made with a mask like everything else + * here, and would still have to be right about which cases there are; these + * have none. They cost about half again what the usual Jacobian formulas do, + * which is the price of that. + * + * Points are kept in homogeneous projective coordinates, where the point at + * infinity is (0 : 1 : 0), and in Montgomery form, so that the only reduction + * is the one the multiplication does anyway. + */ +#include +#include +#include +#include + +/* four bits of scalar per window, so a table of sixteen and no leftover + * bits: a byte holds exactly two windows */ +#define WINDOW_BITS 4 +#define TABLE_SIZE (1 << WINDOW_BITS) + +/* the field the curve is over, and what it takes to work in it */ +typedef struct { + uint32_t n; /* limbs in a field element */ + limb_t n0; /* -p^-1 mod 2^LIMB_BITS */ + const limb_t *p; + const limb_t *a; /* the curve's a, in Montgomery form */ + const limb_t *b3; /* three times the curve's b, in Montgomery form */ + const limb_t *zero; /* n limbs of nothing, to subtract from */ + int a_is_zero; /* a is 0 or p-3 for every curve in use, and then */ + int a_is_minus3; /* multiplying by it is additions instead */ + limb_t *t; /* 2n of scratch, for the multiplication */ + limb_t *s; /* n of scratch, for the addition and subtraction */ + limb_t *s2; /* n more, for multiplying by a, which may write over + * what it is reading */ +} field; + +static void fe_mul(const field *f, limb_t *r, const limb_t *x, const limb_t *y) +{ + mont_mul(r, x, y, f->p, f->n0, f->n, f->t); +} + +static void fe_sqr(const field *f, limb_t *r, const limb_t *x) +{ + mont_sqr(r, x, f->p, f->n0, f->n, f->t); +} + +static void fe_add(const field *f, limb_t *r, const limb_t *x, const limb_t *y) +{ + limb_t carry = add_n(r, x, y, f->n); + limb_t borrow = sub_n(f->s, r, f->p, f->n); + + select_n(r, f->s, r, (carry | (borrow ^ 1)) & 1, f->n); +} + +static void fe_sub(const field *f, limb_t *r, const limb_t *x, const limb_t *y) +{ + limb_t borrow = sub_n(r, x, y, f->n); + + add_n(f->s, r, f->p, f->n); + select_n(r, f->s, r, borrow, f->n); +} + +/* r = -x */ +static void fe_neg(const field *f, limb_t *r, const limb_t *x) +{ + fe_sub(f, r, f->zero, x); +} + +/* r = a * x, where a is the curve's. It is zero or minus three on every + * curve in use, and then this is additions rather than a multiplication. + * Which of the three it is comes from the curve, which is public. */ +static void fe_mul_a(const field *f, limb_t *r, const limb_t *x) +{ + if (f->a_is_zero) { + memset(r, 0, f->n * sizeof(limb_t)); + } else if (f->a_is_minus3) { + /* r and x are the same buffer in places, so this goes through one + * of its own */ + fe_add(f, f->s2, x, x); + fe_add(f, f->s2, f->s2, x); + fe_neg(f, r, f->s2); + } else { + fe_mul(f, r, f->a, x); + } +} + +/* Renes-Costello-Batina algorithm 1: r = x + y, for any two points */ +static void point_add(const field *f, limb_t *r, const limb_t *x, + const limb_t *y, limb_t *w) +{ + uint32_t n = f->n; + const limb_t *x1 = x, *y1 = x + n, *z1 = x + 2 * n; + const limb_t *x2 = y, *y2 = y + n, *z2 = y + 2 * n; + limb_t *t0 = w, *t1 = w + n, *t2 = w + 2 * n, *t3 = w + 3 * n; + limb_t *t4 = w + 4 * n, *t5 = w + 5 * n; + limb_t *x3 = w + 6 * n, *y3 = w + 7 * n, *z3 = w + 8 * n; + + fe_mul(f, t0, x1, x2); + fe_mul(f, t1, y1, y2); + fe_mul(f, t2, z1, z2); + fe_add(f, t3, x1, y1); + fe_add(f, t4, x2, y2); + fe_mul(f, t3, t3, t4); + fe_add(f, t4, t0, t1); + fe_sub(f, t3, t3, t4); + fe_add(f, t4, x1, z1); + fe_add(f, t5, x2, z2); + fe_mul(f, t4, t4, t5); + fe_add(f, t5, t0, t2); + fe_sub(f, t4, t4, t5); + fe_add(f, t5, y1, z1); + fe_add(f, x3, y2, z2); + fe_mul(f, t5, t5, x3); + fe_add(f, x3, t1, t2); + fe_sub(f, t5, t5, x3); + fe_mul_a(f, z3, t4); + fe_mul(f, x3, f->b3, t2); + fe_add(f, z3, x3, z3); + fe_sub(f, x3, t1, z3); + fe_add(f, z3, t1, z3); + fe_mul(f, y3, x3, z3); + fe_add(f, t1, t0, t0); + fe_add(f, t1, t1, t0); + fe_mul_a(f, t2, t2); + fe_mul(f, t4, f->b3, t4); + fe_add(f, t1, t1, t2); + fe_sub(f, t2, t0, t2); + fe_mul_a(f, t2, t2); + fe_add(f, t4, t4, t2); + fe_mul(f, t0, t1, t4); + fe_add(f, y3, y3, t0); + fe_mul(f, t0, t5, t4); + fe_mul(f, x3, t3, x3); + fe_sub(f, x3, x3, t0); + fe_mul(f, t0, t3, t1); + fe_mul(f, t1, t5, z3); + fe_add(f, z3, t1, t0); + + memcpy(r, x3, n * sizeof(limb_t)); + memcpy(r + n, y3, n * sizeof(limb_t)); + memcpy(r + 2 * n, z3, n * sizeof(limb_t)); +} + +/* Renes-Costello-Batina algorithm 3: r = x + x, for any point */ +static void point_double(const field *f, limb_t *r, const limb_t *x, limb_t *w) +{ + uint32_t n = f->n; + const limb_t *px = x, *py = x + n, *pz = x + 2 * n; + limb_t *t0 = w, *t1 = w + n, *t2 = w + 2 * n, *t3 = w + 3 * n; + limb_t *x3 = w + 6 * n, *y3 = w + 7 * n, *z3 = w + 8 * n; + + fe_sqr(f, t0, px); + fe_sqr(f, t1, py); + fe_sqr(f, t2, pz); + fe_mul(f, t3, px, py); + fe_add(f, t3, t3, t3); + fe_mul(f, z3, px, pz); + fe_add(f, z3, z3, z3); + fe_mul_a(f, x3, z3); + fe_mul(f, y3, f->b3, t2); + fe_add(f, y3, x3, y3); + fe_sub(f, x3, t1, y3); + fe_add(f, y3, t1, y3); + fe_mul(f, y3, x3, y3); + fe_mul(f, x3, t3, x3); + fe_mul(f, z3, f->b3, z3); + fe_mul_a(f, t2, t2); + fe_sub(f, t3, t0, t2); + fe_mul_a(f, t3, t3); + fe_add(f, t3, t3, z3); + fe_add(f, z3, t0, t0); + fe_add(f, t0, z3, t0); + fe_add(f, t0, t0, t2); + fe_mul(f, t0, t0, t3); + fe_add(f, y3, y3, t0); + fe_mul(f, t2, py, pz); + fe_add(f, t2, t2, t2); + fe_mul(f, t0, t2, t3); + fe_sub(f, x3, x3, t0); + fe_mul(f, z3, t2, t1); + fe_add(f, z3, z3, z3); + fe_add(f, z3, z3, z3); + + memcpy(r, x3, n * sizeof(limb_t)); + memcpy(r + n, y3, n * sizeof(limb_t)); + memcpy(r + 2 * n, z3, n * sizeof(limb_t)); +} + +/* Everything a curve needs, in one allocation: the field, the buffers the + * formulas work in, and a table of sixteen points. The caller frees it with + * ctx_free. */ +typedef struct { + field f; + limb_t *space; + uint32_t words; + uint32_t n; + limb_t *r2; /* R^2 mod p, which is what takes a number to Montgomery form */ + limb_t *one; /* 1, in Montgomery form */ + limb_t *acc; /* a point */ + limb_t *sel; /* a point */ + limb_t *tmp; /* a point */ + limb_t *work; /* 9n, for the formulas */ + limb_t *table; /* sixteen points */ + uint8_t *bytes; /* 2 * plen, for the inversion */ + uint32_t plen; +} curve_ctx; + +static void ctx_free(curve_ctx *c) +{ + if (c->space != NULL) { + memset(c->space, 0, c->words * sizeof(limb_t)); + free(c->space); + } + if (c->bytes != NULL) { + memset(c->bytes, 0, 2 * c->plen); + free(c->bytes); + } + c->space = NULL; + c->bytes = NULL; +} + +/* r = x, taken into Montgomery form */ +static void to_mont(const curve_ctx *c, limb_t *r, const limb_t *x) +{ + mont_mul(r, x, c->r2, c->f.p, c->f.n0, c->n, c->f.t); +} + +/* r = x, taken back out of it */ +static void from_mont(const curve_ctx *c, limb_t *r, const limb_t *x) +{ + mont_mul(r, x, c->one, c->f.p, c->f.n0, c->n, c->f.t); +} + +static int ctx_init(curve_ctx *c, const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen) +{ + uint32_t n = (plen + LIMB_BYTES - 1) / LIMB_BYTES; + limb_t *mp, *ma, *mb3, *zero, *scratch, *mont_t; + uint32_t i; + + memset(c, 0, sizeof(*c)); + if (plen == 0 || n == 0 || (p[plen - 1] & 1) == 0) + return -1; + + /* six single numbers, three points, four of scratch, nine for the + * formulas, and a table of sixteen points */ + c->n = n; + c->plen = plen; + c->words = (6 + 9 + 4 + 9 + 3 * TABLE_SIZE) * n; + c->space = calloc(c->words, sizeof(limb_t)); + c->bytes = calloc(2, plen); + if (c->space == NULL || c->bytes == NULL) { + ctx_free(c); + return -1; + } + mp = c->space; + ma = mp + n; + mb3 = ma + n; + c->r2 = mb3 + n; + c->one = c->r2 + n; + zero = c->one + n; + c->acc = zero + n; + c->sel = c->acc + 3 * n; + c->tmp = c->sel + 3 * n; + scratch = c->tmp + 3 * n; + mont_t = scratch + 2 * n; + c->work = mont_t + 2 * n; + c->table = c->work + 9 * n; + + if (from_be(mp, n, p, plen) != 0) { + ctx_free(c); + return -1; + } + mont_r2(c->r2, mp, n, mont_t); + + c->f.n = n; + c->f.n0 = mont_n0(mp[0]); + c->f.p = mp; + c->f.a = ma; + c->f.b3 = mb3; + c->f.zero = zero; + c->f.t = mont_t; + c->f.s = scratch; + c->f.s2 = scratch + n; + c->f.a_is_zero = 0; + c->f.a_is_minus3 = 0; + + memset(c->one, 0, n * sizeof(limb_t)); + c->one[0] = 1; + to_mont(c, c->tmp, c->one); + memcpy(c->one, c->tmp, n * sizeof(limb_t)); + + /* the curve's a, and which of the three shapes it has */ + if (from_be(c->tmp, n, a, plen) != 0) { + ctx_free(c); + return -1; + } + { + limb_t nonzero = 0, differs = 0; + + for (i = 0; i < n; i++) + nonzero |= c->tmp[i]; + memset(c->sel, 0, n * sizeof(limb_t)); + c->sel[0] = 3; + sub_n(c->sel, mp, c->sel, n); /* p - 3 */ + for (i = 0; i < n; i++) + differs |= c->tmp[i] ^ c->sel[i]; + c->f.a_is_zero = nonzero == 0; + c->f.a_is_minus3 = differs == 0; + } + to_mont(c, ma, c->tmp); + + /* three times the curve's b, which is what the formulas want */ + if (from_be(c->tmp, n, b, plen) != 0) { + ctx_free(c); + return -1; + } + to_mont(c, mb3, c->tmp); + fe_add(&c->f, c->tmp, mb3, mb3); + fe_add(&c->f, mb3, c->tmp, mb3); + return 0; +} + +/* a point, in Montgomery form, from its coordinates */ +static int point_from_be(const curve_ctx *c, limb_t *r, const uint8_t *px, + const uint8_t *py) +{ + uint32_t n = c->n; + + if (from_be(c->tmp, n, px, c->plen) != 0) + return -1; + to_mont(c, r, c->tmp); + if (from_be(c->tmp, n, py, c->plen) != 0) + return -1; + to_mont(c, r + n, c->tmp); + memcpy(r + 2 * n, c->one, n * sizeof(limb_t)); + return 0; +} + +/* x = X/Z and y = Y/Z, with the inverse from Fermat, which is the + * exponentiation that hides its exponent. Returns 1 for the point at + * infinity, which has no coordinates. */ +static int point_to_be(curve_ctx *c, uint8_t *outx, uint8_t *outy, + const limb_t *pt, const uint8_t *p) +{ + uint32_t n = c->n, plen = c->plen, i; + limb_t empty = 0; + uint8_t *zbytes = c->bytes, *pm2 = c->bytes + plen; + + for (i = 0; i < n; i++) + empty |= pt[2 * n + i]; + if (empty == 0) + return 1; + + from_mont(c, c->tmp, pt + 2 * n); + to_be(zbytes, plen, c->tmp, n); + memset(c->sel, 0, n * sizeof(limb_t)); + c->sel[0] = 2; + sub_n(c->sel, c->f.p, c->sel, n); /* p - 2 */ + to_be(pm2, plen, c->sel, n); + if (crypton_powm_sec(zbytes, zbytes, plen, pm2, plen, p, plen) != 0) + return -1; + if (from_be(c->tmp, n, zbytes, plen) != 0) + return -1; + to_mont(c, c->sel, c->tmp); /* 1/Z, in Montgomery form */ + + fe_mul(&c->f, c->tmp, pt, c->sel); + from_mont(c, c->tmp + n, c->tmp); + to_be(outx, plen, c->tmp + n, n); + + fe_mul(&c->f, c->tmp, pt + n, c->sel); + from_mont(c, c->tmp + n, c->tmp); + to_be(outy, plen, c->tmp + n, n); + return 0; +} + +/* every one of the sixteen entries is read, and a mask keeps the one wanted */ +static void table_select(const curve_ctx *c, limb_t *r, const limb_t *table, + limb_t w) +{ + uint32_t n = c->n, j, l; + + memset(r, 0, 3 * n * sizeof(limb_t)); + for (j = 0; j < TABLE_SIZE; j++) { + limb_t mask = eq_mask(j, w); + + for (l = 0; l < 3 * n; l++) + r[l] |= table[3 * j * n + l] & mask; + } +} + +int crypton_ecc_mul(uint8_t *outx, uint8_t *outy, + const uint8_t *px, const uint8_t *py, + const uint8_t *k, uint32_t klen, + const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen) +{ + curve_ctx c; + uint32_t n, i, j; + int ret = -1; + + if (klen == 0 || ctx_init(&c, a, b, p, plen) != 0) + return -1; + n = c.n; + + /* the table: nothing, the point, and its multiples up to fifteen */ + memset(c.table, 0, 3 * n * sizeof(limb_t)); + memcpy(c.table + n, c.one, n * sizeof(limb_t)); /* (0 : 1 : 0) */ + if (point_from_be(&c, c.table + 3 * n, px, py) != 0) + goto done; + for (i = 2; i < TABLE_SIZE; i++) + point_add(&c.f, c.table + 3 * i * n, c.table + 3 * (i - 1) * n, + c.table + 3 * n, c.work); + + /* four bits at a time, from the top */ + memcpy(c.acc, c.table, 3 * n * sizeof(limb_t)); + for (i = klen * 2; i > 0; i--) { + uint32_t nib = i - 1; + limb_t w = (k[klen - 1 - nib / 2] >> (4 * (nib % 2))) & 0xf; + + for (j = 0; j < WINDOW_BITS; j++) + point_double(&c.f, c.acc, c.acc, c.work); + table_select(&c, c.sel, c.table, w); + point_add(&c.f, c.acc, c.acc, c.sel, c.work); + } + ret = point_to_be(&c, outx, outy, c.acc, p); + +done: + ctx_free(&c); + return ret; +} + +uint32_t crypton_ecc_table_size(uint32_t plen, uint32_t klen) +{ + uint32_t n = (plen + LIMB_BYTES - 1) / LIMB_BYTES; + + if (plen == 0 || klen == 0 || n == 0) + return 0; + return klen * 2 * TABLE_SIZE * 3 * n * (uint32_t) sizeof(limb_t); +} + +int crypton_ecc_table_build(uint8_t *tab, + const uint8_t *gx, const uint8_t *gy, + uint32_t klen, + const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen) +{ + curve_ctx c; + limb_t *t = (limb_t *) (void *) tab; + uint32_t n, i, j, windows; + int ret = -1; + + if (klen == 0 || ctx_init(&c, a, b, p, plen) != 0) + return -1; + n = c.n; + windows = klen * 2; + + /* acc walks the powers: at window i it holds 16^i times the point */ + if (point_from_be(&c, c.acc, gx, gy) != 0) + goto done; + for (i = 0; i < windows; i++) { + limb_t *slot = t + (size_t) i * TABLE_SIZE * 3 * n; + + memset(slot, 0, 3 * n * sizeof(limb_t)); + memcpy(slot + n, c.one, n * sizeof(limb_t)); /* (0 : 1 : 0) */ + memcpy(slot + 3 * n, c.acc, 3 * n * sizeof(limb_t)); + for (j = 2; j < TABLE_SIZE; j++) + point_add(&c.f, slot + 3 * j * n, slot + 3 * (j - 1) * n, + c.acc, c.work); + for (j = 0; j < WINDOW_BITS; j++) + point_double(&c.f, c.acc, c.acc, c.work); + } + ret = 0; + +done: + ctx_free(&c); + return ret; +} + +int crypton_ecc_table_mul(uint8_t *outx, uint8_t *outy, const uint8_t *tab, + const uint8_t *k, uint32_t klen, + const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen) +{ + curve_ctx c; + const limb_t *t = (const limb_t *) (const void *) tab; + uint32_t n, i; + int ret; + + if (klen == 0 || ctx_init(&c, a, b, p, plen) != 0) + return -1; + n = c.n; + + /* nothing to start with, and one addition for every four bits: the + * multiples the doubling would work out are all in the table */ + memset(c.acc, 0, 3 * n * sizeof(limb_t)); + memcpy(c.acc + n, c.one, n * sizeof(limb_t)); + for (i = 0; i < klen * 2; i++) { + limb_t w = (k[klen - 1 - i / 2] >> (4 * (i % 2))) & 0xf; + + table_select(&c, c.sel, t + (size_t) i * TABLE_SIZE * 3 * n, w); + point_add(&c.f, c.acc, c.acc, c.sel, c.work); + } + ret = point_to_be(&c, outx, outy, c.acc, p); + + ctx_free(&c); + return ret; +} diff --git a/cbits/crypton_ecc.h b/cbits/crypton_ecc.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_ecc.h @@ -0,0 +1,57 @@ +#ifndef CRYPTON_ECC_H +#define CRYPTON_ECC_H + +#include + +/* Multiply a point of a curve over a prime field by a scalar, doing the same + * work whatever the scalar is. + * + * The curve is y^2 = x^3 + a*x + b over the field of p, which has to be an + * odd prime; the point has to be on it and not the point at infinity, and its + * coordinates, a and b have to be below p. Every number is a big-endian byte + * string, and the coordinates, a, b and p are all plen bytes. + * + * The scalar is walked four bits at a time over every one of the klen bytes + * it is given, so its value is hidden but its length is not. + * + * Returns 0 with the answer in outx and outy, 1 if the answer is the point at + * infinity, which has no coordinates, and -1 if the arguments are not ones it + * can work with or memory ran out. + */ +int crypton_ecc_mul(uint8_t *outx, uint8_t *outy, + const uint8_t *px, const uint8_t *py, + const uint8_t *k, uint32_t klen, + const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen); + +/* How many bytes a table for a base point takes, for a prime of plen bytes + * and scalars of klen. Zero if those sizes are not ones it can work with. */ +uint32_t crypton_ecc_table_size(uint32_t plen, uint32_t klen); + +/* Fill that many bytes with the multiples of a point that + * crypton_ecc_table_mul wants: for every four bits of a scalar, the sixteen + * points that those bits can call for. The buffer has to be aligned as a + * pointer is, which is what an allocator gives. + * + * The point, a, b and p are as for crypton_ecc_mul. Returns 0, or -1 for + * arguments it cannot work with or memory it could not have. + */ +int crypton_ecc_table_build(uint8_t *table, + const uint8_t *gx, const uint8_t *gy, + uint32_t klen, + const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen); + +/* Multiply the point that table was built for by a scalar of klen bytes, + * which has to be the klen the table was built for. One addition for every + * four bits and no doublings, since the table holds what the doublings would + * work out. + * + * Returns what crypton_ecc_mul returns. + */ +int crypton_ecc_table_mul(uint8_t *outx, uint8_t *outy, const uint8_t *table, + const uint8_t *k, uint32_t klen, + const uint8_t *a, const uint8_t *b, + const uint8_t *p, uint32_t plen); + +#endif diff --git a/cbits/crypton_f2m.c b/cbits/crypton_f2m.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_f2m.c @@ -0,0 +1,551 @@ +/* + * Arithmetic in a binary field, and the scalar multiplication a curve over + * one needs, doing the same work whatever the scalar is. + * + * A carry-less multiplication is the one thing a binary field needs and + * ordinary arithmetic does not give. Where the processor has the instruction + * for it this uses it -- PMULL on aarch64, PCLMULQDQ on x86-64 -- asking the + * machine at run time where the compiler has not already been told. Where it + * does not, each operand is split into four groups of every fourth bit, so + * that the carries of an ordinary multiplication cannot reach the bits that + * matter, and masked away afterwards. None of the three has a table or a + * branch that depends on what it is multiplying. + * + * Reduction folds what is above the degree back in, which the polynomial + * being a trinomial or a pentanomial with exponents that are public makes + * cheap. Inversion is the exponentiation Fermat gives, whose exponent is + * likewise public. + * + * The multiplication itself is Montgomery's ladder: it carries the x + * coordinates of the multiples of two consecutive numbers, whose difference + * is therefore the point, and spends one addition and one doubling on every + * bit of the scalar whichever way the bit goes. + */ +#include +#include +#include +#include +#include + +typedef uint64_t limb_t; +#define LIMB_BITS 64 +#define LIMB_BYTES 8 + +/* the four groups, so that no carry of an ordinary multiplication reaches a + * bit another partial product needs */ +static void clmul32(uint32_t x, uint32_t y, limb_t *out) +{ + limb_t x0 = x & 0x11111111u, x1 = x & 0x22222222u; + limb_t x2 = x & 0x44444444u, x3 = x & 0x88888888u; + limb_t y0 = y & 0x11111111u, y1 = y & 0x22222222u; + limb_t y2 = y & 0x44444444u, y3 = y & 0x88888888u; + limb_t z0 = (x0 * y0) ^ (x1 * y3) ^ (x2 * y2) ^ (x3 * y1); + limb_t z1 = (x0 * y1) ^ (x1 * y0) ^ (x2 * y3) ^ (x3 * y2); + limb_t z2 = (x0 * y2) ^ (x1 * y1) ^ (x2 * y0) ^ (x3 * y3); + limb_t z3 = (x0 * y3) ^ (x1 * y2) ^ (x2 * y1) ^ (x3 * y0); + + *out = (z0 & 0x1111111111111111ULL) | (z1 & 0x2222222222222222ULL) + | (z2 & 0x4444444444444444ULL) | (z3 & 0x8888888888888888ULL); +} + +static inline void clmul(limb_t a, limb_t b, limb_t *lo, limb_t *hi) +{ + limb_t ah = a >> 32, bh = b >> 32, t0, t1, t2; + + clmul32((uint32_t) a, (uint32_t) b, &t0); + clmul32((uint32_t) ah, (uint32_t) bh, &t1); + clmul32((uint32_t) (a ^ ah), (uint32_t) (b ^ bh), &t2); + t2 ^= t0 ^ t1; + *lo = t0 ^ (t2 << 32); + *hi = t1 ^ (t2 >> 32); +} + +/* t = a * b, over 2n limbs */ +static void poly_mul_generic(limb_t *t, const limb_t *a, const limb_t *b, + uint32_t n) +{ + uint32_t i, j; + + memset(t, 0, 2 * n * sizeof(limb_t)); + for (i = 0; i < n; i++) + for (j = 0; j < n; j++) { + limb_t lo, hi; + + clmul(a[i], b[j], &lo, &hi); + t[i + j] ^= lo; + t[i + j + 1] ^= hi; + } +} + +#if defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) +#define HAVE_PMULL 1 +#include + +/* Where the compiler has been told the machine has the crypto extensions -- + * which it is on every Apple processor -- this needs no attribute and no + * question. Where it has not, the attribute lets the instruction be emitted + * in this one function, and the machine is asked before it is called. */ +#if defined(__ARM_FEATURE_CRYPTO) || defined(__ARM_FEATURE_AES) +#define PMULL_ATTR +#define PMULL_ALWAYS 1 +#else +#define PMULL_ATTR __attribute__((target("+crypto"))) +#define PMULL_ALWAYS 0 +#endif + +#if !PMULL_ALWAYS +#if defined(__linux__) || defined(__ANDROID__) +#include +#include +#elif defined(__FreeBSD__) +#include +#include +#elif defined(__APPLE__) +#include +#endif +#endif + +static int have_pmull(void) +{ +#if PMULL_ALWAYS + return 1; +#elif (defined(__linux__) || defined(__ANDROID__)) && defined(HWCAP_PMULL) + static int answer = -1; + + if (answer < 0) + answer = (getauxval(AT_HWCAP) & HWCAP_PMULL) != 0; + return answer; +#elif defined(__FreeBSD__) && defined(HWCAP_PMULL) + static int answer = -1; + + if (answer < 0) { + unsigned long hwcap = 0; + + elf_aux_info(AT_HWCAP, &hwcap, sizeof(hwcap)); + answer = (hwcap & HWCAP_PMULL) != 0; + } + return answer; +#elif defined(__APPLE__) + static int answer = -1; + + if (answer < 0) { + int has = 0; + size_t len = sizeof(has); + + answer = sysctlbyname("hw.optional.arm.FEAT_PMULL", &has, &len, + NULL, 0) == 0 + && has != 0; + } + return answer; +#else + return 0; /* no way to ask, so the four groups it is */ +#endif +} + +PMULL_ATTR +static void poly_mul_pmull(limb_t *t, const limb_t *a, const limb_t *b, + uint32_t n) +{ + uint32_t i, j; + + memset(t, 0, 2 * n * sizeof(limb_t)); + for (i = 0; i < n; i++) + for (j = 0; j < n; j++) { + uint64x2_t v = vreinterpretq_u64_p128( + vmull_p64((poly64_t) a[i], (poly64_t) b[j])); + + t[i + j] ^= vgetq_lane_u64(v, 0); + t[i + j + 1] ^= vgetq_lane_u64(v, 1); + } +} +#else +#define HAVE_PMULL 0 +#endif + +#if defined(__x86_64__) && (defined(__GNUC__) || defined(__clang__)) +#define HAVE_PCLMUL 1 +#include + +/* The same, with the instruction x86 has for it. The attribute is what lets + * one file hold both this and the code for a processor without it: the + * compiler may emit the instruction here and nowhere else, and the caller + * asks the processor before it comes this way. + */ +__attribute__((target("pclmul,sse2"))) +static void poly_mul_pclmul(limb_t *t, const limb_t *a, const limb_t *b, + uint32_t n) +{ + uint32_t i, j; + + memset(t, 0, 2 * n * sizeof(limb_t)); + for (i = 0; i < n; i++) + for (j = 0; j < n; j++) { + __m128i p = _mm_clmulepi64_si128( + _mm_cvtsi64_si128((long long) a[i]), + _mm_cvtsi64_si128((long long) b[j]), 0x00); + + t[i + j] ^= (limb_t) _mm_cvtsi128_si64(p); + t[i + j + 1] ^= + (limb_t) _mm_cvtsi128_si64(_mm_srli_si128(p, 8)); + } +} +#else +#define HAVE_PCLMUL 0 +#endif + +static void poly_mul(limb_t *t, const limb_t *a, const limb_t *b, uint32_t n) +{ +#if HAVE_PMULL + /* what the processor has is not what is being multiplied, so asking is + * not a side channel, and the answer is worked out once */ + if (have_pmull()) { + poly_mul_pmull(t, a, b, n); + return; + } +#endif +#if HAVE_PCLMUL + /* what the processor has is not what is being multiplied, so asking is + * not a side channel, and the answer is worked out once */ + if (crypton_x86_simd_features() & CRYPTON_X86_PCLMUL) { + poly_mul_pclmul(t, a, b, n); + return; + } +#endif + poly_mul_generic(t, a, b, n); +} + +/* the bits of a 32-bit half, spread out with a zero between each pair */ +static limb_t spread(limb_t x) +{ + x = (x | (x << 16)) & 0x0000ffff0000ffffULL; + x = (x | (x << 8)) & 0x00ff00ff00ff00ffULL; + x = (x | (x << 4)) & 0x0f0f0f0f0f0f0f0fULL; + x = (x | (x << 2)) & 0x3333333333333333ULL; + x = (x | (x << 1)) & 0x5555555555555555ULL; + return x; +} + +/* t = a * a, which in a binary field is the bits of a spread out */ +static void poly_sqr(limb_t *t, const limb_t *a, uint32_t n) +{ + uint32_t i; + + for (i = 0; i < n; i++) { + t[2 * i] = spread(a[i] & 0xffffffffULL); + t[2 * i + 1] = spread(a[i] >> 32); + } +} + +/* r = t mod fx, where fx is x^m plus the terms given, which are public + * + * Everything above bit m comes back in as those terms, a word at a time, and + * then what is left above bit m within its own word is folded the same way. + */ +static void poly_reduce(limb_t *r, limb_t *t, uint32_t n, uint32_t m, + const uint32_t *terms, uint32_t nterms) +{ + uint32_t mw = m / LIMB_BITS, mb = m % LIMB_BITS, i, j, pass; + + for (i = 2 * n; i > mw + 1; i--) { + limb_t w = t[i - 1]; + + t[i - 1] = 0; + for (j = 0; j < nterms; j++) { + uint32_t pos = (i - 1) * LIMB_BITS - m + terms[j]; + uint32_t pw = pos / LIMB_BITS, pb = pos % LIMB_BITS; + + t[pw] ^= w << pb; + if (pb != 0) + t[pw + 1] ^= w >> (LIMB_BITS - pb); + } + } + + /* what is left above bit m sits in the word that holds it; folding it + * can put a little back, so it is done twice */ + for (pass = 0; pass < 2; pass++) { + limb_t w; + + if (mb == 0) + break; + w = t[mw] >> mb; + t[mw] &= ((limb_t) 1 << mb) - 1; + for (j = 0; j < nterms; j++) { + uint32_t pw = terms[j] / LIMB_BITS, pb = terms[j] % LIMB_BITS; + + t[pw] ^= w << pb; + if (pb != 0 && pw + 1 <= mw) + t[pw + 1] ^= w >> (LIMB_BITS - pb); + } + } + memcpy(r, t, n * sizeof(limb_t)); +} + +/* the field: its polynomial, and scratch for a product */ +typedef struct { + uint32_t n; + uint32_t m; + uint32_t terms[8]; /* the polynomial without its leading term */ + uint32_t nterms; + limb_t *t; /* 2n */ +} bfield; + +static void fe_mul(const bfield *f, limb_t *r, const limb_t *a, const limb_t *b) +{ + poly_mul(f->t, a, b, f->n); + poly_reduce(r, f->t, f->n, f->m, f->terms, f->nterms); +} + +static void fe_sqr(const bfield *f, limb_t *r, const limb_t *a) +{ + poly_sqr(f->t, a, f->n); + poly_reduce(r, f->t, f->n, f->m, f->terms, f->nterms); +} + +static void fe_add(const bfield *f, limb_t *r, const limb_t *a, const limb_t *b) +{ + uint32_t i; + + for (i = 0; i < f->n; i++) + r[i] = a[i] ^ b[i]; +} + +static int fe_is_zero(const bfield *f, const limb_t *a) +{ + limb_t acc = 0; + uint32_t i; + + for (i = 0; i < f->n; i++) + acc |= a[i]; + return acc == 0; +} + +/* r = 1/a, by Fermat: a to the power 2^m - 2, whose exponent is public */ +static void fe_inv(const bfield *f, limb_t *r, const limb_t *a, limb_t *tmp) +{ + uint32_t i; + + memcpy(tmp, a, f->n * sizeof(limb_t)); + for (i = 1; i + 1 < f->m; i++) { /* a to the power 2^(m-1) - 1 */ + fe_sqr(f, tmp, tmp); + fe_mul(f, tmp, tmp, a); + } + fe_sqr(f, r, tmp); +} + +/* big-endian bytes into limbs, least significant limb first */ +static int from_be(limb_t *r, uint32_t n, const uint8_t *src, uint32_t len) +{ + uint32_t i; + + memset(r, 0, n * sizeof(limb_t)); + for (i = 0; i < len; i++) { + uint8_t byte = src[len - 1 - i]; + + if (i / LIMB_BYTES >= n) { + if (byte != 0) + return 1; + continue; + } + r[i / LIMB_BYTES] |= (limb_t) byte << (8 * (i % LIMB_BYTES)); + } + return 0; +} + +static void to_be(uint8_t *dst, uint32_t len, const limb_t *a, uint32_t n) +{ + uint32_t i; + + for (i = 0; i < len; i++) { + uint32_t pos = len - 1 - i, li = i / LIMB_BYTES; + + dst[pos] = li < n ? (uint8_t) (a[li] >> (8 * (i % LIMB_BYTES))) : 0; + } +} + +/* exchange a and b when swap is one */ +static void cswap(limb_t *a, limb_t *b, limb_t swap, uint32_t n) +{ + limb_t mask = (limb_t) 0 - swap; + uint32_t i; + + for (i = 0; i < n; i++) { + limb_t t = (a[i] ^ b[i]) & mask; + + a[i] ^= t; + b[i] ^= t; + } +} + +int crypton_f2m_mul(uint8_t *outx, uint8_t *outy, + const uint8_t *px, const uint8_t *py, + const uint8_t *k, uint32_t klen, + const uint8_t *b, uint32_t flen, + const uint8_t *fx, uint32_t fxlen) +{ + uint32_t fn = (fxlen + LIMB_BYTES - 1) / LIMB_BYTES; + uint32_t n, words, i; + limb_t *space = NULL, *poly, *x, *y, *bb, *x1, *z1, *x2, *z2; + limb_t *t1, *t2, *t3, *prod; + bfield f; + int ret = -1; + + if (flen == 0 || fxlen == 0 || klen == 0 || fn == 0) + return -1; + + /* the polynomial, and the terms below its leading one */ + { + limb_t *tmp = calloc(fn, sizeof(limb_t)); + uint32_t m = 0; + + if (tmp == NULL) + return -1; + if (from_be(tmp, fn, fx, fxlen) != 0) { + free(tmp); + return -1; + } + for (i = fn; i > 0 && m == 0; i--) + if (tmp[i - 1] != 0) { + limb_t top = tmp[i - 1]; + + m = (i - 1) * LIMB_BITS; + while (top != 0) { + m++; + top >>= 1; + } + m--; /* the degree is one under the bit count */ + } + f.m = m; + f.nterms = 0; + for (i = 0; i < m; i++) + if ((tmp[i / LIMB_BITS] >> (i % LIMB_BITS)) & 1) { + if (f.nterms >= 8) { + free(tmp); + return -1; /* more terms than anything in use has */ + } + f.terms[f.nterms++] = i; + } + free(tmp); + if (m == 0 || f.nterms == 0) + return -1; + } + + n = (f.m + LIMB_BITS) / LIMB_BITS; /* room for the degree itself */ + f.n = n; + words = 12 * n + 2 * n; + space = calloc(words, sizeof(limb_t)); + if (space == NULL) + return -1; + poly = space; /* unused beyond keeping the layout plain */ + x = poly + n; + y = x + n; + bb = y + n; + x1 = bb + n; + z1 = x1 + n; + x2 = z1 + n; + z2 = x2 + n; + t1 = z2 + n; + t2 = t1 + n; + t3 = t2 + n; + prod = t3 + n; /* 2n, and one n before it is spare */ + f.t = prod; + + if (from_be(x, n, px, flen) != 0 || from_be(y, n, py, flen) != 0 + || from_be(bb, n, b, flen) != 0) + goto done; + if (fe_is_zero(&f, x)) + goto done; /* the point with no x is the caller's business */ + + /* nothing, and the point next to it */ + memset(x1, 0, n * sizeof(limb_t)); + x1[0] = 1; + memset(z1, 0, n * sizeof(limb_t)); + memcpy(x2, x, n * sizeof(limb_t)); + memset(z2, 0, n * sizeof(limb_t)); + z2[0] = 1; + + for (i = klen * 8; i > 0; i--) { + uint32_t bit = i - 1; + limb_t sel = (k[klen - 1 - bit / 8] >> (bit % 8)) & 1; + + /* whichever way the bit goes, one addition and one doubling: the + * exchange before and after is what puts them where the bit asks */ + cswap(x1, x2, sel, n); + cswap(z1, z2, sel, n); + + /* the two added, which their difference being the point allows */ + fe_mul(&f, t1, x1, z2); + fe_mul(&f, t2, x2, z1); + fe_add(&f, t3, t1, t2); + fe_sqr(&f, t3, t3); /* the new z */ + fe_mul(&f, t1, t1, t2); + fe_mul(&f, t2, x, t3); + fe_add(&f, t2, t2, t1); /* the new x */ + + /* and one of them doubled */ + fe_sqr(&f, x1, x1); + fe_sqr(&f, z1, z1); + fe_mul(&f, t1, x1, z1); /* z of the double */ + fe_sqr(&f, x1, x1); + fe_sqr(&f, z1, z1); + fe_mul(&f, z1, z1, bb); + fe_add(&f, x1, x1, z1); /* x of the double */ + memcpy(z1, t1, n * sizeof(limb_t)); + + memcpy(x2, t2, n * sizeof(limb_t)); + memcpy(z2, t3, n * sizeof(limb_t)); + + cswap(x1, x2, sel, n); + cswap(z1, z2, sel, n); + } + + if (fe_is_zero(&f, z1)) { + ret = 1; /* the multiple is at infinity */ + goto done; + } + if (fe_is_zero(&f, z2)) { + /* the one after it is, so this one is the negation of the point */ + to_be(outx, flen, x, n); + fe_add(&f, t1, x, y); + to_be(outy, flen, t1, n); + ret = 0; + goto done; + } + + /* x1/z1 and x2/z2, and the y the ladder does not carry, out of one + * inversion: 1/(z1 z2 x) gives each of the three */ + fe_mul(&f, t1, z1, z2); + fe_mul(&f, t1, t1, x); + fe_inv(&f, t2, t1, t3); + { + limb_t *xa = x1, *xb = x2, *u = t1, *v = t3; + + fe_mul(&f, u, z2, x); + fe_mul(&f, u, u, t2); /* 1/z1 */ + fe_mul(&f, xa, x1, u); + fe_mul(&f, v, z1, x); + fe_mul(&f, v, v, t2); /* 1/z2 */ + fe_mul(&f, xb, x2, v); + fe_mul(&f, u, z1, z2); + fe_mul(&f, u, u, t2); /* 1/x */ + + fe_add(&f, v, xa, x); /* x1 + x */ + fe_add(&f, xb, xb, x); /* x2 + x */ + fe_mul(&f, xb, v, xb); /* (x1 + x)(x2 + x) */ + fe_sqr(&f, t2, x); + fe_add(&f, xb, xb, t2); + fe_add(&f, xb, xb, y); /* + x^2 + y */ + fe_mul(&f, xb, v, xb); + fe_mul(&f, xb, xb, u); /* over x */ + fe_add(&f, xb, xb, y); + to_be(outx, flen, xa, n); + to_be(outy, flen, xb, n); + } + ret = 0; + +done: + if (space != NULL) { + memset(space, 0, words * sizeof(limb_t)); + free(space); + } + return ret; +} diff --git a/cbits/crypton_f2m.h b/cbits/crypton_f2m.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_f2m.h @@ -0,0 +1,31 @@ +#ifndef CRYPTON_F2M_H +#define CRYPTON_F2M_H + +#include + +/* Multiply a point of a curve over a binary field by a scalar, doing the same + * work whatever the scalar is. + * + * The curve is y^2 + x*y = x^3 + a*x^2 + b over the field of the polynomial + * fx, and a does not come into it: the ladder carries the x coordinates of + * two consecutive multiples, and what it takes to add them is b alone. The + * point has to be on the curve and to have an x that is not zero -- the one + * point with none is its own negation, and the caller sees to it. + * + * Every number is a big-endian byte string. The coordinates and b are flen + * bytes, and fx is the whole polynomial, x^m included, in fxlen. + * + * The scalar is walked over every bit of the klen bytes it is given, so its + * value is hidden but its length is not. + * + * Returns 0 with the answer in outx and outy, 1 if the answer is the point at + * infinity, and -1 for arguments it cannot work with or memory it could not + * have. + */ +int crypton_f2m_mul(uint8_t *outx, uint8_t *outy, + const uint8_t *px, const uint8_t *py, + const uint8_t *k, uint32_t klen, + const uint8_t *b, uint32_t flen, + const uint8_t *fx, uint32_t fxlen); + +#endif diff --git a/cbits/crypton_memxor.c b/cbits/crypton_memxor.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_memxor.c @@ -0,0 +1,29 @@ +/* + * dst = a xor b. + * + * Data.ByteArray's xor walks a byte at a time through an IO applicative, and + * that allocates: fifty bytes of heap for every byte exclusive-ored, which in + * counter mode cost more than the cipher did. This is the same operation in + * one pass of words. + */ + +#include +#include + +#include "crypton_memxor.h" + +void crypton_memxor(uint8_t *dst, const uint8_t *a, const uint8_t *b, uint32_t len) +{ + uint32_t i = 0; + + for (; i + 8 <= len; i += 8) { + uint64_t x, y; + + memcpy(&x, a + i, 8); + memcpy(&y, b + i, 8); + x ^= y; + memcpy(dst + i, &x, 8); + } + for (; i < len; i++) + dst[i] = a[i] ^ b[i]; +} diff --git a/cbits/crypton_memxor.h b/cbits/crypton_memxor.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_memxor.h @@ -0,0 +1,8 @@ +#ifndef CRYPTON_MEMXOR_H +#define CRYPTON_MEMXOR_H + +#include + +void crypton_memxor(uint8_t *dst, const uint8_t *a, const uint8_t *b, uint32_t len); + +#endif diff --git a/cbits/crypton_poly1305.c b/cbits/crypton_poly1305.c --- a/cbits/crypton_poly1305.c +++ b/cbits/crypton_poly1305.c @@ -39,8 +39,52 @@ #include "crypton_bitfn.h" #include "crypton_align.h" + +/* + * Poly1305 from CRYPTOGAMS, in cbits/asm/poly1305-armv8-*.S and + * cbits/asm/poly1305-x86_64-*.S, which is the whole of the arithmetic + * rather than a bulk loop bolted to the side: it keeps its own accumulator + * -- in base 2^64 while the message is short and base 2^26 once the vector + * loop has started, switching between the two itself -- and its own powers + * of r, so what is left here is the buffering of partial blocks. + * + * 'padbit' is the high bit above each block, which is set for every block + * of the message and clear for the padded last one. + */ +#if (defined(WITH_ARMV8_POLY1305_ASM) && !defined(__AARCH64EB__)) \ + || defined(WITH_X86_POLY1305_ASM) +#define POLY1305_ASM 1 +#include "crypton_cpu.h" + +typedef void (*poly1305_blocks_f)(void *ctx, const uint8_t *inp, size_t len, + uint32_t padbit); +typedef void (*poly1305_emit_f)(void *ctx, uint8_t mac[16], + const uint32_t nonce[4]); + +int crypton_poly1305_asm_init(void *ctx, const uint8_t key[16], void *func[2]); + +/* + * Initialisation hands back the pair of functions its own dispatch would + * use, the vector entry points themselves being local to the module. They + * are the same for every context, so they are kept here rather than in each + * one; two threads racing to fill them write the same values. + */ +static poly1305_blocks_f asm_blocks; +static poly1305_emit_f asm_emit; +#endif + + +#ifdef POLY1305_ASM + static void poly1305_do_chunk(poly1305_ctx *ctx, uint8_t *data, int blocks, int final) { + asm_blocks(ctx->st.opaque, data, (size_t) blocks * 16, final ? 0 : 1); +} + +#else + +static void poly1305_do_chunk(poly1305_ctx *ctx, uint8_t *data, int blocks, int final) +{ /* following is a cleanup copy of code available poly1305-donna */ const uint32_t hibit = (final) ? 0 : (1 << 24); /* 1 << 128 */ uint32_t r0,r1,r2,r3,r4; @@ -49,9 +93,10 @@ uint64_t d0,d1,d2,d3,d4; uint32_t c; + /* load r[i], h[i] */ - h0 = ctx->h[0]; h1 = ctx->h[1]; h2 = ctx->h[2]; h3 = ctx->h[3]; h4 = ctx->h[4]; - r0 = ctx->r[0]; r1 = ctx->r[1]; r2 = ctx->r[2]; r3 = ctx->r[3]; r4 = ctx->r[4]; + h0 = ctx->st.limb.h[0]; h1 = ctx->st.limb.h[1]; h2 = ctx->st.limb.h[2]; h3 = ctx->st.limb.h[3]; h4 = ctx->st.limb.h[4]; + r0 = ctx->st.limb.r[0]; r1 = ctx->st.limb.r[1]; r2 = ctx->st.limb.r[2]; r3 = ctx->st.limb.r[3]; r4 = ctx->st.limb.r[4]; /* s[i] = r[i] * 5 */ s1 = r1 * 5; s2 = r2 * 5; s3 = r3 * 5; s4 = r4 * 5; @@ -81,21 +126,37 @@ } /* store h[i] */ - ctx->h[0] = h0; ctx->h[1] = h1; ctx->h[2] = h2; ctx->h[3] = h3; ctx->h[4] = h4; + ctx->st.limb.h[0] = h0; ctx->st.limb.h[1] = h1; ctx->st.limb.h[2] = h2; ctx->st.limb.h[3] = h3; ctx->st.limb.h[4] = h4; } +#endif + void crypton_poly1305_init(poly1305_ctx *ctx, poly1305_key *key) { uint8_t *k = (uint8_t *) key; memset(ctx, 0, sizeof(poly1305_ctx)); - ctx->r[0] = (load_le32(&k[ 0]) ) & 0x3ffffff; - ctx->r[1] = (load_le32(&k[ 3]) >> 2) & 0x3ffff03; - ctx->r[2] = (load_le32(&k[ 6]) >> 4) & 0x3ffc0ff; - ctx->r[3] = (load_le32(&k[ 9]) >> 6) & 0x3f03fff; - ctx->r[4] = (load_le32(&k[12]) >> 8) & 0x00fffff; +#ifdef POLY1305_ASM + { + void *func[2]; +#ifdef CRYPTON_X86_ASM + /* what the module dispatches on, which it reads directly */ + crypton_x86_ia32cap_resolve(); +#endif + crypton_poly1305_asm_init(ctx->st.opaque, k, func); + asm_blocks = (poly1305_blocks_f) func[0]; + asm_emit = (poly1305_emit_f) func[1]; + } +#else + ctx->st.limb.r[0] = (load_le32(&k[ 0]) ) & 0x3ffffff; + ctx->st.limb.r[1] = (load_le32(&k[ 3]) >> 2) & 0x3ffff03; + ctx->st.limb.r[2] = (load_le32(&k[ 6]) >> 4) & 0x3ffc0ff; + ctx->st.limb.r[3] = (load_le32(&k[ 9]) >> 6) & 0x3f03fff; + ctx->st.limb.r[4] = (load_le32(&k[12]) >> 8) & 0x00fffff; +#endif + ctx->pad[0] = load_le32(&k[16]); ctx->pad[1] = load_le32(&k[20]); ctx->pad[2] = load_le32(&k[24]); @@ -134,11 +195,6 @@ void crypton_poly1305_finalize(poly1305_mac mac8, poly1305_ctx *ctx) { - uint32_t h0,h1,h2,h3,h4,c; - uint32_t g0,g1,g2,g3,g4; - uint64_t f; - uint32_t mask; - uint32_t *mac = (uint32_t *) mac8; int i; if (ctx->index) { @@ -149,10 +205,22 @@ poly1305_do_chunk(ctx, ctx->buf, 1, 1); } +#ifdef POLY1305_ASM + /* the carry, the reduction and the addition of the second half of + * the key are the assembly's, since the accumulator is its own */ + asm_emit(ctx->st.opaque, mac8, ctx->pad); +#else + { + uint32_t h0,h1,h2,h3,h4,c; + uint32_t g0,g1,g2,g3,g4; + uint64_t f; + uint32_t mask; + uint32_t *mac = (uint32_t *) mac8; + /* following is a cleanup copy of code available poly1305-donna */ /* fully carry h */ - h0 = ctx->h[0]; h1 = ctx->h[1]; h2 = ctx->h[2]; h3 = ctx->h[3]; h4 = ctx->h[4]; + h0 = ctx->st.limb.h[0]; h1 = ctx->st.limb.h[1]; h2 = ctx->st.limb.h[2]; h3 = ctx->st.limb.h[3]; h4 = ctx->st.limb.h[4]; c = h1 >> 26; h1 = h1 & 0x3ffffff; h2 += c; c = h2 >> 26; h2 = h2 & 0x3ffffff; @@ -200,4 +268,6 @@ f = (uint64_t)h3 + ctx->pad[3] + (f >> 32); mac[3] = cpu_to_le32((uint32_t) f); + } +#endif } diff --git a/cbits/crypton_poly1305.h b/cbits/crypton_poly1305.h --- a/cbits/crypton_poly1305.h +++ b/cbits/crypton_poly1305.h @@ -30,11 +30,24 @@ #ifndef CRYPTON_POLY1305_H # define CRYPTON_POLY1305_H -/* 8*8+1*16+1*4 = 84 */ +/* + * Either the 26-bit limbs the C implementation works in, or the state the + * assembly keeps: its accumulator, in whichever base it is using at the + * time, the clamped key, and the powers of that laid out for the four-way + * vector loop, which together come to exactly 192 bytes -- OpenSSL allots + * the same for the same thing. + * + * size = 192+16+4+16 = 228, 232 with the alignment the union asks for + */ typedef struct { - uint32_t r[5]; - uint32_t h[5]; + union { + struct { + uint32_t r[5]; + uint32_t h[5]; + } limb; + uint64_t opaque[24]; + } st; uint32_t pad[4]; uint32_t index; uint8_t buf[16]; /* previous partial block */ diff --git a/cbits/crypton_powm.c b/cbits/crypton_powm.c new file mode 100644 --- /dev/null +++ b/cbits/crypton_powm.c @@ -0,0 +1,111 @@ +/* + * Modular exponentiation that does the same work whatever the exponent is. + * + * The exponent is walked four bits at a time: four squarings and one + * multiplication by a small power of the base, taken from a table of sixteen. + * The table is read by touching all sixteen entries and keeping one of them + * with a mask, so the address stream does not follow the exponent, and the + * multiplication itself is Montgomery's, whose only conditional step -- the + * subtraction at the end -- is also done with a mask. + * + * So every window costs the same four squarings, the same multiplication and + * the same sixteen reads, and nothing here branches on, or indexes memory + * with, anything derived from the exponent. + * + * What is still visible is how many bytes the caller passed: the loop runs + * over every bit of them, so the exponent's value is hidden but its length is + * not. GMP's mpz_powm_sec, which this replaces on the GHCs that no longer + * offer it, hides the same amount. + */ +#include +#include +#include + +/* four bits of exponent per window, so a table of sixteen and no leftover + * bits: a byte holds exactly two windows */ +#define WINDOW_BITS 4 +#define TABLE_SIZE (1 << WINDOW_BITS) + +int crypton_powm_sec(uint8_t *out, + const uint8_t *base, uint32_t baselen, + const uint8_t *exp, uint32_t explen, + const uint8_t *mod, uint32_t modlen) +{ + uint32_t n = (modlen + LIMB_BYTES - 1) / LIMB_BYTES; + uint32_t words = (TABLE_SIZE + 7) * n; + limb_t *space, *m, *r2, *acc, *sel, *prod, *table, *t, n0; + uint32_t i, j, k; + + if (modlen == 0 || n == 0 || (mod[modlen - 1] & 1) == 0) + return 1; + + /* the table, five more n-limb numbers and one of 2n */ + space = calloc(words, sizeof(limb_t)); + if (space == NULL) + return 1; + m = space; + r2 = m + n; + acc = r2 + n; + sel = acc + n; + prod = sel + n; + t = prod + n; + table = t + 2 * n; + + if (from_be(m, n, mod, modlen) != 0) + goto fail; + mont_r2(r2, m, n, t); + n0 = mont_n0(m[0]); + + /* table[k] = base^k in Montgomery form, and table[0] = 1 there */ + memset(table, 0, n * sizeof(limb_t)); + table[0] = 1; + mont_mul(acc, table, r2, m, n0, n, t); + memcpy(table, acc, n * sizeof(limb_t)); + + if (from_be(sel, n, base, baselen) != 0) + goto fail; + mont_mul(table + n, sel, r2, m, n0, n, t); + for (k = 2; k < TABLE_SIZE; k++) + mont_mul(table + k * n, table + (k - 1) * n, table + n, m, n0, n, t); + + memcpy(acc, table, n * sizeof(limb_t)); + + for (i = explen * 2; i > 0; i--) { + uint32_t nib = i - 1; + limb_t w = (exp[explen - 1 - nib / 2] >> (4 * (nib % 2))) & 0xf; + + for (j = 0; j < WINDOW_BITS; j++) { + mont_sqr(sel, acc, m, n0, n, t); + memcpy(acc, sel, n * sizeof(limb_t)); + } + + /* every entry is read, and a mask keeps the one wanted */ + memset(sel, 0, n * sizeof(limb_t)); + for (k = 0; k < TABLE_SIZE; k++) { + limb_t mask = eq_mask(k, w); + uint32_t l; + + for (l = 0; l < n; l++) + sel[l] |= table[k * n + l] & mask; + } + mont_mul(prod, acc, sel, m, n0, n, t); + memcpy(acc, prod, n * sizeof(limb_t)); + } + + /* out of Montgomery form */ + memset(sel, 0, n * sizeof(limb_t)); + sel[0] = 1; + mont_mul(prod, acc, sel, m, n0, n, t); + to_be(out, modlen, prod, n); + + /* nothing here is the caller's secret, but the exponent's bits passed + * through the accumulators */ + memset(space, 0, words * sizeof(limb_t)); + free(space); + return 0; + +fail: + memset(space, 0, words * sizeof(limb_t)); + free(space); + return 1; +} diff --git a/cbits/crypton_powm.h b/cbits/crypton_powm.h new file mode 100644 --- /dev/null +++ b/cbits/crypton_powm.h @@ -0,0 +1,23 @@ +#ifndef CRYPTON_POWM_H +#define CRYPTON_POWM_H + +#include + +/* Modular exponentiation whose work does not depend on the exponent's bits. + * + * All three numbers are big-endian byte strings. The modulus has to be odd + * and at least one byte, and the base has to be smaller than it: the caller + * reduces, which it can do in whatever way it likes, because in this library + * the base is always a public value. + * + * The result is written to out, which holds modlen bytes. + * + * Returns 0 on success, and nonzero if the modulus is even or memory ran out, + * in which case out is untouched. + */ +int crypton_powm_sec(uint8_t *out, + const uint8_t *base, uint32_t baselen, + const uint8_t *exp, uint32_t explen, + const uint8_t *mod, uint32_t modlen); + +#endif diff --git a/cbits/crypton_sha1.c b/cbits/crypton_sha1.c --- a/cbits/crypton_sha1.c +++ b/cbits/crypton_sha1.c @@ -26,7 +26,42 @@ #include "crypton_sha1.h" #include "crypton_bitfn.h" #include "crypton_align.h" +/* + * AArch64 can do four rounds at a time with the SHA-1 instructions; see + * sha1_armv8.c. They are optional in ARMv8.0, so ask before using them. + * Two threads racing to answer here both write the same value. + */ +#ifdef WITH_ARMV8_SHA1 +extern void crypton_sha1_armv8_do_chunk(uint32_t state[5], const uint32_t buf[16]); +extern void crypton_sha1_armv8_do_chunks(uint32_t state[5], const uint8_t *data, + uint32_t blocks); +extern int crypton_sha1_armv8_available(void); +#ifdef WITH_ARMV8_SHA1_ASM +/* + * SHA-1 from CRYPTOGAMS, in cbits/asm/sha1-armv8-*.S. The instructions are + * the ones the intrinsics beside it use; what the module does with them is + * schedule the message schedule of the next four rounds against the rounds + * of this one, which a C function cannot be made to do. + * + * The entry point for processors that have the instructions is not + * exported, so the module's own dispatch is what picks it, and the answer + * to the question this file already asks goes into the word that dispatch + * reads. + */ +#define SHA1_ASM 1 +#include "crypton_cpu.h" +extern void crypton_sha1_asm_block_data_order(uint32_t state[5], + const void *data, size_t blocks); +#endif + +static int sha1_use_armv8 = -1; +#endif + +#ifdef WITH_X86_SHA_NI +#include "crypton_cpu.h" +#endif + void crypton_sha1_init(struct sha1_ctx *ctx) { memset(ctx, 0, sizeof(*ctx)); @@ -54,7 +89,7 @@ #define M(i) (w[i & 0x0f] = rol32(w[i & 0x0f] ^ w[(i - 14) & 0x0f] \ ^ w[(i - 8) & 0x0f] ^ w[(i - 3) & 0x0f], 1)) -static inline void sha1_do_chunk(struct sha1_ctx *ctx, uint32_t *buf) +static void sha1_do_chunk_generic(struct sha1_ctx *ctx, uint32_t *buf) { uint32_t a, b, c, d, e; uint32_t w[16]; @@ -156,6 +191,50 @@ ctx->h[4] += e; } +#ifdef WITH_X86_SHA_NI +/* + * x86 can do four rounds at a time with the SHA extensions; see sha1_x86.c. + * They arrived long after the x86-64 baseline, so ask before using them. + * Two threads racing to answer here both write the same value. + */ +extern void crypton_sha1_x86_do_chunk(uint32_t state[5], const uint32_t buf[16]); +extern void crypton_sha1_x86_do_chunks(uint32_t state[5], const uint8_t *data, + uint32_t blocks); + +static int sha1_use_x86 = -1; +#endif + +static inline void sha1_do_chunk(struct sha1_ctx *ctx, uint32_t *buf) +{ +#ifdef WITH_ARMV8_SHA1 + if (sha1_use_armv8 < 0) { + sha1_use_armv8 = crypton_sha1_armv8_available(); +#ifdef SHA1_ASM + if (sha1_use_armv8) + crypton_armcap_P |= CRYPTON_ARMCAP_SHA1; +#endif + } + if (sha1_use_armv8) { +#ifdef SHA1_ASM + crypton_sha1_asm_block_data_order(ctx->h, buf, 1); +#else + crypton_sha1_armv8_do_chunk(ctx->h, buf); +#endif + return; + } +#endif +#ifdef WITH_X86_SHA_NI + if (sha1_use_x86 < 0) + sha1_use_x86 = + (crypton_x86_simd_features() & CRYPTON_X86_SHA_NI) != 0; + if (sha1_use_x86) { + crypton_sha1_x86_do_chunk(ctx->h, buf); + return; + } +#endif + sha1_do_chunk_generic(ctx, buf); +} + void crypton_sha1_update(struct sha1_ctx *ctx, const uint8_t *data, uint32_t len) { uint32_t index, to_fill; @@ -173,6 +252,45 @@ data += to_fill; index = 0; } + + /* + * Where there are instructions for this, the whole run of blocks + * goes over at once: the state then stays in registers from one + * block to the next, and the message is read as bytes, so neither + * the alignment nor the copy below is wanted. + */ +#ifdef WITH_ARMV8_SHA1 + if (sha1_use_armv8 < 0) { + sha1_use_armv8 = crypton_sha1_armv8_available(); +#ifdef SHA1_ASM + if (sha1_use_armv8) + crypton_armcap_P |= CRYPTON_ARMCAP_SHA1; +#endif + } + if (sha1_use_armv8 && len >= 64) { + uint32_t blocks = len / 64; + +#ifdef SHA1_ASM + crypton_sha1_asm_block_data_order(ctx->h, data, blocks); +#else + crypton_sha1_armv8_do_chunks(ctx->h, data, blocks); +#endif + data += blocks * 64; + len -= blocks * 64; + } +#endif +#ifdef WITH_X86_SHA_NI + if (sha1_use_x86 < 0) + sha1_use_x86 = + (crypton_x86_simd_features() & CRYPTON_X86_SHA_NI) != 0; + if (sha1_use_x86 && len >= 64) { + uint32_t blocks = len / 64; + + crypton_sha1_x86_do_chunks(ctx->h, data, blocks); + data += blocks * 64; + len -= blocks * 64; + } +#endif if (need_alignment(data, 4)) { uint32_t tramp[16]; diff --git a/cbits/crypton_sha256.c b/cbits/crypton_sha256.c --- a/cbits/crypton_sha256.c +++ b/cbits/crypton_sha256.c @@ -26,6 +26,9 @@ #include "crypton_sha256.h" #include "crypton_bitfn.h" #include "crypton_align.h" +#ifdef WITH_X86_SHA_NI +#include "crypton_cpu.h" +#endif void crypton_sha224_init(struct sha224_ctx *ctx) { @@ -75,7 +78,7 @@ #define s0(x) (ror32(x, 7) ^ ror32(x,18) ^ (x >> 3)) #define s1(x) (ror32(x,17) ^ ror32(x,19) ^ (x >> 10)) -static void sha256_do_chunk(struct sha256_ctx *ctx, uint32_t buf[]) +static void sha256_do_chunk_generic(struct sha256_ctx *ctx, uint32_t buf[]) { uint32_t a, b, c, d, e, f, g, h, t1, t2; int i; @@ -111,6 +114,66 @@ ctx->h[4] += e; ctx->h[5] += f; ctx->h[6] += g; ctx->h[7] += h; } +#ifdef WITH_ARMV8_SHA2 +/* + * AArch64 can do four rounds at a time with the SHA-2 instructions; see + * sha256_armv8.c. They are optional in ARMv8.0, so ask before using them. + * Two threads racing to answer here both write the same value. + */ +extern void crypton_sha256_armv8_do_chunk(uint32_t state[8], const uint32_t buf[16]); +extern int crypton_sha256_armv8_available(void); + +static int sha256_use_armv8 = -1; +#endif + +#if (defined(WITH_ARMV8_SHA256_ASM) && defined(WITH_ARMV8_SHA2)) \ + || defined(WITH_X86_SHA256_ASM) +/* + * SHA-256 from CRYPTOGAMS, in cbits/asm/sha256-armv8-*.S and + * cbits/asm/sha256-x86_64-*.S, which take any number of blocks at once and + * schedule the instructions across them -- which is where they are ahead + * of the intrinsics above, the instructions being the same ones. Each + * picks its own path from the word the processor was asked about, so the + * answer to the runtime check goes there rather than into a branch here. + */ +#define SHA256_ASM 1 +#include "crypton_cpu.h" +extern void crypton_sha256_asm_block_data_order(uint32_t state[8], + const void *data, size_t blocks); + +static void sha256_asm_ready(void) +{ +#ifdef WITH_ARMV8_SHA256_ASM + if (sha256_use_armv8 < 0) { + if (crypton_sha256_armv8_available()) + crypton_armcap_P |= CRYPTON_ARMCAP_SHA256; + sha256_use_armv8 = 1; + } +#else + crypton_x86_ia32cap_resolve(); +#endif +} +#endif + + +static void sha256_do_chunk(struct sha256_ctx *ctx, uint32_t buf[]) +{ +#ifdef SHA256_ASM + sha256_asm_ready(); + crypton_sha256_asm_block_data_order(ctx->h, buf, 1); + return; +#endif +#if defined(WITH_ARMV8_SHA2) && !defined(SHA256_ASM) + if (sha256_use_armv8 < 0) + sha256_use_armv8 = crypton_sha256_armv8_available(); + if (sha256_use_armv8) { + crypton_sha256_armv8_do_chunk(ctx->h, buf); + return; + } +#endif + sha256_do_chunk_generic(ctx, buf); +} + void crypton_sha224_update(struct sha224_ctx *ctx, const uint8_t *data, uint32_t len) { return crypton_sha256_update(ctx, data, len); @@ -135,6 +198,19 @@ index = 0; } +#ifdef SHA256_ASM + /* the assembly reads the message a byte at a time as far as the + * machine is concerned, so it wants no alignment and no copy, and + * it takes the whole run of blocks in one call */ + if (len >= 64) { + size_t blocks = len / 64; + + sha256_asm_ready(); + crypton_sha256_asm_block_data_order(ctx->h, data, blocks); + data += blocks * 64; + len -= (uint32_t) blocks * 64; + } +#else if (need_alignment(data, 4)) { uint32_t tramp[16]; ASSERT_ALIGNMENT(tramp, 4); @@ -147,6 +223,7 @@ for (; len >= 64; len -= 64, data += 64) sha256_do_chunk(ctx, (uint32_t *) data); } +#endif /* append data into buf */ if (len) diff --git a/cbits/crypton_sha3.c b/cbits/crypton_sha3.c --- a/cbits/crypton_sha3.c +++ b/cbits/crypton_sha3.c @@ -50,6 +50,58 @@ static const int keccak_piln[24] = { 10,7,11,17,18,3,5,16,8,21,24,4,15,23,19,13,12,2,20,14,22,9,6,1 }; +/* + * AArch64 has instructions for this permutation; see sha3_armv8.c. They are + * an ARMv8.2 extension, so ask before using them. Two threads racing to + * answer here both write the same value. + */ +#ifdef WITH_ARMV8_SHA3 +extern void crypton_sha3_armv8_permute(uint64_t state[25]); +extern int crypton_sha3_armv8_available(void); + +static int sha3_use_armv8 = -1; + +/* Two threads racing to answer this both write the same value. */ +static int sha3_armv8_ok(void) +{ + if (sha3_use_armv8 < 0) + sha3_use_armv8 = crypton_sha3_armv8_available(); + return sha3_use_armv8; +} +#endif + +#if defined(WITH_ARMV8_SHA3_ASM) && !defined(__AARCH64EB__) +/* + * Keccak from CRYPTOGAMS, in cbits/asm/keccak1600-armv8-*.S, which takes a + * run of blocks rather than one at a time and schedules the instructions + * across the round it is in and the next. The instructions are the same + * ones the intrinsics beside it use; the arrangement is what is worth + * about a tenth here. It reads the message as bytes, so the run wants + * neither alignment nor a copy. + */ +#define SHA3_ASM 1 +/* the runtime question this file already asks decides whether it is used */ +#define SHA3_ASM_OPTIONAL 1 +extern size_t crypton_keccak_asm_absorb_cext(uint64_t state[25], const void *inp, + size_t len, size_t bsz); +#define sha3_asm_absorb crypton_keccak_asm_absorb_cext +#endif + +#ifdef WITH_X86_SHA3_ASM +/* + * And the same module for x86-64, where there are no instructions for this + * and what the assembly has over the C is the arrangement: the twenty-five + * lanes live in registers across a round, where a compiler given the C + * below spills them, and the rotations are folded into the operations that + * consume them. It needs nothing of the processor beyond the baseline, so + * unlike the AArch64 one it is used wherever it is compiled in. + */ +#define SHA3_ASM 1 +extern size_t crypton_keccak_asm_absorb(uint64_t state[25], const void *inp, + size_t len, size_t bsz); +#define sha3_asm_absorb crypton_keccak_asm_absorb +#endif + static inline void sha3_do_chunk(uint64_t state[25], uint64_t buf[], int bufsz) { int i, j, r; @@ -59,6 +111,13 @@ for (i = 0; i < bufsz; i++) state[i] ^= le64_to_cpu(buf[i]); +#ifdef WITH_ARMV8_SHA3 + if (sha3_armv8_ok()) { + crypton_sha3_armv8_permute(state); + return; + } +#endif + /* run keccak rounds */ for (r = 0; r < KECCAK_NB_ROUNDS; r++) { /* compute the parity of each columns */ @@ -133,6 +192,20 @@ data += to_fill; ctx->bufindex = 0; } + +#ifdef SHA3_ASM + if (len >= ctx->bufsz +#ifdef SHA3_ASM_OPTIONAL + && sha3_armv8_ok() +#endif + ) { + const size_t left = sha3_asm_absorb(ctx->state, data, len, + ctx->bufsz); + + data += len - left; + len = (uint32_t) left; + } +#endif if (need_alignment(data, 8)) { uint64_t tramp[SHA3_BUF_SIZE_MAX/8]; diff --git a/cbits/crypton_sha512.c b/cbits/crypton_sha512.c --- a/cbits/crypton_sha512.c +++ b/cbits/crypton_sha512.c @@ -91,7 +91,7 @@ #define s0(x) (ror64(x, 1) ^ ror64(x, 8) ^ (x >> 7)) #define s1(x) (ror64(x, 19) ^ ror64(x, 61) ^ (x >> 6)) -static void sha512_do_chunk(struct sha512_ctx *ctx, uint64_t *buf) +static void sha512_do_chunk_generic(struct sha512_ctx *ctx, uint64_t *buf) { uint64_t a, b, c, d, e, f, g, h, t1, t2; int i; @@ -128,6 +128,50 @@ ctx->h[4] += e; ctx->h[5] += f; ctx->h[6] += g; ctx->h[7] += h; } +#ifdef WITH_ARMV8_SHA512 +/* + * AArch64 can do two rounds at a time with the SHA-512 instructions; see + * sha512_armv8.c. They are an optional ARMv8.2 extension and much less + * widespread than the SHA-256 ones, so ask before using them. Two threads + * racing to answer here both write the same value. + */ +extern void crypton_sha512_armv8_do_chunk(uint64_t state[8], const uint64_t buf[16]); +extern int crypton_sha512_armv8_available(void); + +static int sha512_use_armv8 = -1; +#endif + + +#ifdef WITH_X86_SHA512_ASM +/* + * SHA-512 from CRYPTOGAMS, in cbits/asm/sha512-x86_64-*.S, which takes any + * number of blocks at once and schedules across them, and picks between + * AVX2, AVX, SSSE3 and plain integer code from crypton_ia32cap_P. + */ +#define SHA512_ASM 1 +#include "crypton_cpu.h" +extern void crypton_sha512_asm_block_data_order(uint64_t state[8], + const void *data, size_t blocks); +#endif + +static void sha512_do_chunk(struct sha512_ctx *ctx, uint64_t *buf) +{ +#ifdef SHA512_ASM + crypton_x86_ia32cap_resolve(); + crypton_sha512_asm_block_data_order(ctx->h, buf, 1); + return; +#endif +#ifdef WITH_ARMV8_SHA512 + if (sha512_use_armv8 < 0) + sha512_use_armv8 = crypton_sha512_armv8_available(); + if (sha512_use_armv8) { + crypton_sha512_armv8_do_chunk(ctx->h, buf); + return; + } +#endif + sha512_do_chunk_generic(ctx, buf); +} + void crypton_sha384_update(struct sha384_ctx *ctx, const uint8_t *data, uint32_t len) { return crypton_sha512_update(ctx, data, len); @@ -154,6 +198,18 @@ index = 0; } +#ifdef SHA512_ASM + /* the assembly reads the message as bytes, so it wants neither the + * alignment nor the copy, and takes the whole run in one call */ + if (len >= 128) { + size_t blocks = len / 128; + + crypton_x86_ia32cap_resolve(); + crypton_sha512_asm_block_data_order(ctx->h, data, blocks); + data += blocks * 128; + len -= (uint32_t) blocks * 128; + } +#else if (need_alignment(data, 8)) { uint64_t tramp[16]; ASSERT_ALIGNMENT(tramp, 8); @@ -166,6 +222,7 @@ for (; len >= 128; len -= 128, data += 128) sha512_do_chunk(ctx, (uint64_t *) data); } +#endif /* append data into buf */ if (len) diff --git a/cbits/ed25519/ed25519.c b/cbits/ed25519/ed25519.c --- a/cbits/ed25519/ed25519.c +++ b/cbits/ed25519/ed25519.c @@ -89,7 +89,7 @@ ge25519 ALIGN(16) R, A; hash_512bits hash; bignum256modm hram, S; - unsigned char checkR[32]; + unsigned char checkR[32], checkS[32]; if ((RS[63] & 224) || !ge25519_unpack_negative_vartime(&A, pk)) return -1; @@ -100,6 +100,11 @@ /* S */ expand256_modm(S, RS + 32, 32); + + /* check that S is canonical */ + contract256_modm(checkS, S); + if (!ed25519_verify(RS + 32, checkS, 32)) + return -1; /* SB - H(R,A,m)A */ ge25519_double_scalarmult_vartime(&R, &A, hram, S); diff --git a/cbits/include64/p256/p256_gf.h b/cbits/include64/p256/p256_gf.h --- a/cbits/include64/p256/p256_gf.h +++ b/cbits/include64/p256/p256_gf.h @@ -63,6 +63,38 @@ #define NLIMBS 5 typedef limb felem[NLIMBS]; +/* On AArch64, the three functions that do the field arithmetic are asked to + * be inlined rather than left for the compiler to decide. + * + * felem_mul and felem_square end in felem_reduce_degree, a carry chain the + * whole width of the number, and that chain is what their latency is: one + * product feeding the next costs 18.1 ns on an Apple M4, while four + * independent ones cost 11.4 ns each. The curve arithmetic has independent + * products to offer -- the two squarings that open a point doubling, the + * multiplication and the squaring that close it -- but only if the compiler + * can see one reduction while the other is still going. Left alone it emits + * felem_reduce_degree once and calls it, and a call is a fence: the two + * chains cannot overlap. Plain `inline` does not change its mind. + * + * Asking costs code: this file's object goes from 30 to 116 kilobytes. That + * is worth it where there are registers to hold two chains at once and not + * where there are not, which is the architecture talking rather than the + * compiler. Measured on a variable-point scalar multiplication: + * + * Apple M4, Apple clang 21 1.23x + * Neoverse, clang 18 1.12x + * Neoverse, gcc 13 1.05x + * EPYC 7763, clang 18 0.95x + * Xeon 8370C, gcc 13 0.82x + * + * so x86-64 keeps the compiler's own judgement. + */ +#if defined(__aarch64__) && (defined(__GNUC__) || defined(__clang__)) +#define FELEM_INLINE static inline __attribute__((always_inline)) +#else +#define FELEM_INLINE static +#endif + static const limb kBottom51Bits = 0x7ffffffffffff; static const limb kBottom52Bits = 0xfffffffffffff; @@ -278,7 +310,7 @@ * * On entry: tmp[i] < 2**128 * On exit: out[0,2,...] < 2**52, out[1,3,...] < 2**53 */ -static void felem_reduce_degree(felem out, u128 tmp[9]) { +FELEM_INLINE void felem_reduce_degree(felem out, u128 tmp[9]) { /* The following table may be helpful when reading this code: * * Limb number: 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 @@ -468,7 +500,7 @@ * * On entry: in[0,2,...] < 2**52, in[1,3,...] < 2**53. * On exit: out[0,2,...] < 2**52, out[1,3,...] < 2**53. */ -static void felem_square(felem out, const felem in) { +FELEM_INLINE void felem_square(felem out, const felem in) { u128 tmp[9], x1x1, x3x3; x1x1 = ((u128) in[1]) * in[1]; @@ -496,7 +528,7 @@ * On entry: in[0,2,...] < 2**52, in[1,3,...] < 2**53 and * in2[0,2,...] < 2**52, in2[1,3,...] < 2**53. * On exit: out[0,2,...] < 2**52, out[1,3,...] < 2**53. */ -static void felem_mul(felem out, const felem in, const felem in2) { +FELEM_INLINE void felem_mul(felem out, const felem in, const felem in2) { u128 tmp[9], x1y1, x1y3, x3y1, x3y3; x1y1 = ((u128) in[1]) * in2[1]; diff --git a/cbits/p256/p256.c b/cbits/p256/p256.c --- a/cbits/p256/p256.c +++ b/cbits/p256/p256.c @@ -352,13 +352,12 @@ } // Verify y^2 == x^3 - 3x + b mod p -// and 0 < x < p and 0 < y < p +// and 0 <= x < p and 0 < y < p int crypton_p256_is_valid_point(const crypton_p256_int* x, const crypton_p256_int* y) { crypton_p256_int y2, x3; if (crypton_p256_cmp(&crypton_SECP256r1_p, x) <= 0 || crypton_p256_cmp(&crypton_SECP256r1_p, y) <= 0 || - crypton_p256_is_zero(x) || crypton_p256_is_zero(y)) return 0; crypton_p256_modmul(&crypton_SECP256r1_p, y, 0, y, &y2); // y^2 @@ -370,6 +369,7 @@ if (crypton_p256_sub(&x3, x, &x3)) crypton_p256_add(&x3, &crypton_SECP256r1_p, &x3); // x^3 - 3x if (crypton_p256_add(&x3, &crypton_SECP256r1_b, &x3)) // x^3 - 3x + b crypton_p256_sub(&x3, &crypton_SECP256r1_p, &x3); + crypton_p256_mod(&crypton_SECP256r1_p, &x3, &x3); return crypton_p256_cmp(&y2, &x3) == 0; } diff --git a/cbits/sha1_armv8.c b/cbits/sha1_armv8.c new file mode 100644 --- /dev/null +++ b/cbits/sha1_armv8.c @@ -0,0 +1,173 @@ +/* + * SHA-1 using the ARMv8-A cryptographic extensions. + * + * crypton_sha1.c computes the compression function a round at a time in plain + * C. AArch64 has instructions for it -- SHA1C, SHA1P, SHA1M, SHA1H, SHA1SU0 + * and SHA1SU1 -- which do four rounds at a time and most of the message + * schedule alongside. They come with the SHA-256 ones this tree already uses, + * under the same optional feature, so anything that has those has these. + * + * SHA-1 is not a hash to choose today, but it is still what a number of + * protocols and file formats ask for. + */ + +#include +#include +#if defined(__linux__) +#include +#include +#endif + +/* + * The instructions are an extension, so a translation unit compiled for + * baseline ARMv8-A may not use them; see sha256_armv8.c for the whole of that + * argument. + */ +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_ARMV8_CRYPTO __attribute__((target("+crypto"))) +#else +#define TARGET_ARMV8_CRYPTO +#endif + +/* + * A group of four rounds, and the schedule that goes with it. + * + * SHA1C, SHA1P and SHA1M each do four rounds with one of the three round + * functions -- choose, parity and majority -- taking the four state words in + * a register, E in a general one, and the four message words with their round + * constant already added. SHA1H is the rotation of A by thirty that carries + * E from one group to the next. + * + * The schedule is the exclusive or of four earlier words rotated left by one. + * SHA1SU0 does the three terms that reach furthest back and SHA1SU1 the last + * one, together with the rotation and the dependency inside the group. + */ +#define GROUP(f, ecur, enext, wk_cur, wk_next, kk, w0, w1, w2, w3) \ + do { \ + enext = vsha1h_u32(vgetq_lane_u32(abcd, 0)); \ + abcd = f(abcd, ecur, wk_cur); \ + wk_next = vaddq_u32(w2, kk); \ + w0 = vsha1su0q_u32(w0, w1, w2); \ + w3 = vsha1su1q_u32(w3, w2); \ + } while (0) + +/* + * One 64-byte block. `state` is the five words of chaining value in host + * order, `buf` the block as it arrived, which SHA-1 reads big-endian. + */ +TARGET_ARMV8_CRYPTO +void crypton_sha1_armv8_do_chunks(uint32_t state[5], const uint8_t *data, + uint32_t blocks) +{ + const uint32x4_t k0 = vdupq_n_u32(0x5a827999); + const uint32x4_t k1 = vdupq_n_u32(0x6ed9eba1); + const uint32x4_t k2 = vdupq_n_u32(0x8f1bbcdc); + const uint32x4_t k3 = vdupq_n_u32(0xca62c1d6); + uint32x4_t abcd, abcd_prev; + uint32x4_t m0, m1, m2, m3; + uint32x4_t wk0, wk1; + uint32_t e0, e1, e_prev; + + abcd = vld1q_u32(state); + e0 = state[4]; + + for (; blocks > 0; blocks--, data += 64) { + const uint32_t *buf = (const uint32_t *) data; + + abcd_prev = abcd; + e_prev = e0; + + m0 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf)))); + m1 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf + 4)))); + m2 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf + 8)))); + m3 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf + 12)))); + + wk0 = vaddq_u32(m0, k0); + wk1 = vaddq_u32(m1, k0); + + /* rounds 0 to 15, where the schedule has less to do each group until + * it is running a whole group ahead */ + e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1cq_u32(abcd, e0, wk0); + wk0 = vaddq_u32(m2, k0); + m0 = vsha1su0q_u32(m0, m1, m2); + + e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1cq_u32(abcd, e1, wk1); + wk1 = vaddq_u32(m3, k0); + m1 = vsha1su0q_u32(m1, m2, m3); + m0 = vsha1su1q_u32(m0, m3); + + e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1cq_u32(abcd, e0, wk0); + wk0 = vaddq_u32(m0, k0); + m2 = vsha1su0q_u32(m2, m3, m0); + m1 = vsha1su1q_u32(m1, m0); + + e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1cq_u32(abcd, e1, wk1); + wk1 = vaddq_u32(m1, k1); + m3 = vsha1su0q_u32(m3, m0, m1); + m2 = vsha1su1q_u32(m2, m1); + + /* rounds 16 to 19, still the choose function, and then twenty of each + * of the others; the message registers come back to the same roles + * every fourth group */ + GROUP(vsha1cq_u32, e0, e1, wk0, wk0, k1, m0, m1, m2, m3); + GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k1, m1, m2, m3, m0); + GROUP(vsha1pq_u32, e0, e1, wk0, wk0, k1, m2, m3, m0, m1); + GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k1, m3, m0, m1, m2); + GROUP(vsha1pq_u32, e0, e1, wk0, wk0, k2, m0, m1, m2, m3); + GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k2, m1, m2, m3, m0); + GROUP(vsha1mq_u32, e0, e1, wk0, wk0, k2, m2, m3, m0, m1); + GROUP(vsha1mq_u32, e1, e0, wk1, wk1, k2, m3, m0, m1, m2); + GROUP(vsha1mq_u32, e0, e1, wk0, wk0, k2, m0, m1, m2, m3); + GROUP(vsha1mq_u32, e1, e0, wk1, wk1, k3, m1, m2, m3, m0); + GROUP(vsha1mq_u32, e0, e1, wk0, wk0, k3, m2, m3, m0, m1); + GROUP(vsha1pq_u32, e1, e0, wk1, wk1, k3, m3, m0, m1, m2); + + /* rounds 64 to 79, where the schedule runs out a piece at a time */ + e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1pq_u32(abcd, e0, wk0); + wk0 = vaddq_u32(m2, k3); + m3 = vsha1su1q_u32(m3, m2); + + e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1pq_u32(abcd, e1, wk1); + wk1 = vaddq_u32(m3, k3); + + e1 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1pq_u32(abcd, e0, wk0); + + e0 = vsha1h_u32(vgetq_lane_u32(abcd, 0)); + abcd = vsha1pq_u32(abcd, e1, wk1); + + abcd = vaddq_u32(abcd, abcd_prev); + e0 += e_prev; + } + + vst1q_u32(state, abcd); + state[4] = e0; +} + +/* the one-block form, for the partial block a message ends with */ +void crypton_sha1_armv8_do_chunk(uint32_t state[5], const uint32_t buf[16]) +{ + crypton_sha1_armv8_do_chunks(state, (const uint8_t *) buf, 1); +} + +/* + * The SHA-1 instructions are optional in ARMv8.0, and arrive with the SHA-256 + * ones. They are always there on Apple silicon; elsewhere the kernel reports + * them. + */ +int crypton_sha1_armv8_available(void) +{ +#if defined(__APPLE__) + return 1; +#elif defined(__linux__) + return (getauxval(AT_HWCAP) & HWCAP_SHA1) != 0; +#else + return 0; +#endif +} diff --git a/cbits/sha1_x86.c b/cbits/sha1_x86.c new file mode 100644 --- /dev/null +++ b/cbits/sha1_x86.c @@ -0,0 +1,174 @@ +/* + * SHA-1 using the Intel SHA extensions. + * + * crypton_sha1.c computes the compression function a round at a time in plain + * C. The same extension that carries SHA256RNDS2 carries four instructions + * for this one -- SHA1RNDS4, SHA1NEXTE, SHA1MSG1 and SHA1MSG2 -- which do four + * rounds at a time and most of the message schedule alongside. + * + * SHA-1 is not a hash to choose today, but it is still what a number of + * protocols and file formats ask for, and the instructions are already there + * on any processor that has the SHA-256 ones. + */ + +#include +#include +#include "crypton_cpu.h" + +/* + * The instructions are an extension, so a translation unit compiled for the + * x86-64 baseline may not use them; see cbits/sha256_x86.c for the whole of + * that argument. SSE4.1 and SSSE3 come along for the same reasons there. + */ +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_X86_SHA __attribute__((target("sha,sse4.1,ssse3"))) +#else +#define TARGET_X86_SHA +#endif + +/* + * A group of four rounds, and the schedule that goes with it. + * + * SHA1RNDS4 takes the four state words in one register -- A in the top lane, + * which is why both the state and each block are loaded reversed -- and the + * four message words with E already added into the first, which is what + * SHA1NEXTE produces from the state as it stood four rounds ago. The round + * function and constant come from the immediate: 0 for rounds 0 to 19, then + * one per twenty. + * + * The schedule is the exclusive or of four earlier words rotated left by one. + * SHA1MSG1 does the part that reaches furthest back, the exclusive or with + * the word eight before is an ordinary one, and SHA1MSG2 does the last part + * together with the rotation and the dependency inside the group of four. + */ +#define GROUP(imm, ecur, enext, w0, w1, w2, w3) \ + do { \ + ecur = _mm_sha1nexte_epu32(ecur, w0); \ + enext = abcd; \ + w1 = _mm_sha1msg2_epu32(w1, w0); \ + abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm); \ + w3 = _mm_sha1msg1_epu32(w3, w0); \ + w2 = _mm_xor_si128(w2, w0); \ + } while (0) + +/* the same without the part of the schedule that has run out */ +#define GROUP_NOMSG1(imm, ecur, enext, w0, w1, w2) \ + do { \ + ecur = _mm_sha1nexte_epu32(ecur, w0); \ + enext = abcd; \ + w1 = _mm_sha1msg2_epu32(w1, w0); \ + abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm); \ + w2 = _mm_xor_si128(w2, w0); \ + } while (0) + +#define GROUP_MSG2(imm, ecur, enext, w0, w1) \ + do { \ + ecur = _mm_sha1nexte_epu32(ecur, w0); \ + enext = abcd; \ + w1 = _mm_sha1msg2_epu32(w1, w0); \ + abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm); \ + } while (0) + +#define GROUP_ROUNDS(imm, ecur, enext, w0) \ + do { \ + ecur = _mm_sha1nexte_epu32(ecur, w0); \ + enext = abcd; \ + abcd = _mm_sha1rnds4_epu32(abcd, ecur, imm); \ + } while (0) + +/* + * Any number of 64-byte blocks. `state` is the five words of chaining + * value in host order, `data` the blocks as they arrived, which SHA-1 + * reads big-endian. + * + * The state stays in registers from one block to the next. Taking them a + * block at a time, which is what this did, spends a load, a store and two + * shuffles either side of every block putting state back where it already + * was, and against the hundred-odd cycles a block costs with these + * instructions that is worth having. + */ +TARGET_X86_SHA +void crypton_sha1_x86_do_chunks(uint32_t state[5], const uint8_t *data, + uint32_t blocks) +{ + /* the whole register reversed, which byte-swaps each word and puts + * the first of them in the top lane */ + const __m128i bswap = _mm_setr_epi8(15, 14, 13, 12, 11, 10, 9, 8, + 7, 6, 5, 4, 3, 2, 1, 0); + __m128i abcd, e0, e1, abcd_prev, e_prev; + __m128i m0, m1, m2, m3; + + abcd = _mm_shuffle_epi32(_mm_loadu_si128((const __m128i *) state), 0x1b); + e0 = _mm_set_epi32((int) state[4], 0, 0, 0); + + for (; blocks > 0; blocks--, data += 64) { + const uint32_t *buf = (const uint32_t *) data; + + abcd_prev = abcd; + e_prev = e0; + + m0 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) buf), bswap); + m1 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) (buf + 4)), bswap); + m2 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) (buf + 8)), bswap); + m3 = _mm_shuffle_epi8(_mm_loadu_si128((const __m128i *) (buf + 12)), bswap); + + /* rounds 0 to 15, where the schedule has nothing to extend yet: the + * first group takes E by an ordinary addition rather than SHA1NEXTE, + * there being no state from four rounds ago */ + e0 = _mm_add_epi32(e0, m0); + e1 = abcd; + abcd = _mm_sha1rnds4_epu32(abcd, e0, 0); + + e1 = _mm_sha1nexte_epu32(e1, m1); + e0 = abcd; + abcd = _mm_sha1rnds4_epu32(abcd, e1, 0); + m0 = _mm_sha1msg1_epu32(m0, m1); + + e0 = _mm_sha1nexte_epu32(e0, m2); + e1 = abcd; + abcd = _mm_sha1rnds4_epu32(abcd, e0, 0); + m1 = _mm_sha1msg1_epu32(m1, m2); + m0 = _mm_xor_si128(m0, m2); + + GROUP(0, e1, e0, m3, m0, m1, m2); + + /* rounds 16 to 63, where every group both hashes and schedules; the + * four message registers come back to the same roles every fourth + * group, and the round function changes every twentieth round */ + GROUP(0, e0, e1, m0, m1, m2, m3); + GROUP(1, e1, e0, m1, m2, m3, m0); + GROUP(1, e0, e1, m2, m3, m0, m1); + GROUP(1, e1, e0, m3, m0, m1, m2); + GROUP(1, e0, e1, m0, m1, m2, m3); + GROUP(1, e1, e0, m1, m2, m3, m0); + GROUP(2, e0, e1, m2, m3, m0, m1); + GROUP(2, e1, e0, m3, m0, m1, m2); + GROUP(2, e0, e1, m0, m1, m2, m3); + GROUP(2, e1, e0, m1, m2, m3, m0); + GROUP(2, e0, e1, m2, m3, m0, m1); + GROUP(3, e1, e0, m3, m0, m1, m2); + + /* rounds 64 to 79, where the schedule runs out a piece at a time. The + * first of these still extends: the part of the last four words that + * reaches sixteen back is taken here, three groups before they are + * finished */ + GROUP(3, e0, e1, m0, m1, m2, m3); + GROUP_NOMSG1(3, e1, e0, m1, m2, m3); + GROUP_MSG2(3, e0, e1, m2, m3); + GROUP_ROUNDS(3, e1, e0, m3); + + /* and the chaining value, E through the same instruction that would + * have carried it into a fifth round */ + e0 = _mm_sha1nexte_epu32(e0, e_prev); + abcd = _mm_add_epi32(abcd, abcd_prev); + } + + _mm_storeu_si128((__m128i *) state, _mm_shuffle_epi32(abcd, 0x1b)); + state[4] = (uint32_t) _mm_extract_epi32(e0, 3); +} + +/* the one-block form, for the partial block a message ends with */ +void crypton_sha1_x86_do_chunk(uint32_t state[5], const uint32_t buf[16]) +{ + crypton_sha1_x86_do_chunks(state, (const uint8_t *) buf, 1); +} diff --git a/cbits/sha256_armv8.c b/cbits/sha256_armv8.c new file mode 100644 --- /dev/null +++ b/cbits/sha256_armv8.c @@ -0,0 +1,145 @@ +/* + * SHA-256 using the ARMv8-A cryptographic extensions. + * + * crypton_sha256.c computes the compression function a round at a time in + * plain C. AArch64 has instructions for it -- SHA256H, SHA256H2, SHA256SU0 + * and SHA256SU1 -- which do four rounds at a time and compute the message + * schedule alongside. This provides that version; crypton_sha256.c picks + * between the two at runtime. + * + * SHA-224 shares the compression function, so it comes along for free. + */ + +#include +#include +#if defined(__linux__) +#include +#include +#endif + +/* + * The SHA-2 instructions are an extension, so a translation unit compiled for + * baseline ARMv8-A may not use them. Mark the function that does, the way + * cbits/aes/x86ni.h marks its x86 counterparts, rather than raising + * -march for every file in the library: the flag use_target_attributes picks + * between the two, and with it set -- which is the default -- nothing else + * enables the extensions, so without these the file does not compile at all on + * a toolchain whose baseline lacks them. Apple's does not lack them, which is + * why only Linux noticed. + * + * "+crypto" rather than "crypto": GCC rejects the latter. + */ +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_ARMV8_CRYPTO __attribute__((target("+crypto"))) +#else +#define TARGET_ARMV8_CRYPTO +#endif + +static const uint32_t K[64] = { + 0x428a2f98, 0x71374491, 0xb5c0fbcf, 0xe9b5dba5, + 0x3956c25b, 0x59f111f1, 0x923f82a4, 0xab1c5ed5, + 0xd807aa98, 0x12835b01, 0x243185be, 0x550c7dc3, + 0x72be5d74, 0x80deb1fe, 0x9bdc06a7, 0xc19bf174, + 0xe49b69c1, 0xefbe4786, 0x0fc19dc6, 0x240ca1cc, + 0x2de92c6f, 0x4a7484aa, 0x5cb0a9dc, 0x76f988da, + 0x983e5152, 0xa831c66d, 0xb00327c8, 0xbf597fc7, + 0xc6e00bf3, 0xd5a79147, 0x06ca6351, 0x14292967, + 0x27b70a85, 0x2e1b2138, 0x4d2c6dfc, 0x53380d13, + 0x650a7354, 0x766a0abb, 0x81c2c92e, 0x92722c85, + 0xa2bfe8a1, 0xa81a664b, 0xc24b8b70, 0xc76c51a3, + 0xd192e819, 0xd6990624, 0xf40e3585, 0x106aa070, + 0x19a4c116, 0x1e376c08, 0x2748774c, 0x34b0bcb5, + 0x391c0cb3, 0x4ed8aa4a, 0x5b9cca4f, 0x682e6ff3, + 0x748f82ee, 0x78a5636f, 0x84c87814, 0x8cc70208, + 0x90befffa, 0xa4506ceb, 0xbef9a3f7, 0xc67178f2, +}; + +/* + * One 64-byte block. `state` is the eight words of chaining value in host + * order, `buf` the block as it arrived, which SHA-256 reads big-endian. + */ +TARGET_ARMV8_CRYPTO +void crypton_sha256_armv8_do_chunk(uint32_t state[8], const uint32_t buf[16]) +{ + uint32x4_t abcd, efgh, abcd_prev, efgh_prev, abcd_save, tmp; + uint32x4_t m0, m1, m2, m3; + int i; + + abcd_prev = abcd = vld1q_u32(state); + efgh_prev = efgh = vld1q_u32(state + 4); + + m0 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf)))); + m1 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf + 4)))); + m2 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf + 8)))); + m3 = vreinterpretq_u32_u8(vrev32q_u8(vreinterpretq_u8_u32(vld1q_u32(buf + 12)))); + + /* twelve groups of four rounds that also extend the schedule ... */ + for (i = 0; i < 48; i += 16) { + uint32x4_t n0, n1, n2, n3; + + n0 = vsha256su1q_u32(vsha256su0q_u32(m0, m1), m2, m3); + tmp = vaddq_u32(m0, vld1q_u32(&K[i])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + n1 = vsha256su1q_u32(vsha256su0q_u32(m1, m2), m3, n0); + tmp = vaddq_u32(m1, vld1q_u32(&K[i + 4])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + n2 = vsha256su1q_u32(vsha256su0q_u32(m2, m3), n0, n1); + tmp = vaddq_u32(m2, vld1q_u32(&K[i + 8])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + n3 = vsha256su1q_u32(vsha256su0q_u32(m3, n0), n1, n2); + tmp = vaddq_u32(m3, vld1q_u32(&K[i + 12])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + m0 = n0; m1 = n1; m2 = n2; m3 = n3; + } + + /* ... and the last four, where there is no more schedule to extend */ + tmp = vaddq_u32(m0, vld1q_u32(&K[48])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + tmp = vaddq_u32(m1, vld1q_u32(&K[52])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + tmp = vaddq_u32(m2, vld1q_u32(&K[56])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + tmp = vaddq_u32(m3, vld1q_u32(&K[60])); + abcd_save = abcd; + abcd = vsha256hq_u32(abcd, efgh, tmp); + efgh = vsha256h2q_u32(efgh, abcd_save, tmp); + + vst1q_u32(state, vaddq_u32(abcd, abcd_prev)); + vst1q_u32(state + 4, vaddq_u32(efgh, efgh_prev)); +} + +/* + * The SHA-2 instructions are optional in ARMv8.0. They are always there on + * Apple silicon; elsewhere the kernel reports them. + */ +int crypton_sha256_armv8_available(void) +{ +#if defined(__APPLE__) + return 1; +#elif defined(__linux__) + return (getauxval(AT_HWCAP) & HWCAP_SHA2) != 0; +#else + return 0; +#endif +} diff --git a/cbits/sha3_armv8.c b/cbits/sha3_armv8.c new file mode 100644 --- /dev/null +++ b/cbits/sha3_armv8.c @@ -0,0 +1,176 @@ +/* + * Keccak-f[1600] using the ARMv8.2 SHA-3 instructions. + * + * crypton_sha3.c runs the permutation in plain C, a round at a time over a + * table of rotation amounts and lane positions. AArch64 has four + * instructions that exist for exactly this: + * + * EOR3 a ^ b ^ c the column parities of theta + * RAX1 a ^ ROL(b, 1) the rest of theta + * XAR ROR(a ^ b, n) theta's exclusive or and rho's rotation at once + * BCAX a ^ (b & ~c) chi + * + * They work on 128-bit registers and the permutation has twenty-five 64-bit + * lanes, so each lane sits in the low half of a register and the high half + * goes unused. Rho and pi move one lane of every row into every other row, + * so the round cannot be done in place: the twenty-five rotated words are + * computed first and chi then writes the state from them. + * + * The body is generated from the definitions in FIPS 202 rather than copied + * in: the rotation amounts are the triangular numbers modulo 64, pi sends + * lane (x, y) to (y, 2x + 3y), and the script that worked those out checked + * the result against the published digests of the empty string and of "abc" + * before emitting any of this. + */ + +#include +#include +#if defined(__APPLE__) +#include +#elif defined(__linux__) +#include +#include +#endif + +/* + * The SHA-3 instructions are an ARMv8.2 extension, so a translation unit + * compiled for the baseline may not use them; see sha256_armv8.c for the whole + * of that argument. The flag for a build without attributes already asks for + * "+sha3", which the SHA-512 path needed. + */ +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_ARMV8_SHA3 __attribute__((target("+sha3"))) +#else +#define TARGET_ARMV8_SHA3 +#endif + +static const uint64_t rc[24] = { + 0x0000000000000001ULL, 0x0000000000008082ULL, 0x800000000000808aULL, + 0x8000000080008000ULL, 0x000000000000808bULL, 0x0000000080000001ULL, + 0x8000000080008081ULL, 0x8000000000008009ULL, 0x000000000000008aULL, + 0x0000000000000088ULL, 0x0000000080008009ULL, 0x000000008000000aULL, + 0x000000008000808bULL, 0x800000000000008bULL, 0x8000000000008089ULL, + 0x8000000000008003ULL, 0x8000000000008002ULL, 0x8000000000000080ULL, + 0x000000000000800aULL, 0x800000008000000aULL, 0x8000000080008081ULL, + 0x8000000000008080ULL, 0x0000000080000001ULL, 0x8000000080008008ULL, +}; + +#define ROUND(k) \ + do { \ + /* theta: the parity of each column, and what it adds */ \ + c[0] = veor3q_u64(a[0], a[5], a[10]); \ + c[0] = veor3q_u64(c[0], a[15], a[20]); \ + c[1] = veor3q_u64(a[1], a[6], a[11]); \ + c[1] = veor3q_u64(c[1], a[16], a[21]); \ + c[2] = veor3q_u64(a[2], a[7], a[12]); \ + c[2] = veor3q_u64(c[2], a[17], a[22]); \ + c[3] = veor3q_u64(a[3], a[8], a[13]); \ + c[3] = veor3q_u64(c[3], a[18], a[23]); \ + c[4] = veor3q_u64(a[4], a[9], a[14]); \ + c[4] = veor3q_u64(c[4], a[19], a[24]); \ + d[0] = vrax1q_u64(c[4], c[1]); \ + d[1] = vrax1q_u64(c[0], c[2]); \ + d[2] = vrax1q_u64(c[1], c[3]); \ + d[3] = vrax1q_u64(c[2], c[4]); \ + d[4] = vrax1q_u64(c[3], c[0]); \ + /* theta's exclusive or, rho's rotation and pi's move, in one */ \ + b[0 ] = veorq_u64(a[0 ], d[0]); \ + b[1 ] = vxarq_u64(a[6 ], d[1], 20); \ + b[2 ] = vxarq_u64(a[12], d[2], 21); \ + b[3 ] = vxarq_u64(a[18], d[3], 43); \ + b[4 ] = vxarq_u64(a[24], d[4], 50); \ + b[5 ] = vxarq_u64(a[3 ], d[3], 36); \ + b[6 ] = vxarq_u64(a[9 ], d[4], 44); \ + b[7 ] = vxarq_u64(a[10], d[0], 61); \ + b[8 ] = vxarq_u64(a[16], d[1], 19); \ + b[9 ] = vxarq_u64(a[22], d[2], 3); \ + b[10] = vxarq_u64(a[1 ], d[1], 63); \ + b[11] = vxarq_u64(a[7 ], d[2], 58); \ + b[12] = vxarq_u64(a[13], d[3], 39); \ + b[13] = vxarq_u64(a[19], d[4], 56); \ + b[14] = vxarq_u64(a[20], d[0], 46); \ + b[15] = vxarq_u64(a[4 ], d[4], 37); \ + b[16] = vxarq_u64(a[5 ], d[0], 28); \ + b[17] = vxarq_u64(a[11], d[1], 54); \ + b[18] = vxarq_u64(a[17], d[2], 49); \ + b[19] = vxarq_u64(a[23], d[3], 8); \ + b[20] = vxarq_u64(a[2 ], d[2], 2); \ + b[21] = vxarq_u64(a[8 ], d[3], 9); \ + b[22] = vxarq_u64(a[14], d[4], 25); \ + b[23] = vxarq_u64(a[15], d[0], 23); \ + b[24] = vxarq_u64(a[21], d[1], 62); \ + /* chi, along each row */ \ + a[0 ] = vbcaxq_u64(b[0 ], b[2 ], b[1 ]); \ + a[1 ] = vbcaxq_u64(b[1 ], b[3 ], b[2 ]); \ + a[2 ] = vbcaxq_u64(b[2 ], b[4 ], b[3 ]); \ + a[3 ] = vbcaxq_u64(b[3 ], b[0 ], b[4 ]); \ + a[4 ] = vbcaxq_u64(b[4 ], b[1 ], b[0 ]); \ + a[5 ] = vbcaxq_u64(b[5 ], b[7 ], b[6 ]); \ + a[6 ] = vbcaxq_u64(b[6 ], b[8 ], b[7 ]); \ + a[7 ] = vbcaxq_u64(b[7 ], b[9 ], b[8 ]); \ + a[8 ] = vbcaxq_u64(b[8 ], b[5 ], b[9 ]); \ + a[9 ] = vbcaxq_u64(b[9 ], b[6 ], b[5 ]); \ + a[10] = vbcaxq_u64(b[10], b[12], b[11]); \ + a[11] = vbcaxq_u64(b[11], b[13], b[12]); \ + a[12] = vbcaxq_u64(b[12], b[14], b[13]); \ + a[13] = vbcaxq_u64(b[13], b[10], b[14]); \ + a[14] = vbcaxq_u64(b[14], b[11], b[10]); \ + a[15] = vbcaxq_u64(b[15], b[17], b[16]); \ + a[16] = vbcaxq_u64(b[16], b[18], b[17]); \ + a[17] = vbcaxq_u64(b[17], b[19], b[18]); \ + a[18] = vbcaxq_u64(b[18], b[15], b[19]); \ + a[19] = vbcaxq_u64(b[19], b[16], b[15]); \ + a[20] = vbcaxq_u64(b[20], b[22], b[21]); \ + a[21] = vbcaxq_u64(b[21], b[23], b[22]); \ + a[22] = vbcaxq_u64(b[22], b[24], b[23]); \ + a[23] = vbcaxq_u64(b[23], b[20], b[24]); \ + a[24] = vbcaxq_u64(b[24], b[21], b[20]); \ + /* iota */ \ + a[0] = veorq_u64(a[0], vld1q_dup_u64(&rc[k])); \ + } while (0) + +/* the twenty-four rounds over the state, in place */ +TARGET_ARMV8_SHA3 +void crypton_sha3_armv8_permute(uint64_t state[25]) +{ + uint64x2_t a[25], b[25], c[5], d[5]; + int i, round; + + for (i = 0; i < 25; i++) + a[i] = vld1q_dup_u64(&state[i]); + + /* four rounds to an iteration: a round is a chain -- the column + * parities wait for the last chi of the round before -- so giving the + * processor more than one of them to look at is worth something. One + * round an iteration measured 802 MB/s of SHA3-256, two 949 and four + * 991, against 551 for the plain C */ + for (round = 0; round < 24; round += 4) { + ROUND(round); + ROUND(round + 1); + ROUND(round + 2); + ROUND(round + 3); + } + + for (i = 0; i < 25; i++) + state[i] = vgetq_lane_u64(a[i], 0); +} + +/* + * Whether the extension is there. It is on Apple silicon; elsewhere the + * kernel reports it. + */ +int crypton_sha3_armv8_available(void) +{ +#if defined(__APPLE__) + int v = 0; + size_t n = sizeof(v); + + if (sysctlbyname("hw.optional.arm.FEAT_SHA3", &v, &n, NULL, 0) != 0) + return 0; + return v != 0; +#elif defined(__linux__) + return (getauxval(AT_HWCAP) & HWCAP_SHA3) != 0; +#else + return 0; +#endif +} diff --git a/cbits/sha512_armv8.c b/cbits/sha512_armv8.c new file mode 100644 --- /dev/null +++ b/cbits/sha512_armv8.c @@ -0,0 +1,161 @@ +/* + * SHA-512 using the ARMv8.2 SHA-512 extension. + * + * The same idea as sha256_armv8.c: SHA512H, SHA512H2, SHA512SU0 and + * SHA512SU1 do two rounds at a time and compute the message schedule + * alongside. This extension is a good deal less common than the SHA-256 + * one -- it arrived in ARMv8.2 and is optional there -- so the runtime + * check matters more here, and it is asked rather than assumed even on + * Apple, where the SHA-256 one is taken for granted. + * + * SHA-384 and the truncated SHA-512/t variants share the compression + * function, so they come along. + */ + +#include +#include +#if defined(__linux__) +#include +#include +#endif +#if defined(__APPLE__) +#include +#include +#endif + +/* + * The instructions are an extension, so a translation unit compiled for + * baseline ARMv8-A may not use them; mark the function that does. The + * SHA-512 instructions live behind "+sha3" in both GCC and clang. + */ +#ifdef WITH_TARGET_ATTRIBUTES +#define TARGET_ARMV8_SHA3 __attribute__((target("+sha3"))) +#else +#define TARGET_ARMV8_SHA3 +#endif + +static const uint64_t K[80] = { + 0x428a2f98d728ae22ULL, 0x7137449123ef65cdULL, 0xb5c0fbcfec4d3b2fULL, + 0xe9b5dba58189dbbcULL, 0x3956c25bf348b538ULL, 0x59f111f1b605d019ULL, + 0x923f82a4af194f9bULL, 0xab1c5ed5da6d8118ULL, 0xd807aa98a3030242ULL, + 0x12835b0145706fbeULL, 0x243185be4ee4b28cULL, 0x550c7dc3d5ffb4e2ULL, + 0x72be5d74f27b896fULL, 0x80deb1fe3b1696b1ULL, 0x9bdc06a725c71235ULL, + 0xc19bf174cf692694ULL, 0xe49b69c19ef14ad2ULL, 0xefbe4786384f25e3ULL, + 0x0fc19dc68b8cd5b5ULL, 0x240ca1cc77ac9c65ULL, 0x2de92c6f592b0275ULL, + 0x4a7484aa6ea6e483ULL, 0x5cb0a9dcbd41fbd4ULL, 0x76f988da831153b5ULL, + 0x983e5152ee66dfabULL, 0xa831c66d2db43210ULL, 0xb00327c898fb213fULL, + 0xbf597fc7beef0ee4ULL, 0xc6e00bf33da88fc2ULL, 0xd5a79147930aa725ULL, + 0x06ca6351e003826fULL, 0x142929670a0e6e70ULL, 0x27b70a8546d22ffcULL, + 0x2e1b21385c26c926ULL, 0x4d2c6dfc5ac42aedULL, 0x53380d139d95b3dfULL, + 0x650a73548baf63deULL, 0x766a0abb3c77b2a8ULL, 0x81c2c92e47edaee6ULL, + 0x92722c851482353bULL, 0xa2bfe8a14cf10364ULL, 0xa81a664bbc423001ULL, + 0xc24b8b70d0f89791ULL, 0xc76c51a30654be30ULL, 0xd192e819d6ef5218ULL, + 0xd69906245565a910ULL, 0xf40e35855771202aULL, 0x106aa07032bbd1b8ULL, + 0x19a4c116b8d2d0c8ULL, 0x1e376c085141ab53ULL, 0x2748774cdf8eeb99ULL, + 0x34b0bcb5e19b48a8ULL, 0x391c0cb3c5c95a63ULL, 0x4ed8aa4ae3418acbULL, + 0x5b9cca4f7763e373ULL, 0x682e6ff3d6b2b8a3ULL, 0x748f82ee5defb2fcULL, + 0x78a5636f43172f60ULL, 0x84c87814a1f0ab72ULL, 0x8cc702081a6439ecULL, + 0x90befffa23631e28ULL, 0xa4506cebde82bde9ULL, 0xbef9a3f7b2c67915ULL, + 0xc67178f2e372532bULL, 0xca273eceea26619cULL, 0xd186b8c721c0c207ULL, + 0xeada7dd6cde0eb1eULL, 0xf57d4f7fee6ed178ULL, 0x06f067aa72176fbaULL, + 0x0a637dc5a2c898a6ULL, 0x113f9804bef90daeULL, 0x1b710b35131c471bULL, + 0x28db77f523047d84ULL, 0x32caab7b40c72493ULL, 0x3c9ebe0a15c9bebcULL, + 0x431d67c49c100d4cULL, 0x4cc5d4becb3e42b6ULL, 0x597f299cfc657e2aULL, + 0x5fcb6fab3ad6faecULL, 0x6c44198c4a475817ULL, +}; + +/* + * One 128-byte block. `state` is the eight words of chaining value in host + * order, `buf` the block as it arrived, which SHA-512 reads big-endian. + * + * ab, cd, ef and gh hold the working variables in pairs. Each step covers + * two rounds and rotates which pair plays which part, so four steps return + * to the start; a group of eight steps is one pass over the schedule. + */ +TARGET_ARMV8_SHA3 +void crypton_sha512_armv8_do_chunk(uint64_t state[8], const uint64_t buf[16]) +{ + uint64x2_t ab, cd, ef, gh, ab0, cd0, ef0, gh0; + uint64x2_t s[8]; + int i; + + ab0 = ab = vld1q_u64(state); + cd0 = cd = vld1q_u64(state + 2); + ef0 = ef = vld1q_u64(state + 4); + gh0 = gh = vld1q_u64(state + 6); + + for (i = 0; i < 8; i++) + s[i] = vreinterpretq_u64_u8(vrev64q_u8( + vld1q_u8((const uint8_t *) (buf + 2 * i)))); + +/* two rounds; A, B, C, D is a rotation of gh, ef, cd, ab */ +#define RND(A, B, C, D, sv, ki) \ + do { \ + uint64x2_t is_ = vaddq_u64((sv), vld1q_u64(&K[ki])); \ + uint64x2_t sum_ = vaddq_u64(vextq_u64(is_, is_, 1), (A)); \ + uint64x2_t im_ = vsha512hq_u64(sum_, vextq_u64((B), (A), 1),\ + vextq_u64((C), (B), 1)); \ + (A) = vsha512h2q_u64(im_, (C), (D)); \ + (C) = vaddq_u64((C), im_); \ + } while (0) + +/* extend the schedule in place, for the next sixteen rounds */ +#define SCHED(j) \ + s[j] = vsha512su1q_u64(vsha512su0q_u64(s[j], s[((j) + 1) & 7]), \ + s[((j) + 7) & 7], \ + vextq_u64(s[((j) + 4) & 7], s[((j) + 5) & 7], 1)) + +#define PASS(base) \ + SCHED(0); RND(gh, ef, cd, ab, s[0], (base) + 0); \ + SCHED(1); RND(ef, cd, ab, gh, s[1], (base) + 2); \ + SCHED(2); RND(cd, ab, gh, ef, s[2], (base) + 4); \ + SCHED(3); RND(ab, gh, ef, cd, s[3], (base) + 6); \ + SCHED(4); RND(gh, ef, cd, ab, s[4], (base) + 8); \ + SCHED(5); RND(ef, cd, ab, gh, s[5], (base) + 10); \ + SCHED(6); RND(cd, ab, gh, ef, s[6], (base) + 12); \ + SCHED(7); RND(ab, gh, ef, cd, s[7], (base) + 14) + + /* rounds 0..15 run straight off the message */ + RND(gh, ef, cd, ab, s[0], 0); + RND(ef, cd, ab, gh, s[1], 2); + RND(cd, ab, gh, ef, s[2], 4); + RND(ab, gh, ef, cd, s[3], 6); + RND(gh, ef, cd, ab, s[4], 8); + RND(ef, cd, ab, gh, s[5], 10); + RND(cd, ab, gh, ef, s[6], 12); + RND(ab, gh, ef, cd, s[7], 14); + + PASS(16); + PASS(32); + PASS(48); + PASS(64); + +#undef PASS +#undef SCHED +#undef RND + + vst1q_u64(state, vaddq_u64(ab, ab0)); + vst1q_u64(state + 2, vaddq_u64(cd, cd0)); + vst1q_u64(state + 4, vaddq_u64(ef, ef0)); + vst1q_u64(state + 6, vaddq_u64(gh, gh0)); +} + +/* + * Whether the extension is there. Unlike the SHA-256 one this is not + * something to take for granted anywhere, so both platforms are asked. + */ +int crypton_sha512_armv8_available(void) +{ +#if defined(__APPLE__) + int v = 0; + size_t n = sizeof(v); + + if (sysctlbyname("hw.optional.arm.FEAT_SHA512", &v, &n, NULL, 0) != 0) + return 0; + return v != 0; +#elif defined(__linux__) + return (getauxval(AT_HWCAP) & HWCAP_SHA512) != 0; +#else + return 0; +#endif +} diff --git a/crypton.cabal b/crypton.cabal --- a/crypton.cabal +++ b/crypton.cabal @@ -1,14 +1,14 @@ -cabal-version: 1.18 +cabal-version: 3.0 name: crypton -version: 1.1.5 -license: BSD3 +version: 2.0.0 +license: BSD-3-Clause license-file: LICENSE copyright: Vincent Hanquez maintainer: Kazu Yamamoto author: Vincent Hanquez stability: experimental tested-with: - ghc ==9.2.8 || ==9.4.8 || ==9.6.7 || ==9.8.4 || ==9.10.1 || ==9.12.1 + ghc ==9.2.8 || ==9.4.8 || ==9.6.7 || ==9.8.4 || ==9.10.2 || ==9.12.4 || ==9.14.1 homepage: https://github.com/kazu-yamamoto/crypton bug-reports: https://github.com/kazu-yamamoto/crypton/issues @@ -43,7 +43,25 @@ extra-source-files: cbits/*.h cbits/aes/*.h + cbits/aes/armv8_impl.c cbits/aes/x86ni_impl.c + cbits/asm/LICENSE.cryptogams + cbits/asm/README.md + cbits/asm/aesni-gcm-x86_64.pl + cbits/asm/arm-xlate.pl + cbits/asm/arm_arch.h + cbits/asm/chacha-armv8.pl + cbits/asm/chacha-x86_64.pl + cbits/asm/generate.sh + cbits/asm/keccak1600-armv8.pl + cbits/asm/keccak1600-x86_64.pl + cbits/asm/poly1305-armv8.pl + cbits/asm/poly1305-x86_64.pl + cbits/asm/sha1-armv8.pl + cbits/asm/sha512-armv8.pl + cbits/asm/sha512-x86_64.pl + cbits/asm/x86_64-xlate.pl + cbits/chacha_sse_impl.c cbits/argon2/*.c cbits/argon2/*.h cbits/blake2/ref/*.h @@ -144,6 +162,7 @@ Crypto.Data.Padding Crypto.ECC Crypto.ECC.Edwards25519 + Crypto.Debug Crypto.Error Crypto.Hash Crypto.Hash.Algorithms @@ -182,6 +201,7 @@ Crypto.PubKey.ECC.Types Crypto.PubKey.ECDSA Crypto.PubKey.ECIES + Crypto.PubKey.ElGamal Crypto.PubKey.Ed25519 Crypto.PubKey.Ed448 Crypto.PubKey.EdDSA @@ -205,20 +225,36 @@ Crypto.System.CPU Crypto.Tutorial - cc-options: -std=gnu99 + -- -O3 over -O2, which is what GHC passes: measured on x86-64, AES-128-GCM + -- 3455 to 3708 MB/s, AES-128-OCB 2187 to 2484, a P-256 base point + -- multiplication 71.0 to 59.8 us and SHA-256 312 to 318, with ChaCha20, + -- Poly1305, SHA-1 and MD5 within a couple of per cent either way; on Apple + -- silicon the same P-256 multiplication goes from 26.0 to 24.3 us. The + -- code that must not branch on a secret does not: the masked selections in + -- the curve and field code compile to no conditional jumps at either + -- level, and what -O3 adds in cbits/crypton_powm.c is loop control over + -- limb counts, which are public. + cc-options: -std=gnu99 -O3 c-sources: cbits/argon2/argon2.c cbits/crypton_blake2b.c cbits/crypton_blake2bp.c cbits/crypton_blake2s.c cbits/crypton_blake2sp.c + cbits/crypton_blowfish.c + cbits/crypton_camellia.c cbits/crypton_chacha.c cbits/crypton_cpu.c + cbits/crypton_des.c + cbits/crypton_ecc.c + cbits/crypton_f2m.c cbits/crypton_md2.c + cbits/crypton_memxor.c cbits/crypton_md4.c cbits/crypton_md5.c cbits/crypton_pbkdf2.c cbits/crypton_poly1305.c + cbits/crypton_powm.c cbits/crypton_rc4.c cbits/crypton_ripemd.c cbits/crypton_salsa.c @@ -238,7 +274,6 @@ other-modules: Crypto.Cipher.AES.Primitive - Crypto.Cipher.Blowfish.Box Crypto.Cipher.Blowfish.Primitive Crypto.Cipher.Camellia.Primitive Crypto.Cipher.CAST5.Primitive @@ -281,13 +316,13 @@ Crypto.Internal.Compat Crypto.Internal.CompatPrim Crypto.Internal.DeepSeq + Crypto.Internal.ECC Crypto.Internal.Endian Crypto.Internal.Imports Crypto.Internal.Nat Crypto.Internal.WordArray Crypto.Internal.Words Crypto.Number.Compat - Crypto.PubKey.ElGamal Crypto.PubKey.Internal Crypto.Random.ChaChaDRG Crypto.Random.Entropy.Backend @@ -360,12 +395,128 @@ if arch(x86_64) cpp-options: -DARCH_X86_64 + -- SSE2 is part of the x86-64 baseline, so this needs no flag and no + -- runtime check; i386 keeps the scalar code. + if arch(x86_64) + cc-options: + -DWITH_X86_SSE2 -DWITH_X86_AVX2 -DWITH_X86_SHA_NI + c-sources: + cbits/chacha_avx2.c + cbits/chacha_sse2.c + cbits/sha1_x86.c + + -- The SHA extensions are not part of the baseline, so without the + -- attributes this raises it for every file, as the AES-NI and + -- AArch64 paths do. + if !flag(use_target_attributes) + cc-options: -msha -msse4.1 -mssse3 + + -- Poly1305, ChaCha20 and the SHA-2 pair from CRYPTOGAMS, + -- hand-scheduled. Poly1305 + -- there has paths for AVX and AVX2 where the C here has only the + -- second, and replaces it entirely; ChaCha20 there is ahead of the + -- C from one block up, having vector code for lengths the C takes + -- a block at a time. What either has beyond AVX2 is not asked + -- for: no machine here can run it, and a path nothing has executed + -- is not worth the few per cent. See cbits/asm/README.md. + cc-options: + -DWITH_X86_POLY1305_ASM -DWITH_X86_CHACHA_ASM + -DWITH_X86_SHA256_ASM -DWITH_X86_SHA512_ASM -DWITH_X86_SHA3_ASM + + if os(osx) + asm-sources: + cbits/asm/chacha-x86_64-macosx.S + cbits/asm/keccak1600-x86_64-macosx.S + cbits/asm/poly1305-x86_64-macosx.S + cbits/asm/sha256-x86_64-macosx.S + cbits/asm/sha512-x86_64-macosx.S + + elif os(windows) + asm-sources: + cbits/asm/chacha-x86_64-mingw64.S + cbits/asm/keccak1600-x86_64-mingw64.S + cbits/asm/poly1305-x86_64-mingw64.S + cbits/asm/sha256-x86_64-mingw64.S + cbits/asm/sha512-x86_64-mingw64.S + + else + asm-sources: + cbits/asm/chacha-x86_64-elf.S + cbits/asm/keccak1600-x86_64-elf.S + cbits/asm/poly1305-x86_64-elf.S + cbits/asm/sha256-x86_64-elf.S + cbits/asm/sha512-x86_64-elf.S + if ((flag(support_rdrand) && (arch(i386) || arch(x86_64))) && !os(windows)) cpp-options: -DSUPPORT_RDRAND c-sources: cbits/crypton_rdrand.c other-modules: Crypto.Random.Entropy.RDRand - if ((flag(support_aesni) && ((os(linux) || os(freebsd)) || os(osx))) && (arch(i386) || arch(x86_64))) + if (flag(support_aesni) && arch(aarch64)) + cc-options: -DWITH_ARMV8_CRYPTO + c-sources: + cbits/aes/generic.c + cbits/aes/gf.c + cbits/aes/armv8.c + cbits/crypton_aes.c + + if !flag(use_target_attributes) + cc-options: -march=armv8-a+crypto + + if arch(aarch64) + cc-options: + -DWITH_ARMV8_SHA1 -DWITH_ARMV8_SHA2 -DWITH_ARMV8_SHA3 + -DWITH_ARMV8_SHA512 + -DWITH_ARMV8_NEON + c-sources: + cbits/chacha_neon.c + cbits/sha1_armv8.c + cbits/sha256_armv8.c + cbits/sha3_armv8.c + cbits/sha512_armv8.c + + -- +sha3 covers the SHA-512 instructions as well as the SHA-3 ones, + -- and leaves the baseline at ARMv8-A: it says what the compiler may + -- emit where asked, not what the machine is assumed to have. + if !flag(use_target_attributes) + cc-options: -march=armv8-a+crypto+sha3 + + -- ChaCha20, Poly1305 and SHA-256 from CRYPTOGAMS. ChaCha20 there runs a + -- fifth block through the + -- general registers alongside four in the vector ones -- eight + -- and six above 512 bytes. The vector registers hold four states + -- and no more, so the rest of the parallelism has to come from the + -- integer side, which is a matter of naming registers and so + -- cannot be written in C; Poly1305 there is the whole of the + -- arithmetic, hand-scheduled, and replaces both the C loops; + -- the SHA-1, SHA-256 and Keccak there use the same instructions + -- as the intrinsics do but schedule them across a run of blocks + -- rather than one at a time. See + -- cbits/asm/README.md. Windows on AArch64 is left out: the object + -- format is neither of these two. + if !os(windows) + cc-options: + -DWITH_ARMV8_CHACHA_ASM -DWITH_ARMV8_POLY1305_ASM + -DWITH_ARMV8_SHA1_ASM -DWITH_ARMV8_SHA256_ASM + -DWITH_ARMV8_SHA3_ASM + + if os(osx) + asm-sources: + cbits/asm/chacha-armv8-ios64.S + cbits/asm/poly1305-armv8-ios64.S + cbits/asm/keccak1600-armv8-ios64.S + cbits/asm/sha1-armv8-ios64.S + cbits/asm/sha256-armv8-ios64.S + + else + asm-sources: + cbits/asm/chacha-armv8-linux64.S + cbits/asm/poly1305-armv8-linux64.S + cbits/asm/keccak1600-armv8-linux64.S + cbits/asm/sha1-armv8-linux64.S + cbits/asm/sha256-armv8-linux64.S + + if ((flag(support_aesni) && (((os(linux) || os(freebsd)) || os(osx)) || os(windows))) && (arch(i386) || arch(x86_64))) cc-options: -DWITH_AESNI c-sources: cbits/aes/generic.c @@ -382,6 +533,28 @@ if !flag(use_target_attributes) cc-options: -msse4.1 -mpclmul + -- AES-GCM from CRYPTOGAMS, which interleaves the counter-mode + -- rounds with the multiplies of the block before at + -- instruction granularity. The two do not want the same + -- execution ports, so held against each other they cost about + -- what the rounds alone cost; written in C the compiler sinks + -- every multiply to the end of the group instead. See + -- cbits/asm/README.md. The assembly is x86-64 only, and is + -- checked in per object format since it comes from a + -- generator. + if arch(x86_64) + cc-options: -DWITH_X86_GCM_ASM + c-sources: cbits/aes/gcm_x86_asm.c + + if os(osx) + asm-sources: cbits/asm/aesni-gcm-x86_64-macosx.S + + elif os(windows) + asm-sources: cbits/asm/aesni-gcm-x86_64-mingw64.S + + else + asm-sources: cbits/asm/aesni-gcm-x86_64-elf.S + else c-sources: cbits/aes/generic.c @@ -439,81 +612,89 @@ test-suite test-crypton type: exitcode-stdio-1.0 - main-is: Tests.hs + main-is: Spec.hs hs-source-dirs: tests other-modules: - BCrypt - BCryptPBKDF + AFISSpec BlockCipher - ChaCha - ChaChaPoly1305 - ECC - ECC.Edwards25519 - ECDSA - Hash + BlockCipher.AES.CBC + BlockCipher.AES.CCM + BlockCipher.AES.CTR + BlockCipher.AES.ECB + BlockCipher.AES.GCM + BlockCipher.AES.GCMLong + BlockCipher.AES.OCB3 + BlockCipher.AES.XTS + BlockCipher.AESGCMSIVSpec + BlockCipher.AESSpec + BlockCipher.BlowfishSpec + BlockCipher.CamelliaSpec + BlockCipher.CAST5Spec + BlockCipher.DESSpec + BlockCipher.ModesSpec + BlockCipher.TripleDESSpec + BlockCipher.TwofishSpec + ConstructHash.MiyaguchiPreneelSpec + Curve25519Spec + Curve448Spec + ECC.Edwards25519Spec + ECCSpec + ECDSASpec + Ed25519Spec + Ed448Spec + EdDSASpec + HashSpec Imports - KAT_AES - KAT_AES.KATCBC - KAT_AES.KATCCM - KAT_AES.KATECB - KAT_AES.KATGCM - KAT_AES.KATOCB3 - KAT_AES.KATXTS - KAT_AESGCMSIV - KAT_AFIS - KAT_Argon2 - KAT_Blake2 - KAT_Blowfish - KAT_Camellia - KAT_CAST5 - KAT_CMAC - KAT_Curve25519 - KAT_Curve448 - KAT_DES - KAT_Ed25519 - KAT_Ed448 - KAT_EdDSA - KAT_HKDF - KAT_HMAC - KAT_KMAC - KAT_MiyaguchiPreneel - KAT_OTP - KAT_PBKDF2 - KAT_PubKey - KAT_PubKey.DSA - KAT_PubKey.ECC - KAT_PubKey.ECDSA - KAT_PubKey.OAEP - KAT_PubKey.P256 - KAT_PubKey.PSS - KAT_PubKey.Rabin - KAT_PubKey.RSA - KAT_RC4 - KAT_Scrypt - KAT_TripleDES - KAT_Twofish - Number - Number.F2m - Padding - Poly1305 - Salsa + KDF.Argon2Spec + KDF.BCryptPBKDFSpec + KDF.BCryptSpec + KDF.HKDFSpec + KDF.PBKDF2Spec + KDF.ScryptSpec + MAC.Blake2Spec + MAC.CMACSpec + MAC.HMACSpec + MAC.KMACSpec + MAC.Poly1305Spec + MAC.Poly1305Vectors + Number.F2mSpec + NumberSpec + OTPSpec + PaddingSpec + PubKey.DHSpec + PubKey.DSASpec + PubKey.ECCSpec + PubKey.ECDSASpec + PubKey.ElGamalSpec + PubKey.MGF1Spec + PubKey.OAEPSpec + PubKey.P256Spec + PubKey.PSSSpec + PubKey.RabinSpec + PubKey.SecrecySpec + PubKey.RSASpec + RuntimeSpec + StreamCipher.ChaChaPoly1305Spec + StreamCipher.ChaChaSpec + StreamCipher.RC4Spec + StreamCipher.SalsaSpec + StreamCipher.XSalsaSpec Utils - XSalsa - default-language: Haskell2010 - ghc-options: - -Wall -fno-warn-orphans -fno-warn-missing-signatures -rtsopts - build-depends: base >=4.13 && <5, bytestring, + QuickCheck, crypton, - ram, - tasty, - tasty-hunit, - tasty-kat, - tasty-quickcheck + hspec, + ram + build-tool-depends: hspec-discover:hspec-discover + + default-language: Haskell2010 + ghc-options: + -Wall -fno-warn-orphans -fno-warn-missing-signatures -rtsopts + benchmark bench-crypton type: exitcode-stdio-1.0 main-is: Bench.hs @@ -526,6 +707,6 @@ bytestring, crypton, deepseq, - gauge, ram, - random + random, + tasty-bench diff --git a/tests/AFISSpec.hs b/tests/AFISSpec.hs new file mode 100644 --- /dev/null +++ b/tests/AFISSpec.hs @@ -0,0 +1,103 @@ +{-# LANGUAGE ExistentialQuantification #-} +{-# LANGUAGE OverloadedStrings #-} + +module AFISSpec (spec) where + +import Imports + +import Control.Exception (evaluate) +import qualified Crypto.Data.AFIS as AFIS +import Crypto.Error +import Crypto.Hash +import Crypto.Random +import qualified Data.ByteString as B + +mergeVec :: [(Int, SHA1, B.ByteString, B.ByteString)] +mergeVec = + [ + ( 3 + , SHA1 + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\xd4\x76\xc8\x58\xbd\xf0\x15\xbe\x9f\x40\xe3\x65\x20\x1c\x9c\xb8\xd8\x1c\x16\x64" + ) + , + ( 3 + , SHA1 + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17" + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\xd6\x75\xc8\x59\xbb\xf7\x11\xbb\x95\x4b\xeb\x6c\x2e\x13\x90\xb5\xca\x0f\x06\x75\x17\x70\x39\x28" + ) + ] + +mergeKATs = zipWith toProp mergeVec [(0 :: Int) ..] + where + toProp (nbExpands, hashAlg, expected, dat) i = + it ("merge " ++ show i) (AFIS.merge hashAlg nbExpands dat `shouldBe` expected) + +data AFISParams = AFISParams B.ByteString Int SHA1 ChaChaDRG + +instance Show AFISParams where + show (AFISParams dat expand _ _) = "data: " ++ show dat ++ " expanded: " ++ show expand + +instance Arbitrary AFISParams where + arbitrary = + AFISParams + <$> arbitraryBSof 3 46 + <*> choose (2, 2) + <*> elements [SHA1] + <*> arbitrary + +instance Arbitrary ChaChaDRG where + arbitrary = drgNewTest <$> arbitrary + +-- | Parameters neither function can work with. An expand count of zero used +-- to divide by zero in merge, a negative one reported the data as null, and an +-- expand count of one was accepted and handed the diffused data straight back +-- as though it were the secret -- which is the one that does not announce +-- itself. split already refused all three, so it had nothing to say about a +-- secret of no bytes, which it split into nothing that merge then refused. +invalidParameterTests :: Spec +invalidParameterTests = + describe "invalid parameters" $ do + it "merge refuses an expand count of zero" $ + evaluate (tryMerge 0 diffused) `shouldThrow` refused + it "merge refuses a negative expand count" $ + evaluate (tryMerge (-1) diffused) `shouldThrow` refused + it "merge refuses an expand count of one" $ + evaluate (tryMerge 1 diffused) `shouldThrow` refused + it "merge refuses data that is not a multiple of the expand count" $ + evaluate (tryMerge 3 diffused) `shouldThrow` refused + it "merge refuses empty data" $ + evaluate (tryMerge 4 B.empty) `shouldThrow` refused + it "split refuses an expand count below two" $ do + evaluate (trySplit 0 secret) `shouldThrow` refused + evaluate (trySplit 1 secret) `shouldThrow` refused + evaluate (trySplit (-1) secret) `shouldThrow` refused + it "split refuses an empty secret" $ + evaluate (trySplit 4 B.empty) `shouldThrow` refused + it "the recoverable variants report instead of raising" $ do + AFIS.tryMerge SHA1 0 diffused `shouldBe` failed + AFIS.tryMerge SHA1 1 diffused `shouldBe` failed + AFIS.tryMerge SHA1 3 diffused `shouldBe` failed + AFIS.tryMerge SHA1 4 B.empty `shouldBe` failed + fmap fst (AFIS.trySplit SHA1 rng 1 secret) `shouldBe` failed + fmap fst (AFIS.trySplit SHA1 rng 4 B.empty) `shouldBe` failed + it "the recoverable variants still split and merge" $ do + let d = fmap fst (AFIS.trySplit SHA1 rng 4 secret) + d `shouldBe` CryptoPassed diffused + (d >>= AFIS.tryMerge SHA1 4) `shouldBe` CryptoPassed secret + it "a good split still merges back" $ + AFIS.merge SHA1 4 diffused `shouldBe` secret + where + rng = drgNewTest (1, 2, 3, 4, 5) + secret = "0123456789abcdef0123" :: B.ByteString + diffused = fst (AFIS.split SHA1 rng 4 secret) :: B.ByteString + tryMerge e d = AFIS.merge SHA1 e d :: B.ByteString + trySplit e d = fst (AFIS.split SHA1 rng e d) :: B.ByteString + failed = CryptoFailed CryptoError_ParameterInvalid :: CryptoFailable B.ByteString + refused e = e == CryptoError_ParameterInvalid + +spec :: Spec +spec = do + describe "KAT merge" $ sequence_ mergeKATs + invalidParameterTests + prop "merge.split == id" $ \(AFISParams bs e hf rng) -> bs == (AFIS.merge hf e $ fst (AFIS.split hf rng e bs)) diff --git a/tests/BCrypt.hs b/tests/BCrypt.hs deleted file mode 100644 --- a/tests/BCrypt.hs +++ /dev/null @@ -1,111 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module BCrypt ( - tests, -) -where - -import Crypto.KDF.BCrypt -import qualified Data.ByteString as B -import Imports - --- Openwall bcrypt tests, with 2x versions and 0xFF special cases removed. -expected :: [(ByteString, ByteString)] -expected = - [ ("$2a$05$CCCCCCCCCCCCCCCCCCCCC.E5YPO9kmyuRGyh0XouQYb4YMJKvyOeW", "U*U") - , ("$2a$05$CCCCCCCCCCCCCCCCCCCCC.VGOzA784oUp/Z0DY336zx7pLYAy0lwK", "U*U*") - , ("$2a$05$XXXXXXXXXXXXXXXXXXXXXOAcXxm9kjPGEMsLznoKqmqw7tc8WCx4a", "U*U*U") - , - ( "$2a$05$abcdefghijklmnopqrstuu5s2v8.iXieOjg/.AySBTTZIIVFJeBui" - , "0123456789abcdefghijklmnopqrstuvwxyz\ - \ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789\ - \chars after 72 are ignored" - ) - , ("$2y$05$/OK.fbVrR/bpIqNJ5ianF.CE5elHaaO4EbggVDjb8P19RukzXSM3e", "\xff\xff\xa3") - , ("$2b$05$/OK.fbVrR/bpIqNJ5ianF.CE5elHaaO4EbggVDjb8P19RukzXSM3e", "\xff\xff\xa3") - , ("$2y$05$/OK.fbVrR/bpIqNJ5ianF.Sa7shbm4.OzKpvFnX1pQLmQW96oUlCq", "\xa3") - , ("$2a$05$/OK.fbVrR/bpIqNJ5ianF.Sa7shbm4.OzKpvFnX1pQLmQW96oUlCq", "\xa3") - , ("$2b$05$/OK.fbVrR/bpIqNJ5ianF.Sa7shbm4.OzKpvFnX1pQLmQW96oUlCq", "\xa3") - , - ( "$2a$05$/OK.fbVrR/bpIqNJ5ianF.swQOIzjOiJ9GHEPuhEkvqrUyvWhEMx6" - , "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ - \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ - \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ - \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ - \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ - \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ - \chars after 72 are ignored as usual" - ) - , - ( "$2a$05$/OK.fbVrR/bpIqNJ5ianF.R9xrDjiycxMbQE2bp.vgqlYpW5wx2yy" - , "\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ - \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ - \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ - \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ - \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ - \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55" - ) - , - ( "$2a$05$/OK.fbVrR/bpIqNJ5ianF.9tQZzcJfm3uj2NvJ/n5xkhpqLrMpWCe" - , "\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ - \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ - \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ - \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ - \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ - \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff" - ) - , ("$2a$05$CCCCCCCCCCCCCCCCCCCCC.7uG0VCzI2bS7j6ymqJi9CdcdxiRTWNy", "") - , ("$2a$06$DCq7YPn5Rq63x1Lad4cll.TV4S6ytwfsfvkgY8jIucDrjc8deX1s.", "") - , ("$2a$08$HqWuK6/Ng6sg9gQzbLrgb.Tl.ZHfXLhvt/SgVyWhQqgqcZ7ZuUtye", "") - , ("$2a$10$k1wbIrmNyFAPwPVPSVa/zecw2BCEnBwVS2GbrmgzxFUOqW9dk4TCW", "") - , ("$2a$12$k42ZFHFWqBp3vWli.nIn8uYyIkbvYRvodzbfbK18SSsY.CsIQPlxO", "") - , ("$2a$06$m0CrhHm10qJ3lXRY.5zDGO3rS2KdeeWLuGmsfGlMfOxih58VYVfxe", "a") - , ("$2a$08$cfcvVd2aQ8CMvoMpP2EBfeodLEkkFJ9umNEfPD18.hUF62qqlC/V.", "a") - , ("$2a$12$8NJH3LsPrANStV6XtBakCez0cKHXVxmvxIlcz785vxAIZrihHZpeS", "a") - , ("$2a$06$If6bvum7DFjUnE9p2uDeDu0YHzrHM6tf.iqN8.yx.jNN1ILEf7h0i", "abc") - , ("$2a$08$Ro0CUfOqk6cXEKf3dyaM7OhSCvnwM9s4wIX9JeLapehKK5YdLxKcm", "abc") - , ("$2a$10$WvvTPHKwdBJ3uk0Z37EMR.hLA2W6N9AEBhEgrAOljy2Ae5MtaSIUi", "abc") - , - ( "$2a$06$.rCVZVOThsIa97pEDOxvGuRRgzG64bvtJ0938xuqzv18d3ZpQhstC" - , "abcdefghijklmnopqrstuvwxyz" - ) - ] - -makeKATs = concatMap maketest (zip3 is passwords hashes) - where - is :: [Int] - is = [1 ..] - - passwords = map snd expected - hashes = map fst expected - - maketest (i, password, hash) = - [ testCase (show i) (assertBool "" (validatePassword password hash)) - ] - -tests = - testGroup - "bcrypt" - [ testGroup "KATs" makeKATs - , testCase - "Invalid hash length" - ( assertEqual - "" - (Left "Invalid hash format") - ( validatePasswordEither - B.empty - ("$2a$06$DCq7YPn5Rq63x1Lad4cll.TV4S6ytwfsfvkgY8jIucDrjc8deX1s" :: B.ByteString) - ) - ) - , testCase - "Hash and validate" - ( assertBool - "Hashed password should validate" - (validatePassword somePassword (bcrypt 5 aSalt somePassword :: B.ByteString)) - ) - ] - where - somePassword = "some password" :: B.ByteString - aSalt = - "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" - :: B.ByteString diff --git a/tests/BCryptPBKDF.hs b/tests/BCryptPBKDF.hs deleted file mode 100644 --- a/tests/BCryptPBKDF.hs +++ /dev/null @@ -1,212 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module BCryptPBKDF (tests) where - -import qualified Data.ByteString as B - -import Test.Tasty -import Test.Tasty.HUnit - -import Crypto.KDF.BCryptPBKDF ( - Parameters (..), - generate, - hashInternal, - ) - -tests :: TestTree -tests = - testGroup - "BCryptPBKDF" - [ testGroup - "generate" - [ testCase "1" generate1 - , testCase "2" generate2 - , testCase "3" generate3 - ] - , testGroup - "hashInternal" - [ testCase "1" hashInternal1 - ] - ] - where - -- test vector taken from the go implementation by @dchest - generate1 = expected @=? generate params pass salt - where - params = Parameters 12 32 - pass = "password" :: B.ByteString - salt = "salt" :: B.ByteString - expected = - B.pack - [ 0x1a - , 0xe4 - , 0x2c - , 0x05 - , 0xd4 - , 0x87 - , 0xbc - , 0x02 - , 0xf6 - , 0x49 - , 0x21 - , 0xa4 - , 0xeb - , 0xe4 - , 0xea - , 0x93 - , 0xbc - , 0xac - , 0xfe - , 0x13 - , 0x5f - , 0xda - , 0x99 - , 0x97 - , 0x4c - , 0x06 - , 0xb7 - , 0xb0 - , 0x1f - , 0xae - , 0x14 - , 0x9a - ] - :: B.ByteString - - -- test vector generated with the go implemenation by @dchest - generate2 = expected @=? generate params pass salt - where - params = Parameters 7 71 - pass = "DieWuerdeDesMenschenIstUnantastbar" :: B.ByteString - salt = "Tafelsalz" :: B.ByteString - expected = - B.pack - [ 0x17 - , 0xb4 - , 0x76 - , 0xaa - , 0xd7 - , 0x42 - , 0x33 - , 0x49 - , 0x5c - , 0xe8 - , 0x79 - , 0x49 - , 0x15 - , 0x74 - , 0x4c - , 0x71 - , 0xf9 - , 0x99 - , 0x66 - , 0x89 - , 0x7a - , 0x60 - , 0xc3 - , 0x70 - , 0xb4 - , 0x3c - , 0xa8 - , 0x83 - , 0x80 - , 0x5a - , 0x56 - , 0xde - , 0x38 - , 0xbc - , 0x51 - , 0x8c - , 0xd4 - , 0xeb - , 0xd1 - , 0xcf - , 0x46 - , 0x0a - , 0x68 - , 0x3d - , 0xc8 - , 0x12 - , 0xcf - , 0xf8 - , 0x43 - , 0xce - , 0x21 - , 0x9d - , 0x98 - , 0x81 - , 0x20 - , 0x26 - , 0x6e - , 0x42 - , 0x0f - , 0xaa - , 0x75 - , 0x5d - , 0x09 - , 0x8d - , 0x45 - , 0xda - , 0xd5 - , 0x15 - , 0x6e - , 0x65 - , 0x1d - ] - :: B.ByteString - - -- test vector generated with the go implemenation by @dchest - generate3 = expected @=? generate params pass salt - where - params = Parameters 5 5 - pass = "ABC" :: B.ByteString - salt = "DEF" :: B.ByteString - expected = - B.pack - [ 0xdd - , 0x6e - , 0xa0 - , 0x69 - , 0x29 - ] - :: B.ByteString - - hashInternal1 = expected @=? hashInternal passHash saltHash - where - passHash = B.pack [0 .. 63] :: B.ByteString - saltHash = B.pack [64 .. 127] :: B.ByteString - expected = - B.pack - [ 0x87 - , 0x90 - , 0x48 - , 0x70 - , 0xee - , 0xf9 - , 0xde - , 0xdd - , 0xf8 - , 0xe7 - , 0x61 - , 0x1a - , 0x14 - , 0x01 - , 0x06 - , 0xe6 - , 0xaa - , 0xf1 - , 0xa3 - , 0x63 - , 0xd9 - , 0xa2 - , 0xc5 - , 0x04 - , 0xdb - , 0x35 - , 0x64 - , 0x43 - , 0x72 - , 0x1e - , 0xb5 - , 0x55 - ] - :: B.ByteString diff --git a/tests/BlockCipher.hs b/tests/BlockCipher.hs --- a/tests/BlockCipher.hs +++ b/tests/BlockCipher.hs @@ -10,6 +10,7 @@ KATs (..), defaultKATs, testBlockCipher, + testBlockCipher128, CipherInfo, ) where @@ -132,50 +133,50 @@ {- testECB (_, _, cipherInit) ecbEncrypt ecbDecrypt kats = - testGroup "ECB" (concatMap katTest (zip is kats) {- ++ propTests-}) + describe "ECB" $ mapM_ (katTest (zip is kats) {- ++ propTests-}) where katTest (i,d) = - [ testCase ("E" ++ show i) (ecbEncrypt ctx (ecbPlaintext d) @?= ecbCiphertext d) - , testCase ("D" ++ show i) (ecbDecrypt ctx (ecbCiphertext d) @?= ecbPlaintext d) + [ it ("E" ++ show i) (ecbEncrypt ctx (ecbPlaintext d) `shouldBe` ecbCiphertext d) + , it ("D" ++ show i) (ecbDecrypt ctx (ecbCiphertext d) `shouldBe` ecbPlaintext d) ] where ctx = cipherInit (ecbKey d) - --propTest = testProperty "decrypt.encrypt" (ECBUnit key plaintext) = + --propTest = prop "decrypt.encrypt" (ECBUnit key plaintext) = --testProperty_ECB (ECBUnit (cipherInit -> ctx) (toBytes -> plaintext)) = -- plaintext `assertEq` ecbDecrypt ctx (ecbEncrypt ctx plaintext) testKatCBC cbcInit cbcEncrypt cbcDecrypt (i,d) = - [ testCase ("E" ++ show i) (cbcEncrypt ctx iv (cbcPlaintext d) @?= cbcCiphertext d) - , testCase ("D" ++ show i) (cbcDecrypt ctx iv (cbcCiphertext d) @?= cbcPlaintext d) + [ it ("E" ++ show i) (cbcEncrypt ctx iv (cbcPlaintext d) `shouldBe` cbcCiphertext d) + , it ("D" ++ show i) (cbcDecrypt ctx iv (cbcCiphertext d) `shouldBe` cbcPlaintext d) ] where ctx = cbcInit $ cbcKey d iv = cbcIV d testKatCFB cfbInit cfbEncrypt cfbDecrypt (i,d) = - [ testCase ("E" ++ show i) (cfbEncrypt ctx iv (cfbPlaintext d) @?= cfbCiphertext d) - , testCase ("D" ++ show i) (cfbDecrypt ctx iv (cfbCiphertext d) @?= cfbPlaintext d) + [ it ("E" ++ show i) (cfbEncrypt ctx iv (cfbPlaintext d) `shouldBe` cfbCiphertext d) + , it ("D" ++ show i) (cfbDecrypt ctx iv (cfbCiphertext d) `shouldBe` cfbPlaintext d) ] where ctx = cfbInit $ cfbKey d iv = cfbIV d testKatCTR ctrInit ctrCombine (i,d) = - [ testCase ("E" ++ i) (ctrCombine ctx iv (ctrPlaintext d) @?= ctrCiphertext d) - , testCase ("D" ++ i) (ctrCombine ctx iv (ctrCiphertext d) @?= ctrPlaintext d) + [ it ("E" ++ i) (ctrCombine ctx iv (ctrPlaintext d) `shouldBe` ctrCiphertext d) + , it ("D" ++ i) (ctrCombine ctx iv (ctrCiphertext d) `shouldBe` ctrPlaintext d) ] where ctx = ctrInit $ ctrKey d iv = ctrIV d testKatXTS xtsInit xtsEncrypt xtsDecrypt (i,d) = - [ testCase ("E" ++ i) (xtsEncrypt ctx iv 0 (xtsPlaintext d) @?= xtsCiphertext d) - , testCase ("D" ++ i) (xtsDecrypt ctx iv 0 (xtsCiphertext d) @?= xtsPlaintext d) + [ it ("E" ++ i) (xtsEncrypt ctx iv 0 (xtsPlaintext d) `shouldBe` xtsCiphertext d) + , it ("D" ++ i) (xtsDecrypt ctx iv 0 (xtsCiphertext d) `shouldBe` xtsPlaintext d) ] where ctx = xtsInit (xtsKey1 d, xtsKey2 d) iv = xtsIV d testKatAEAD cipherInit aeadInit aeadAppendHeader aeadEncrypt aeadDecrypt aeadFinalize (i,d) = - [ testCase ("AE" ++ i) (etag @?= aeadTag d) - , testCase ("AD" ++ i) (dtag @?= aeadTag d) - , testCase ("E" ++ i) (ebs @?= aeadCiphertext d) - , testCase ("D" ++ i) (dbs @?= aeadPlaintext d) + [ it ("AE" ++ i) (etag `shouldBe` aeadTag d) + , it ("AD" ++ i) (dtag `shouldBe` aeadTag d) + , it ("E" ++ i) (ebs `shouldBe` aeadCiphertext d) + , it ("D" ++ i) (dbs `shouldBe` aeadPlaintext d) ] where ctx = cipherInit $ aeadKey d (Just aead) = aeadInit ctx (aeadIV d) @@ -190,61 +191,43 @@ :: BlockCipher cipher => KATs -> cipher - -> TestTree -testKATs kats cipher = - testGroup - "KAT" - ( maybeGroup makeECBTest "ECB" (kat_ECB kats) - ++ maybeGroup makeCBCTest "CBC" (kat_CBC kats) - ++ maybeGroup makeCFBTest "CFB" (kat_CFB kats) - ++ maybeGroup makeCTRTest "CTR" (kat_CTR kats) - -- ++ maybeGroup makeXTSTest "XTS" (kat_XTS kats) - ++ maybeGroup makeAEADTest "AEAD" (kat_AEAD kats) - ) + -> Spec +testKATs kats cipher = describe "KAT" $ do + maybeGroup makeECBTest "ECB" (kat_ECB kats) + maybeGroup makeCBCTest "CBC" (kat_CBC kats) + maybeGroup makeCFBTest "CFB" (kat_CFB kats) + maybeGroup makeCTRTest "CTR" (kat_CTR kats) + -- XTS needs a 128-bit block, so testBlockCipher128 runs kat_XTS + maybeGroup makeAEADTest "AEAD" (kat_AEAD kats) where - makeECBTest i d = - [ testCase ("E" ++ i) (ecbEncrypt ctx (ecbPlaintext d) @?= ecbCiphertext d) - , testCase ("D" ++ i) (ecbDecrypt ctx (ecbCiphertext d) @?= ecbPlaintext d) - ] + makeECBTest i d = do + it ("E" ++ i) (ecbEncrypt ctx (ecbPlaintext d) `shouldBe` ecbCiphertext d) + it ("D" ++ i) (ecbDecrypt ctx (ecbCiphertext d) `shouldBe` ecbPlaintext d) where ctx = cipherInitNoErr (cipherMakeKey cipher $ ecbKey d) - makeCBCTest i d = - [ testCase ("E" ++ i) (cbcEncrypt ctx iv (cbcPlaintext d) @?= cbcCiphertext d) - , testCase ("D" ++ i) (cbcDecrypt ctx iv (cbcCiphertext d) @?= cbcPlaintext d) - ] + makeCBCTest i d = do + it ("E" ++ i) (cbcEncrypt ctx iv (cbcPlaintext d) `shouldBe` cbcCiphertext d) + it ("D" ++ i) (cbcDecrypt ctx iv (cbcCiphertext d) `shouldBe` cbcPlaintext d) where ctx = cipherInitNoErr (cipherMakeKey cipher $ cbcKey d) iv = cipherMakeIV cipher $ cbcIV d - makeCFBTest i d = - [ testCase ("E" ++ i) (cfbEncrypt ctx iv (cfbPlaintext d) @?= cfbCiphertext d) - , testCase ("D" ++ i) (cfbDecrypt ctx iv (cfbCiphertext d) @?= cfbPlaintext d) - ] + makeCFBTest i d = do + it ("E" ++ i) (cfbEncrypt ctx iv (cfbPlaintext d) `shouldBe` cfbCiphertext d) + it ("D" ++ i) (cfbDecrypt ctx iv (cfbCiphertext d) `shouldBe` cfbPlaintext d) where ctx = cipherInitNoErr (cipherMakeKey cipher $ cfbKey d) iv = cipherMakeIV cipher $ cfbIV d - makeCTRTest i d = - [ testCase ("E" ++ i) (ctrCombine ctx iv (ctrPlaintext d) @?= ctrCiphertext d) - , testCase ("D" ++ i) (ctrCombine ctx iv (ctrCiphertext d) @?= ctrPlaintext d) - ] + makeCTRTest i d = do + it ("E" ++ i) (ctrCombine ctx iv (ctrPlaintext d) `shouldBe` ctrCiphertext d) + it ("D" ++ i) (ctrCombine ctx iv (ctrCiphertext d) `shouldBe` ctrPlaintext d) where ctx = cipherInitNoErr (cipherMakeKey cipher $ ctrKey d) iv = cipherMakeIV cipher $ ctrIV d - {- - makeXTSTest i d = - [ testCase ("E" ++ i) (xtsEncrypt ctx iv 0 (xtsPlaintext d) @?= xtsCiphertext d) - , testCase ("D" ++ i) (xtsDecrypt ctx iv 0 (xtsCiphertext d) @?= xtsPlaintext d) - ] - where ctx1 = cipherInitNoErr (cipherMakeKey cipher $ xtsKey1 d) - ctx2 = cipherInitNoErr (cipherMakeKey cipher $ xtsKey2 d) - ctx = (ctx1, ctx2) - iv = cipherMakeIV cipher $ xtsIV d - -} - makeAEADTest i d = - [ testCase ("AE" ++ i) (etag @?= AuthTag (B.convert (aeadTag d))) - , testCase ("AD" ++ i) (dtag @?= AuthTag (B.convert (aeadTag d))) - , testCase ("E" ++ i) (ebs @?= aeadCiphertext d) - , testCase ("D" ++ i) (dbs @?= aeadPlaintext d) - ] + makeAEADTest i d = do + it ("AE" ++ i) (etag `shouldBe` AuthTag (B.convert (aeadTag d))) + it ("AD" ++ i) (dtag `shouldBe` AuthTag (B.convert (aeadTag d))) + it ("E" ++ i) (ebs `shouldBe` aeadCiphertext d) + it ("D" ++ i) (dbs `shouldBe` aeadPlaintext d) where ctx = cipherInitNoErr (cipherMakeKey cipher $ aeadKey d) aead = aeadInitNoErr (aeadMode d) ctx (aeadIV d) @@ -435,8 +418,8 @@ <$> generateKey <*> generatePlaintext -testBlockCipherBasic :: BlockCipher a => a -> [TestTree] -testBlockCipherBasic cipher = [testProperty "ECB" ecbProp] +testBlockCipherBasic :: BlockCipher a => a -> Spec +testBlockCipherBasic cipher = prop "ECB" ecbProp where ecbProp = toTests cipher toTests :: BlockCipher a => a -> (ECBUnit a -> Bool) @@ -444,13 +427,12 @@ testProperty_ECB (ECBUnit key (unPlaintextBS -> plaintext)) = withCtx key $ \ctx -> plaintext `assertEq` ecbDecrypt ctx (ecbEncrypt ctx plaintext) -testBlockCipherModes :: BlockCipher a => a -> [TestTree] -testBlockCipherModes cipher = - [ testProperty "CBC" cbcProp - , testProperty "CFB" cfbProp - , -- , testProperty "CFB8" cfb8Prop - testProperty "CTR" ctrProp - ] +testBlockCipherModes :: BlockCipher a => a -> Spec +testBlockCipherModes cipher = do + prop "CBC" cbcProp + prop "CFB" cfbProp + -- prop "CFB8" cfb8Prop + prop "CTR" ctrProp where (cbcProp, cfbProp, ctrProp) = toTests cipher toTests @@ -480,14 +462,13 @@ testProperty_CTR (CTRUnit key testIV (unPlaintext -> plaintext)) = withCtx key $ \ctx -> plaintext `assertEq` ctrCombine ctx testIV (ctrCombine ctx testIV plaintext) -testBlockCipherAEAD :: BlockCipher a => a -> [TestTree] -testBlockCipherAEAD cipher = - [ testProperty "OCB" (aeadProp AEAD_OCB) - , testProperty "CCM" (aeadProp (AEAD_CCM 0 CCM_M16 CCM_L2)) - , testProperty "EAX" (aeadProp AEAD_EAX) - , testProperty "CWC" (aeadProp AEAD_CWC) - , testProperty "GCM" (aeadProp AEAD_GCM) - ] +testBlockCipherAEAD :: BlockCipher a => a -> Spec +testBlockCipherAEAD cipher = do + prop "OCB" (aeadProp AEAD_OCB) + prop "CCM" (aeadProp (AEAD_CCM 0 CCM_M16 CCM_L2)) + prop "EAX" (aeadProp AEAD_EAX) + prop "CWC" (aeadProp AEAD_CWC) + prop "GCM" (aeadProp AEAD_GCM) where aeadProp = toTests cipher toTests :: BlockCipher a => a -> (AEADMode -> AEADUnit a -> Bool) @@ -518,8 +499,8 @@ CryptoPassed ctx -> f ctx {- -testBlockCipherXTS :: BlockCipher a => a -> [TestTree] -testBlockCipherXTS cipher = [testProperty "XTS" xtsProp] +testBlockCipherXTS :: BlockCipher a => a -> [Spec] +testBlockCipherXTS cipher = [prop "XTS" xtsProp] where xtsProp = toTests cipher toTests :: BlockCipher a => a -> (XTSUnit a -> Bool) toTests _ = testProperty_XTS @@ -531,26 +512,21 @@ -- | Test a generic block cipher for properties -- related to block cipher modes. -testModes :: BlockCipher a => a -> [TestTree] +testModes :: BlockCipher a => a -> Spec testModes cipher = - [ testGroup - "decrypt.encrypt==id" - -- (testBlockCipherBasic cipher ++ testBlockCipherModes cipher ++ testBlockCipherAEAD cipher ++ testBlockCipherXTS cipher) - ( testBlockCipherBasic cipher - ++ testBlockCipherModes cipher - ++ testBlockCipherAEAD cipher - ) - ] + describe "decrypt.encrypt==id" $ do + testBlockCipherBasic cipher + testBlockCipherModes cipher + testBlockCipherAEAD cipher -- | Test IV arithmetic (based on the cipher block size) -testIvArith :: BlockCipher a => a -> [TestTree] -testIvArith cipher = - [ testCase "nullIV is null" $ - True @=? B.all (== 0) (ivNull cipher) - , testProperty "ivAdd is linear" $ \a b -> do +testIvArith :: BlockCipher a => a -> Spec +testIvArith cipher = do + it "nullIV is null" $ + B.all (== 0) (ivNull cipher) `shouldBe` True + prop "ivAdd is linear" $ \a b -> do iv <- generateIvFromCipher cipher return $ ivAdd iv (a + b) `propertyEq` ivAdd (ivAdd iv a) b - ] where ivNull :: BlockCipher a => a -> IV a ivNull = const nullIV @@ -565,26 +541,53 @@ return $ cipherMakeIV c (B.pack $ zeros ++ ones) -- | Return tests for a specific blockcipher and a list of KATs -testBlockCipher :: BlockCipher a => KATs -> a -> TestTree -testBlockCipher kats cipher = - testGroup - (cipherName cipher) - ( (if kats == defaultKATs then [] else [testKATs kats cipher]) - ++ testModes cipher - ++ testIvArith cipher - ) +testBlockCipher :: BlockCipher a => KATs -> a -> Spec +testBlockCipher = testBlockCipherWith (return ()) +-- | The same for a cipher with a 128-bit block, whose KATs may include XTS. +-- The mode is defined only for that block size, so its vectors cannot run +-- from 'testBlockCipher', which promises nothing about the size -- which is +-- how they came to sit in the tree unused. +testBlockCipher128 :: BlockCipher128 a => KATs -> a -> Spec +testBlockCipher128 kats cipher = + testBlockCipherWith + (maybeGroup (makeXTSTest cipher) "XTS" (kat_XTS kats)) + kats + cipher + +testBlockCipherWith :: BlockCipher a => Spec -> KATs -> a -> Spec +testBlockCipherWith extra kats cipher = + describe (cipherName cipher) $ do + unless (kats == defaultKATs) $ testKATs kats cipher + extra + testModes cipher + testIvArith cipher + +makeXTSTest :: BlockCipher128 cipher => cipher -> String -> KAT_XTS -> Spec +makeXTSTest cipher i d = do + it ("E" ++ i) (xtsEncrypt ctx iv 0 (xtsPlaintext d) `shouldBe` xtsCiphertext d) + it ("D" ++ i) (xtsDecrypt ctx iv 0 (xtsCiphertext d) `shouldBe` xtsPlaintext d) + where + ctx = (keyed (xtsKey1 d), keyed (xtsKey2 d)) + iv = cipherMakeIV cipher (xtsIV d) + + keyed :: BlockCipher c => ByteString -> c + keyed k = + case cipherInit k of + CryptoPassed a -> a + CryptoFailed e -> error (show e) + cipherMakeKey :: Cipher cipher => cipher -> ByteString -> Key cipher cipherMakeKey _ bs = Key bs cipherMakeIV :: BlockCipher cipher => cipher -> ByteString -> IV cipher cipherMakeIV _ bs = fromJust $ makeIV bs -maybeGroup :: (String -> t -> [TestTree]) -> TestName -> [t] -> [TestTree] +maybeGroup :: (String -> t -> Spec) -> String -> [t] -> Spec maybeGroup mkTest groupName l - | null l = [] + | null l = return () | otherwise = - [testGroup groupName (concatMap (\(i, d) -> mkTest (show i) d) $ zip nbs l)] + describe groupName $ mapM_ (\(i, d) -> mkTest (show i) d) (zip nbs l) where nbs :: [Int] nbs = [0 ..] diff --git a/tests/BlockCipher/AES/CBC.hs b/tests/BlockCipher/AES/CBC.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/CBC.hs @@ -0,0 +1,460 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.AES.CBC where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () + +type KATCBC = (B.ByteString, B.ByteString, B.ByteString, B.ByteString) + +vectors_aes128_enc + , vectors_aes128_dec + , vectors_aes192_enc + , vectors_aes192_dec + , vectors_aes256_enc + , vectors_aes256_dec + :: [KATCBC] +vectors_aes128_enc = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x66\xe9\x4b\xd4\xef\x8a\x2c\x3b\x88\x4c\xfa\x59\xca\x34\x2b\x2e" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xb6\xae\xaf\xfa\x75\x2d\xc0\x8b\x51\x63\x97\x31\x76\x1a\xed\x00" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xcb\x64\xcf\x3f\x42\x2a\xe8\x4b\xb9\x0e\x3a\xb4\xdb\xa7\xbd\x86" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xe5\xb5\x07\x7f\x93\x46\x46\x2c\x62\xa0\x75\xc0\xc7\x08\xee\x96" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xe1\x4d\x5d\x0e\xe2\x77\x15\xdf\x08\xb4\x15\x2b\xa2\x3d\xa8\xe0" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x5e\x77\xe5\x9f\x8f\x85\x94\x34\x89\xa2\x41\x49\xc7\x5f\x4e\xc9" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x8f\x42\xc2\x4b\xee\x6e\x63\x47\x2b\x16\x5a\xa9\x41\x31\x2f\x7c" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xb0\xea\x4a\xc0\xd2\x5c\xcd\x7c\x82\xcb\x8a\x30\x68\xc6\xfe\x2e" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\xe1\x4d\x5d\x0e\xe2\x77\x15\xdf\x08\xb4\x15\x2b\xa2\x3d\xa8\xe0" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x17\xd6\x14\xf3\x79\xa9\x35\x90\x77\xe9\x55\x77\xfd\x31\xc2\x0a" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x8f\x42\xc2\x4b\xee\x6e\x63\x47\x2b\x16\x5a\xa9\x41\x31\x2f\x7c" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\xe5\xb5\x07\x7f\x93\x46\x46\x2c\x62\xa0\x75\xc0\xc7\x08\xee\x96" + ) + ] +vectors_aes192_enc = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xaa\xe0\x69\x92\xac\xbf\x52\xa3\xe8\xf4\xa9\x6e\xc9\x30\x0b\xd7" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x5f\x88\xef\x3f\xbd\xeb\xf2\xe4\xe2\x66\x65\x12\xd3\xbc\xb7\x0f" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xdb\x42\xf5\x1c\xd2\x0e\xca\xd2\x9e\xb0\x13\x2b\x0f\xaa\x4b\x85" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xda\xb4\x01\x5f\x98\x70\x25\xeb\xb8\xa8\x5f\x3c\x7f\x73\x70\x19" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xcf\x1e\xce\x3c\x44\xb0\x78\xfb\x27\xcb\x0a\x3e\x07\x1b\x08\x20" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x98\xb8\x95\xa1\x45\xca\x4e\x0b\xf8\x3e\x69\x32\x81\xc1\xa0\x97" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xf2\xf0\xae\xd8\xcd\xc9\x21\xca\x4b\x55\x84\x5d\xa4\x15\x21\xc2" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x5e\xea\x4b\x13\xdd\xd9\x17\x12\xb0\x14\xe2\x82\x2d\x18\x76\xfb" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\xcf\x1e\xce\x3c\x44\xb0\x78\xfb\x27\xcb\x0a\x3e\x07\x1b\x08\x20" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\xeb\x8c\x17\x30\x90\xc7\x5b\x77\xd6\x72\xb4\x57\xa7\x78\xd9\xd0" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\xf2\xf0\xae\xd8\xcd\xc9\x21\xca\x4b\x55\x84\x5d\xa4\x15\x21\xc2" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\xda\xb4\x01\x5f\x98\x70\x25\xeb\xb8\xa8\x5f\x3c\x7f\x73\x70\x19" + ) + ] +vectors_aes256_enc = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xdc\x95\xc0\x78\xa2\x40\x89\x89\xad\x48\xa2\x14\x92\x84\x20\x87" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x72\x98\xca\xa5\x65\x03\x1e\xad\xc6\xce\x23\xd2\x3e\xa6\x63\x78" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xf4\x35\xa1\x11\xa3\xe4\xa1\x94\x49\x19\xf9\x12\xc5\xa2\x41\xde" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x91\xc0\x87\x62\x87\x6d\xcc\xf9\xba\x20\x4a\x33\x76\x8f\xa5\xfe" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x7b\xc3\x02\x6c\xd7\x37\x10\x3e\x62\x90\x2b\xcd\x18\xfb\x01\x63" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x9c\xac\x94\xc6\xb4\x85\x61\xf8\xff\xaa\xa7\x86\x16\xba\x48\x92" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\xf9\xc7\x44\x4b\xb0\xcc\x80\x6c\x7c\x39\xee\x22\x11\xf1\x46" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x6d\xed\xd0\xa3\xe6\x94\xa0\xde\x65\x1d\x68\xa6\xb5\x5a\x64\xa2" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x7b\xc3\x02\x6c\xd7\x37\x10\x3e\x62\x90\x2b\xcd\x18\xfb\x01\x63" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x62\xae\x12\xf3\x24\xbf\xea\x08\xd5\xf6\x75\xb5\x13\x02\x6b\xbf" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\xf9\xc7\x44\x4b\xb0\xcc\x80\x6c\x7c\x39\xee\x22\x11\xf1\x46" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x91\xc0\x87\x62\x87\x6d\xcc\xf9\xba\x20\x4a\x33\x76\x8f\xa5\xfe" + ) + ] +vectors_aes128_dec = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x14\x0f\x0f\x10\x11\xb5\x22\x3d\x79\x58\x77\x17\xff\xd9\xec\x3a" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x33\x08\x32\x40\xd6\x5c\xbc\x72\xaa\x0b\x44\xf3\xe1\x9e\xa9\x5a" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x65\x0a\x42\xa0\x3c\x4b\x93\xa4\xb7\x43\xdc\x9e\x9c\xf4\xc0\x9b" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x80\xcd\x20\xe1\xbd\x89\x3c\x5e\xe4\x20\x76\x85\xb0\x9a\x0e\x3e" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x15\x0e\x0e\x11\x10\xb4\x23\x3c\x78\x59\x76\x16\xfe\xd8\xed\x3b" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x32\x09\x33\x41\xd7\x5d\xbd\x73\xab\x0a\x45\xf2\xe0\x9f\xa8\x5b" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x64\x0b\x43\xa1\x3d\x4a\x92\xa5\xb6\x42\xdd\x9f\x9d\xf5\xc1\x9a" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x81\xcc\x21\xe0\xbc\x88\x3d\x5f\xe5\x21\x77\x84\xb1\x9b\x0f\x3f" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\xf5\x06\x41\x7e\x6a\x8f\xbc\x32\xdd\xa5\x52\x73\xbf\x9f\x4d\x5c" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\xbf\x6d\x28\xac\x20\xc9\x1d\x65\xa9\xd4\xb0\x96\xc2\xd5\xa5\x09" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x5f\x2a\x46\xab\x8d\xb9\x5b\x22\x15\xfe\x1a\xa4\xdd\x69\x59\x26" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x71\x9b\x21\xb5\x39\x7c\x2f\x16\x7c\x8b\x45\x22\xb5\x20\xec\x2e" + ) + ] +vectors_aes192_dec = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x13\x46\x0e\x87\xa8\xfc\x02\x3e\xf2\x50\x1a\xfe\x7f\xf5\x1c\x51" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x91\x75\x27\xfc\xd4\xa0\x6f\x32\x27\x29\x90\x14\xca\xde\xd4\x1a" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x29\x64\x80\xb6\xa5\xd6\xcf\xb3\x78\x3f\x21\x6b\x80\x31\x3d\xb3" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xbc\xa5\x06\x07\xd0\x67\x30\x85\x2d\x3a\x50\x4b\x68\x0a\x19\xcc" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x12\x47\x0f\x86\xa9\xfd\x03\x3f\xf3\x51\x1b\xff\x7e\xf4\x1d\x50" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x90\x74\x26\xfd\xd5\xa1\x6e\x33\x26\x28\x91\x15\xcb\xdf\xd5\x1b" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x28\x65\x81\xb7\xa4\xd7\xce\xb2\x79\x3e\x20\x6a\x81\x30\x3c\xb2" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xbd\xa4\x07\x06\xd1\x66\x31\x84\x2c\x3b\x51\x4a\x69\x0b\x18\xcd" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x38\xf9\xf9\xd1\x7e\x2c\x82\xaf\xdc\xed\x68\x03\xb6\x31\x46\x3e" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x35\x4e\xc1\x01\x0f\x17\x50\x5e\x63\x37\x40\x4b\x9a\xf2\xc0\x5c" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\xa7\x7c\xc9\xd1\x4f\x44\xf7\xf7\xcc\x45\x80\x83\x19\xb7\xa4\x71" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\xf9\x1d\xb1\x13\x0b\xd1\xc0\x66\x9f\xfa\xc2\x0e\xbe\xdd\xcb\xca" + ) + ] +vectors_aes256_dec = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x67\x67\x1c\xe1\xfa\x91\xdd\xeb\x0f\x8f\xbb\xb3\x66\xb5\x31\xb4" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x7b\xd3\xfb\x90\x65\x56\x9f\x39\x8b\x09\xcb\x93\x4b\x1e\x01\x23" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xda\xa8\xbf\x5c\xde\x2e\x52\x45\x5f\xa3\xb3\xfe\x33\x32\x47\xca" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x83\x24\xdc\xb4\x30\x12\x73\x6c\xed\x58\xab\x8f\x4b\x05\xca\x0b" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x66\x66\x1d\xe0\xfb\x90\xdc\xea\x0e\x8e\xba\xb2\x67\xb4\x30\xb5" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x7a\xd2\xfa\x91\x64\x57\x9e\x38\x8a\x08\xca\x92\x4a\x1f\x00\x22" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\xdb\xa9\xbe\x5d\xdf\x2f\x53\x44\x5e\xa2\xb2\xff\x32\x33\x46\xcb" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x82\x25\xdd\xb5\x31\x13\x72\x6d\xec\x59\xaa\x8e\x4a\x04\xcb\x0a" + ) + , + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x68\xe9\x07\x16\xe3\x66\x1b\x1d\xb1\x89\x74\xb0\x9c\x46\x47\xe4" + ) + , + ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , "\x7f\xb9\xeb\xa4\xd3\x5f\x70\x40\xab\x52\xec\xd2\x3b\x48\xb7\x6e" + ) + , + ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" + , "\x6c\x58\x0f\x41\x82\x36\xbc\xff\x64\x1d\xac\xa7\x3e\x34\x11\x18" + ) + , + ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" + , "\x3f\x62\xd6\x8c\xb1\xf7\x62\x28\xa4\xc3\x82\x4f\x8b\x24\xe7\x4b" + ) + ] diff --git a/tests/BlockCipher/AES/CCM.hs b/tests/BlockCipher/AES/CCM.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/CCM.hs @@ -0,0 +1,263 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.AES.CCM where + +import qualified Data.ByteString as B + +-- (key, iv, header, in, out+atag, taglen) +type KATCCM = + (B.ByteString, B.ByteString, B.ByteString, B.ByteString, B.ByteString, Int) + +vectors_aes128_enc :: [KATCCM] +vectors_aes128_enc = + [ + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x03\x02\x01\x00\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" + , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x58\x8c\x97\x9a\x61\xc6\x63\xd2\xf0\x66\xd0\xc2\xc0\xf9\x89\x80\x6d\x5f\x6b\x61\xda\xc3\x84\x17\xe8\xd1\x2c\xfd\xf9\x26\xe0" + , {- M = -} 8 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x04\x03\x02\x01\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" + , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x72\xc9\x1a\x36\xe1\x35\xf8\xcf\x29\x1c\xa8\x94\x08\x5c\x87\xe3\xcc\x15\xc4\x39\xc9\xe4\x3a\x3b\xa0\x91\xd5\x6e\x10\x40\x09\x16" + , {- M = -} 8 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x05\x04\x03\x02\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" + , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x51\xb1\xe5\xf4\x4a\x19\x7d\x1d\xa4\x6b\x0f\x8e\x2d\x28\x2a\xe8\x71\xe8\x38\xbb\x64\xda\x85\x96\x57\x4a\xda\xa7\x6f\xbd\x9f\xb0\xc5" + , {- M = -} 8 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x06\x05\x04\x03\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" + , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\xa2\x8c\x68\x65\x93\x9a\x9a\x79\xfa\xaa\x5c\x4c\x2a\x9d\x4a\x91\xcd\xac\x8c\x96\xc8\x61\xb9\xc9\xe6\x1e\xf1" + , {- M = -} 8 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x07\x06\x05\x04\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" + , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\xdc\xf1\xfb\x7b\x5d\x9e\x23\xfb\x9d\x4e\x13\x12\x53\x65\x8a\xd8\x6e\xbd\xca\x3e\x51\xe8\x3f\x07\x7d\x9c\x2d\x93" + , {- M = -} 8 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x08\x07\x06\x05\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" + , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x6f\xc1\xb0\x11\xf0\x06\x56\x8b\x51\x71\xa4\x2d\x95\x3d\x46\x9b\x25\x70\xa4\xbd\x87\x40\x5a\x04\x43\xac\x91\xcb\x94" + , {- M = -} 8 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x09\x08\x07\x06\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" + , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x01\x35\xd1\xb2\xc9\x5f\x41\xd5\xd1\xd4\xfe\xc1\x85\xd1\x66\xb8\x09\x4e\x99\x9d\xfe\xd9\x6c\x04\x8c\x56\x60\x2c\x97\xac\xbb\x74\x90" + , {- M = -} 10 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x0a\x09\x08\x07\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" + , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x7b\x75\x39\x9a\xc0\x83\x1d\xd2\xf0\xbb\xd7\x58\x79\xa2\xfd\x8f\x6c\xae\x6b\x6c\xd9\xb7\xdb\x24\xc1\x7b\x44\x33\xf4\x34\x96\x3f\x34\xb4" + , {- M = -} 10 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x0b\x0a\x09\x08\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" + , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x82\x53\x1a\x60\xcc\x24\x94\x5a\x4b\x82\x79\x18\x1a\xb5\xc8\x4d\xf2\x1c\xe7\xf9\xb7\x3f\x42\xe1\x97\xea\x9c\x07\xe5\x6b\x5e\xb1\x7e\x5f\x4e" + , {- M = -} 10 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x0c\x0b\x0a\x09\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" + , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x07\x34\x25\x94\x15\x77\x85\x15\x2b\x07\x40\x98\x33\x0a\xbb\x14\x1b\x94\x7b\x56\x6a\xa9\x40\x6b\x4d\x99\x99\x88\xdd" + , {- M = -} 10 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x0d\x0c\x0b\x0a\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" + , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x67\x6b\xb2\x03\x80\xb0\xe3\x01\xe8\xab\x79\x59\x0a\x39\x6d\xa7\x8b\x83\x49\x34\xf5\x3a\xa2\xe9\x10\x7a\x8b\x6c\x02\x2c" + , {- M = -} 10 + ) + , + ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" + , {- iv = -} "\x00\x00\x00\x0e\x0d\x0c\x0b\xa0\xa1\xa2\xa3\xa4\xa5" + , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" + , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" + , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\xc0\xff\xa0\xd6\xf0\x5b\xdb\x67\xf2\x4d\x43\xa4\x33\x8d\x2a\xa4\xbe\xd7\xb2\x0e\x43\xcd\x1a\xa3\x16\x62\xe7\xad\x65\xd6\xdb" + , {- M = -} 10 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x41\x2b\x4e\xa9\xcd\xbe\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x0b\xe1\xa8\x8b\xac\xe0\x18\xb1" + , {- in = -} "\x08\xe8\xcf\x97\xd8\x20\xea\x25\x84\x60\xe9\x6a\xd9\xcf\x52\x89\x05\x4d\x89\x5c\xea\xc4\x7c" + , {- out = -} "\x0b\xe1\xa8\x8b\xac\xe0\x18\xb1\x4c\xb9\x7f\x86\xa2\xa4\x68\x9a\x87\x79\x47\xab\x80\x91\xef\x53\x86\xa6\xff\xbd\xd0\x80\xf8\xe7\x8c\xf7\xcb\x0c\xdd\xd7\xb3" + , {- M = -} 8 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x33\x56\x8e\xf7\xb2\x63\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x63\x01\x8f\x76\xdc\x8a\x1b\xcb" + , {- in = -} "\x90\x20\xea\x6f\x91\xbd\xd8\x5a\xfa\x00\x39\xba\x4b\xaf\xf9\xbf\xb7\x9c\x70\x28\x94\x9c\xd0\xec" + , {- out = -} "\x63\x01\x8f\x76\xdc\x8a\x1b\xcb\x4c\xcb\x1e\x7c\xa9\x81\xbe\xfa\xa0\x72\x6c\x55\xd3\x78\x06\x12\x98\xc8\x5c\x92\x81\x4a\xbc\x33\xc5\x2e\xe8\x1d\x7d\x77\xc0\x8a" + , {- M = -} 8 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x10\x3f\xe4\x13\x36\x71\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\xaa\x6c\xfa\x36\xca\xe8\x6b\x40" + , {- in = -} "\xb9\x16\xe0\xea\xcc\x1c\x00\xd7\xdc\xec\x68\xec\x0b\x3b\xbb\x1a\x02\xde\x8a\x2d\x1a\xa3\x46\x13\x2e" + , {- out = -} "\xaa\x6c\xfa\x36\xca\xe8\x6b\x40\xb1\xd2\x3a\x22\x20\xdd\xc0\xac\x90\x0d\x9a\xa0\x3c\x61\xfc\xf4\xa5\x59\xa4\x41\x77\x67\x08\x97\x08\xa7\x76\x79\x6e\xdb\x72\x35\x06" + , {- M = -} 8 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x76\x4c\x63\xb8\x05\x8e\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\xd0\xd0\x73\x5c\x53\x1e\x1b\xec\xf0\x49\xc2\x44" + , {- in = -} "\x12\xda\xac\x56\x30\xef\xa5\x39\x6f\x77\x0c\xe1\xa6\x6b\x21\xf7\xb2\x10\x1c" + , {- out = -} "\xd0\xd0\x73\x5c\x53\x1e\x1b\xec\xf0\x49\xc2\x44\x14\xd2\x53\xc3\x96\x7b\x70\x60\x9b\x7c\xbb\x7c\x49\x91\x60\x28\x32\x45\x26\x9a\x6f\x49\x97\x5b\xca\xde\xaf" + , {- M = -} 8 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\xf8\xb6\x78\x09\x4e\x3b\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x77\xb6\x0f\x01\x1c\x03\xe1\x52\x58\x99\xbc\xae" + , {- in = -} "\xe8\x8b\x6a\x46\xc7\x8d\x63\xe5\x2e\xb8\xc5\x46\xef\xb5\xde\x6f\x75\xe9\xcc\x0d" + , {- out = -} "\x77\xb6\x0f\x01\x1c\x03\xe1\x52\x58\x99\xbc\xae\x55\x45\xff\x1a\x08\x5e\xe2\xef\xbf\x52\xb2\xe0\x4b\xee\x1e\x23\x36\xc7\x3e\x3f\x76\x2c\x0c\x77\x44\xfe\x7e\x3c" + , {- M = -} 8 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\xd5\x60\x91\x2d\x3f\x70\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\xcd\x90\x44\xd2\xb7\x1f\xdb\x81\x20\xea\x60\xc0" + , {- in = -} "\x64\x35\xac\xba\xfb\x11\xa8\x2e\x2f\x07\x1d\x7c\xa4\xa5\xeb\xd9\x3a\x80\x3b\xa8\x7f" + , {- out = -} "\xcd\x90\x44\xd2\xb7\x1f\xdb\x81\x20\xea\x60\xc0\x00\x97\x69\xec\xab\xdf\x48\x62\x55\x94\xc5\x92\x51\xe6\x03\x57\x22\x67\x5e\x04\xc8\x47\x09\x9e\x5a\xe0\x70\x45\x51" + , {- M = -} 8 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x42\xff\xf8\xf1\x95\x1c\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\xd8\x5b\xc7\xe6\x9f\x94\x4f\xb8" + , {- in = -} "\x8a\x19\xb9\x50\xbc\xf7\x1a\x01\x8e\x5e\x67\x01\xc9\x17\x87\x65\x98\x09\xd6\x7d\xbe\xdd\x18" + , {- out = -} "\xd8\x5b\xc7\xe6\x9f\x94\x4f\xb8\xbc\x21\x8d\xaa\x94\x74\x27\xb6\xdb\x38\x6a\x99\xac\x1a\xef\x23\xad\xe0\xb5\x29\x39\xcb\x6a\x63\x7c\xf9\xbe\xc2\x40\x88\x97\xc6\xba" + , {- M = -} 10 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x92\x0f\x40\xe5\x6c\xdc\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x74\xa0\xeb\xc9\x06\x9f\x5b\x37" + , {- in = -} "\x17\x61\x43\x3c\x37\xc5\xa3\x5f\xc1\xf3\x9f\x40\x63\x02\xeb\x90\x7c\x61\x63\xbe\x38\xc9\x84\x37" + , {- out = -} "\x74\xa0\xeb\xc9\x06\x9f\x5b\x37\x58\x10\xe6\xfd\x25\x87\x40\x22\xe8\x03\x61\xa4\x78\xe3\xe9\xcf\x48\x4a\xb0\x4f\x44\x7e\xff\xf6\xf0\xa4\x77\xcc\x2f\xc9\xbf\x54\x89\x44" + , {- M = -} 10 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x27\xca\x0c\x71\x20\xbc\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x44\xa3\xaa\x3a\xae\x64\x75\xca" + , {- in = -} "\xa4\x34\xa8\xe5\x85\x00\xc6\xe4\x15\x30\x53\x88\x62\xd6\x86\xea\x9e\x81\x30\x1b\x5a\xe4\x22\x6b\xfa" + , {- out = -} "\x44\xa3\xaa\x3a\xae\x64\x75\xca\xf2\xbe\xed\x7b\xc5\x09\x8e\x83\xfe\xb5\xb3\x16\x08\xf8\xe2\x9c\x38\x81\x9a\x89\xc8\xe7\x76\xf1\x54\x4d\x41\x51\xa4\xed\x3a\x8b\x87\xb9\xce" + , {- M = -} 10 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x5b\x8c\xcb\xcd\x9a\xf8\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\xec\x46\xbb\x63\xb0\x25\x20\xc3\x3c\x49\xfd\x70" + , {- in = -} "\xb9\x6b\x49\xe2\x1d\x62\x17\x41\x63\x28\x75\xdb\x7f\x6c\x92\x43\xd2\xd7\xc2" + , {- out = -} "\xec\x46\xbb\x63\xb0\x25\x20\xc3\x3c\x49\xfd\x70\x31\xd7\x50\xa0\x9d\xa3\xed\x7f\xdd\xd4\x9a\x20\x32\xaa\xbf\x17\xec\x8e\xbf\x7d\x22\xc8\x08\x8c\x66\x6b\xe5\xc1\x97" + , {- M = -} 10 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x3e\xbe\x94\x04\x4b\x9a\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x47\xa6\x5a\xc7\x8b\x3d\x59\x42\x27\xe8\x5e\x71" + , {- in = -} "\xe2\xfc\xfb\xb8\x80\x44\x2c\x73\x1b\xf9\x51\x67\xc8\xff\xd7\x89\x5e\x33\x70\x76" + , {- out = -} "\x47\xa6\x5a\xc7\x8b\x3d\x59\x42\x27\xe8\x5e\x71\xe8\x82\xf1\xdb\xd3\x8c\xe3\xed\xa7\xc2\x3f\x04\xdd\x65\x07\x1e\xb4\x13\x42\xac\xdf\x7e\x00\xdc\xce\xc7\xae\x52\x98\x7d" + , {- M = -} 10 + ) + , + ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" + , {- iv = -} "\x00\x8d\x49\x3b\x30\xae\x8b\x3c\x96\x96\x76\x6c\xfa" + , {- hdr = -} "\x6e\x37\xa6\xef\x54\x6d\x95\x5d\x34\xab\x60\x59" + , {- in = -} "\xab\xf2\x1c\x0b\x02\xfe\xb8\x8f\x85\x6d\xf4\xa3\x73\x81\xbc\xe3\xcc\x12\x85\x17\xd4" + , {- out = -} "\x6e\x37\xa6\xef\x54\x6d\x95\x5d\x34\xab\x60\x59\xf3\x29\x05\xb8\x8a\x64\x1b\x04\xb9\xc9\xff\xb5\x8c\xc3\x90\x90\x0f\x3d\xa1\x2a\xb1\x6d\xce\x9e\x82\xef\xa1\x6d\xa6\x20\x59" + , {- M = -} 10 + ) + ] + +-- From OpenSSL 3.5, in the same shape: header, ciphertext and tag as one +-- string. The suite had CCM vectors at 128 bits only. +vectors_aes192_enc :: [KATCCM] +vectors_aes192_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac" + , "\x50\x51\x52\x53\x54\x55\x56\x57" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d" + , "\x50\x51\x52\x53\x54\x55\x56\x57\xea\xaa\x82\xdb\xd8\x0f\xec\xb3\xe5\xf9\x8a\x66\x1b\x5c\x0c\x46\xc0\x91\xec\xbe\x47\xdd\x46\x13\x1c\xbc\xa4\x27\x96\xaf\xc4" + , 8 + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\xea\xaa\x82\xdb\xd8\x0f\xec\xb3\xe5\xf9\x8a\x66\x1b\x5c\x0c\x46\xc0\x91\xec\xbe\x47\xdd\x46\x61\x34\x59\x25\xf2\x67\x4d\x0e\x97\x5c\x63\x50\x49\x9c\xc2\x24\x13\x2d\x04\x73\xb2\xbc\x9c\x8d\xc1\xc1\xde\xcb\x5c\xbf\x45\xe4\x42\x82\x75\x01\xe0\x9b\x69\xe2\x9c\x45\x10\x0b\xd2\xe2\x2d\x13\xef\x92\x4b\x5c\xe8" + , 16 + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac" + , "" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec\xf3\xfa" + , "\xea\xaa\x82\xdb\xd8\x0f\xec\xb3\xe5\xf9\x8a\x66\x1b\x5c\x0c\x46\xc0\x91\xec\xbe\x47\xdd\x46\x61\x34\x59\x25\xf2\x67\x4d\x0e\x97\x5c\x63\x50\x49\x9c\xc2\x24\x13\x2d\x04\x73\xb2\xbc\x9c\x8d\xc1\xc1\xde\xcb\x5c\xbf\x45\xe4\x42\x82\x75\x01\xe0\x42\x93\x60\x1b\xa0\x42\x2f\xb8\xdf\x2c\x62\xb8\x98\x9e\xe4\x6e\x7a\xe1\xda\xaa\x3e\x79\x14\x7d\x42\xd0\xaf\xc0\xbd\x89\x71\x81\x08\xad\x97\x29\xfd\x7c\xab\x91\x6c\x98\x66\x13\xc7\x67\xad\x8a\xa3\x12\x7d\xa3\x28\xd2\xcc\x6d\xf1\x84\xb7\x42\x9d\x67\x8d\xa9\xab\x5f\x87\x18\x28\x0b\x47\xb5\xe1\x44\xd1\x1e\x1e\x26\x14\xc7\xec\x15\x04\xa1\x54\x46\xca\xac\xb7\x2e\xf7\xae\x3f\xc0\xd5\xc6\x84\x4c\xa9\xc1\x06\x1c" + , 16 + ) + ] + +vectors_aes256_enc :: [KATCCM] +vectors_aes256_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac" + , "\x50\x51\x52\x53\x54\x55\x56\x57" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d" + , "\x50\x51\x52\x53\x54\x55\x56\x57\xad\x3e\xcd\xd8\x8e\x85\x91\xc7\x44\xab\x19\x1c\xa6\x37\x1f\xf2\xf2\x09\x1d\xc1\xb0\x48\x87\xdc\x9e\x16\xe2\x37\xbf\x01\x89" + , 8 + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\xad\x3e\xcd\xd8\x8e\x85\x91\xc7\x44\xab\x19\x1c\xa6\x37\x1f\xf2\xf2\x09\x1d\xc1\xb0\x48\x87\xa3\xc8\x80\x9d\xd1\x15\xd6\xab\x3c\x81\xdc\xa8\x90\x01\x34\xa3\x49\xa8\xc7\x84\xff\x55\x5f\x57\x28\x67\x8e\x1d\x08\x46\x01\x4d\x93\x2c\x80\x1f\x0e\x37\x7a\x9e\xee\x95\xc1\x1a\xb1\xbe\xc1\xeb\x10\xc8\x88\x32\xe8" + , 16 + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac" + , "" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec\xf3\xfa" + , "\xad\x3e\xcd\xd8\x8e\x85\x91\xc7\x44\xab\x19\x1c\xa6\x37\x1f\xf2\xf2\x09\x1d\xc1\xb0\x48\x87\xa3\xc8\x80\x9d\xd1\x15\xd6\xab\x3c\x81\xdc\xa8\x90\x01\x34\xa3\x49\xa8\xc7\x84\xff\x55\x5f\x57\x28\x67\x8e\x1d\x08\x46\x01\x4d\x93\x2c\x80\x1f\x0e\x42\x24\x4c\xb2\x9a\xa4\x3d\x8a\x5a\x72\x0d\x98\x45\x81\xa9\x53\x11\x60\xf8\x7c\x3d\x13\x6d\x2b\x27\x45\x76\x08\xb1\xa7\xe5\x6e\x73\x7b\xab\x7f\x21\x4a\x6a\xfe\xc8\x12\xfc\x37\x1b\x37\xc6\x34\x82\x3d\xa4\x4a\x79\x71\xdf\x05\x1d\xb4\x42\x02\x4d\xcd\x93\x13\xcf\x53\x23\xd9\x68\xf6\x7a\x75\xb4\x36\xd8\x67\x31\x9a\xbf\xfb\xff\x0b\x47\xd5\x5e\x16\x9e\xb6\x27\xef\x9a\xe4\x57\xb6\x8e\x15\x50\x41\xd1\x29\x99\x28" + , 16 + ) + ] diff --git a/tests/BlockCipher/AES/CTR.hs b/tests/BlockCipher/AES/CTR.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/CTR.hs @@ -0,0 +1,82 @@ +{-# LANGUAGE OverloadedStrings #-} + +-- | Counter mode, which had no known-answer test at any key size: the +-- suite checked it only by encrypting and decrypting again, which a +-- wrong-but-consistent implementation passes. From OpenSSL 3.5. +-- +-- The lengths are one block, nine blocks, and eight and a bit, so that +-- both the group of eight the hardware paths take at a time and the +-- block-at-a-time tail after it are covered. +module BlockCipher.AES.CTR where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () + +-- (key, iv, input, out) +type KATCTR = (B.ByteString, B.ByteString, B.ByteString, B.ByteString) + +vectors_aes128_enc :: [KATCTR] +vectors_aes128_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c" + , "\xca\x1b\xb2\x0a\xf3\x52\xb1\x9b\xf0\xb0\xdc\xde\xb3\x0b\x77\x47" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec" + , "\xca\x1b\xb2\x0a\xf3\x52\xb1\x9b\xf0\xb0\xdc\xde\xb3\x0b\x77\x47\x38\x57\x21\x94\xc8\xf7\x8d\xc3\x7b\x19\x83\xa3\x08\x94\x57\xdd\xf9\x33\x8f\xd5\x03\x3e\xef\x0d\x05\x74\x18\xae\xe5\x2a\x25\xc4\xa3\x44\xed\x31\xa7\x06\x32\x44\x63\x7d\xe5\xc4\xd6\x8c\x42\xe5\x23\xb5\x7d\x54\x08\x30\x16\x6c\x7c\x3b\x54\x24\x67\x23\x18\x2c\xdb\x08\xc4\x38\x65\xca\xb7\x0a\x48\xe1\xa9\x19\x4d\x9e\x97\xd8\x53\x50\x20\xfa\xe1\x27\xc4\x07\x3a\x6b\xb4\x94\x49\xdd\x07\x87\xfb\x0b\x15\x88\xd2\x98\x98\xc6\xeb\x5a\x7d\xbf\x38\x1d\x78\x56\x44\x6e\xd3\x54\xfd\x48\x20\x4c\xc2\xd5\x14\xf5\xa2\xe0\xf7\xa3" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a" + , "\xca\x1b\xb2\x0a\xf3\x52\xb1\x9b\xf0\xb0\xdc\xde\xb3\x0b\x77\x47\x38\x57\x21\x94\xc8\xf7\x8d\xc3\x7b\x19\x83\xa3\x08\x94\x57\xdd\xf9\x33\x8f\xd5\x03\x3e\xef\x0d\x05\x74\x18\xae\xe5\x2a\x25\xc4\xa3\x44\xed\x31\xa7\x06\x32\x44\x63\x7d\xe5\xc4\xd6\x8c\x42\xe5\x23\xb5\x7d\x54\x08\x30\x16\x6c\x7c\x3b\x54\x24\x67\x23\x18\x2c\xdb\x08\xc4\x38\x65\xca\xb7\x0a\x48\xe1\xa9\x19\x4d\x9e\x97\xd8\x53\x50\x20\xfa\xe1\x27\xc4\x07\x3a\x6b\xb4\x94\x49\xdd\x07\x87\xfb\x0b\x15\x88\xd2\x98\x98\xc6\xeb\x5a\x7d\xbf\x38\x1d\x78\x56\x44\x6e" + ) + ] + +vectors_aes192_enc :: [KATCTR] +vectors_aes192_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c" + , "\xe6\xd7\xe3\x96\xd1\x4c\xd0\x2a\x6a\x22\xe5\x71\x61\xcd\x1d\xc0" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec" + , "\xe6\xd7\xe3\x96\xd1\x4c\xd0\x2a\x6a\x22\xe5\x71\x61\xcd\x1d\xc0\x85\xfc\x17\x66\x30\x7c\x51\x03\xbb\x13\x61\x16\x60\x11\xff\xaa\x5a\xbd\x39\xf2\xcf\x3c\x87\x73\x39\x02\x6b\xaa\x8c\x71\xce\xb6\x3a\x30\x81\xad\x8b\xbf\xc7\x1d\x0c\xd8\x2b\x65\xa8\xe9\x08\x7c\x46\xec\xaf\x61\xe3\xa8\x66\x11\x32\x8b\x17\x37\xb1\xec\x3e\xd6\x12\x28\xb0\xcc\x7d\x38\xc3\x0e\xee\xf0\xc8\xbf\x82\x47\xb9\x8c\x2a\x59\x89\xa3\x3d\x9a\x47\x09\xb6\xdf\xc5\xb4\x1d\x06\x9d\xe5\x8e\x7b\x82\x0f\xf6\x13\xad\xf4\x9c\xc3\x27\x94\x56\xe5\x70\x17\xd9\x24\x99\x77\xb7\xcd\x06\x17\x91\x9b\xf5\x7d\x29\x95\xa3\xe2" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a" + , "\xe6\xd7\xe3\x96\xd1\x4c\xd0\x2a\x6a\x22\xe5\x71\x61\xcd\x1d\xc0\x85\xfc\x17\x66\x30\x7c\x51\x03\xbb\x13\x61\x16\x60\x11\xff\xaa\x5a\xbd\x39\xf2\xcf\x3c\x87\x73\x39\x02\x6b\xaa\x8c\x71\xce\xb6\x3a\x30\x81\xad\x8b\xbf\xc7\x1d\x0c\xd8\x2b\x65\xa8\xe9\x08\x7c\x46\xec\xaf\x61\xe3\xa8\x66\x11\x32\x8b\x17\x37\xb1\xec\x3e\xd6\x12\x28\xb0\xcc\x7d\x38\xc3\x0e\xee\xf0\xc8\xbf\x82\x47\xb9\x8c\x2a\x59\x89\xa3\x3d\x9a\x47\x09\xb6\xdf\xc5\xb4\x1d\x06\x9d\xe5\x8e\x7b\x82\x0f\xf6\x13\xad\xf4\x9c\xc3\x27\x94\x56\xe5\x70\x17\xd9\x24" + ) + ] + +vectors_aes256_enc :: [KATCTR] +vectors_aes256_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c" + , "\xcd\x79\x9a\x1e\xf2\xd4\x81\x6b\x8a\xaa\x64\xb5\x73\x0c\x49\x80" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec" + , "\xcd\x79\x9a\x1e\xf2\xd4\x81\x6b\x8a\xaa\x64\xb5\x73\x0c\x49\x80\x26\xa2\x41\x4b\x82\xcf\xf3\xbb\xb5\x04\x99\x2d\xc4\xe8\xfc\xed\xe2\xc9\x2d\x86\xa7\x78\xb5\x1c\xe4\x75\x81\xe2\x89\x99\xae\x3c\xcf\x56\xfe\x04\x71\xc4\x9b\x1f\x09\xeb\xff\x94\xaa\x19\x74\x8a\x4b\xfd\x59\x18\x0c\xbe\xe0\x4e\x9b\x92\xf8\x6b\x10\xb0\xe8\x67\x76\xd6\x76\x80\x38\x09\xe5\xc7\x9b\x8a\x7d\x4e\x48\x43\xa2\x9b\x87\x26\xbb\x4d\x30\xf7\x23\xcb\x62\x53\xc2\x70\x58\x5c\xf0\xa9\xbd\x2d\x48\xf7\x05\x33\xaf\x3c\x2a\xa6\xf6\xc0\x23\x7f\xb7\x0b\x8d\x8a\xe3\xd1\x9d\x31\xbe\x99\x3d\x83\xf3\x24\xa6\x10\x4b\x71" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a" + , "\xcd\x79\x9a\x1e\xf2\xd4\x81\x6b\x8a\xaa\x64\xb5\x73\x0c\x49\x80\x26\xa2\x41\x4b\x82\xcf\xf3\xbb\xb5\x04\x99\x2d\xc4\xe8\xfc\xed\xe2\xc9\x2d\x86\xa7\x78\xb5\x1c\xe4\x75\x81\xe2\x89\x99\xae\x3c\xcf\x56\xfe\x04\x71\xc4\x9b\x1f\x09\xeb\xff\x94\xaa\x19\x74\x8a\x4b\xfd\x59\x18\x0c\xbe\xe0\x4e\x9b\x92\xf8\x6b\x10\xb0\xe8\x67\x76\xd6\x76\x80\x38\x09\xe5\xc7\x9b\x8a\x7d\x4e\x48\x43\xa2\x9b\x87\x26\xbb\x4d\x30\xf7\x23\xcb\x62\x53\xc2\x70\x58\x5c\xf0\xa9\xbd\x2d\x48\xf7\x05\x33\xaf\x3c\x2a\xa6\xf6\xc0\x23\x7f\xb7\x0b\x8d\x8a" + ) + ] diff --git a/tests/BlockCipher/AES/ECB.hs b/tests/BlockCipher/AES/ECB.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/ECB.hs @@ -0,0 +1,720 @@ +module BlockCipher.AES.ECB where + +import qualified Data.ByteString as B + +vectors_aes128_enc = + [ + ( B.pack + [ 0x10 + , 0xa5 + , 0x88 + , 0x69 + , 0xd7 + , 0x4b + , 0xe5 + , 0xa3 + , 0x74 + , 0xcf + , 0x86 + , 0x7c + , 0xfb + , 0x47 + , 0x38 + , 0x59 + ] + , B.replicate 16 0 + , B.pack + [ 0x6d + , 0x25 + , 0x1e + , 0x69 + , 0x44 + , 0xb0 + , 0x51 + , 0xe0 + , 0x4e + , 0xaa + , 0x6f + , 0xb4 + , 0xdb + , 0xf7 + , 0x84 + , 0x65 + ] + ) + , + ( B.replicate 16 0 + , B.replicate 16 0 + , B.pack + [ 0x66 + , 0xe9 + , 0x4b + , 0xd4 + , 0xef + , 0x8a + , 0x2c + , 0x3b + , 0x88 + , 0x4c + , 0xfa + , 0x59 + , 0xca + , 0x34 + , 0x2b + , 0x2e + ] + ) + , + ( B.replicate 16 0 + , B.replicate 16 1 + , B.pack + [ 0xe1 + , 0x4d + , 0x5d + , 0x0e + , 0xe2 + , 0x77 + , 0x15 + , 0xdf + , 0x08 + , 0xb4 + , 0x15 + , 0x2b + , 0xa2 + , 0x3d + , 0xa8 + , 0xe0 + ] + ) + , + ( B.replicate 16 1 + , B.replicate 16 2 + , B.pack + [ 0x17 + , 0xd6 + , 0x14 + , 0xf3 + , 0x79 + , 0xa9 + , 0x35 + , 0x90 + , 0x77 + , 0xe9 + , 0x55 + , 0x77 + , 0xfd + , 0x31 + , 0xc2 + , 0x0a + ] + ) + , + ( B.replicate 16 2 + , B.replicate 16 1 + , B.pack + [ 0x8f + , 0x42 + , 0xc2 + , 0x4b + , 0xee + , 0x6e + , 0x63 + , 0x47 + , 0x2b + , 0x16 + , 0x5a + , 0xa9 + , 0x41 + , 0x31 + , 0x2f + , 0x7c + ] + ) + , + ( B.replicate 16 3 + , B.replicate 16 2 + , B.pack + [ 0x90 + , 0x98 + , 0x85 + , 0xe4 + , 0x77 + , 0xbc + , 0x20 + , 0xf5 + , 0x8a + , 0x66 + , 0x97 + , 0x1d + , 0xa0 + , 0xbc + , 0x75 + , 0xe3 + ] + ) + ] + +vectors_aes192_enc = + [ + ( B.replicate 24 0 + , B.replicate 16 0 + , B.pack + [ 0xaa + , 0xe0 + , 0x69 + , 0x92 + , 0xac + , 0xbf + , 0x52 + , 0xa3 + , 0xe8 + , 0xf4 + , 0xa9 + , 0x6e + , 0xc9 + , 0x30 + , 0x0b + , 0xd7 + ] + ) + , + ( B.replicate 24 0 + , B.replicate 16 1 + , B.pack + [ 0xcf + , 0x1e + , 0xce + , 0x3c + , 0x44 + , 0xb0 + , 0x78 + , 0xfb + , 0x27 + , 0xcb + , 0x0a + , 0x3e + , 0x07 + , 0x1b + , 0x08 + , 0x20 + ] + ) + , + ( B.replicate 24 1 + , B.replicate 16 2 + , B.pack + [ 0xeb + , 0x8c + , 0x17 + , 0x30 + , 0x90 + , 0xc7 + , 0x5b + , 0x77 + , 0xd6 + , 0x72 + , 0xb4 + , 0x57 + , 0xa7 + , 0x78 + , 0xd9 + , 0xd0 + ] + ) + , + ( B.replicate 24 2 + , B.replicate 16 1 + , B.pack + [ 0xf2 + , 0xf0 + , 0xae + , 0xd8 + , 0xcd + , 0xc9 + , 0x21 + , 0xca + , 0x4b + , 0x55 + , 0x84 + , 0x5d + , 0xa4 + , 0x15 + , 0x21 + , 0xc2 + ] + ) + , + ( B.replicate 24 3 + , B.replicate 16 2 + , B.pack + [ 0xca + , 0xcc + , 0x30 + , 0x79 + , 0xe4 + , 0xb7 + , 0x95 + , 0x27 + , 0x63 + , 0xd2 + , 0x55 + , 0xd6 + , 0x34 + , 0x10 + , 0x46 + , 0x14 + ] + ) + ] + +vectors_aes256_enc = + [ + ( B.replicate 32 0 + , B.replicate 16 0 + , B.pack + [ 0xdc + , 0x95 + , 0xc0 + , 0x78 + , 0xa2 + , 0x40 + , 0x89 + , 0x89 + , 0xad + , 0x48 + , 0xa2 + , 0x14 + , 0x92 + , 0x84 + , 0x20 + , 0x87 + ] + ) + , + ( B.replicate 32 0 + , B.replicate 16 1 + , B.pack + [ 0x7b + , 0xc3 + , 0x02 + , 0x6c + , 0xd7 + , 0x37 + , 0x10 + , 0x3e + , 0x62 + , 0x90 + , 0x2b + , 0xcd + , 0x18 + , 0xfb + , 0x01 + , 0x63 + ] + ) + , + ( B.replicate 32 1 + , B.replicate 16 2 + , B.pack + [ 0x62 + , 0xae + , 0x12 + , 0xf3 + , 0x24 + , 0xbf + , 0xea + , 0x08 + , 0xd5 + , 0xf6 + , 0x75 + , 0xb5 + , 0x13 + , 0x02 + , 0x6b + , 0xbf + ] + ) + , + ( B.replicate 32 2 + , B.replicate 16 1 + , B.pack + [ 0x00 + , 0xf9 + , 0xc7 + , 0x44 + , 0x4b + , 0xb0 + , 0xcc + , 0x80 + , 0x6c + , 0x7c + , 0x39 + , 0xee + , 0x22 + , 0x11 + , 0xf1 + , 0x46 + ] + ) + , + ( B.replicate 32 3 + , B.replicate 16 2 + , B.pack + [ 0xb4 + , 0x05 + , 0x87 + , 0x3e + , 0xa0 + , 0x76 + , 0x1b + , 0x9c + , 0xa9 + , 0x9f + , 0x70 + , 0xb0 + , 0x16 + , 0x16 + , 0xce + , 0xb1 + ] + ) + ] + +vectors_aes128_dec = + [ + ( B.replicate 16 0 + , B.replicate 16 0 + , B.pack + [ 0x14 + , 0x0f + , 0x0f + , 0x10 + , 0x11 + , 0xb5 + , 0x22 + , 0x3d + , 0x79 + , 0x58 + , 0x77 + , 0x17 + , 0xff + , 0xd9 + , 0xec + , 0x3a + ] + ) + , + ( B.replicate 16 0 + , B.replicate 16 1 + , B.pack + [ 0x15 + , 0x6d + , 0x0f + , 0x85 + , 0x75 + , 0xd5 + , 0x33 + , 0x07 + , 0x52 + , 0xf8 + , 0x4a + , 0xf2 + , 0x72 + , 0xff + , 0x30 + , 0x50 + ] + ) + , + ( B.replicate 16 1 + , B.replicate 16 2 + , B.pack + [ 0x34 + , 0x37 + , 0xd6 + , 0xe2 + , 0x31 + , 0xd7 + , 0x02 + , 0x41 + , 0x9b + , 0x51 + , 0xb4 + , 0x94 + , 0x72 + , 0x71 + , 0xb6 + , 0x11 + ] + ) + , + ( B.replicate 16 2 + , B.replicate 16 1 + , B.pack + [ 0xe3 + , 0xcd + , 0xe2 + , 0x37 + , 0xc8 + , 0xf2 + , 0xd9 + , 0x7b + , 0x8d + , 0x79 + , 0xf9 + , 0x17 + , 0x1d + , 0x4b + , 0xda + , 0xc1 + ] + ) + , + ( B.replicate 16 3 + , B.replicate 16 2 + , B.pack + [ 0x5b + , 0x94 + , 0xaa + , 0xed + , 0xd7 + , 0x83 + , 0x99 + , 0x8c + , 0xd5 + , 0x15 + , 0x35 + , 0x35 + , 0x18 + , 0xcc + , 0x45 + , 0xe2 + ] + ) + ] + +vectors_aes192_dec = + [ + ( B.replicate 24 0 + , B.replicate 16 0 + , B.pack + [ 0x13 + , 0x46 + , 0x0e + , 0x87 + , 0xa8 + , 0xfc + , 0x02 + , 0x3e + , 0xf2 + , 0x50 + , 0x1a + , 0xfe + , 0x7f + , 0xf5 + , 0x1c + , 0x51 + ] + ) + , + ( B.replicate 24 0 + , B.replicate 16 1 + , B.pack + [ 0x92 + , 0x17 + , 0x07 + , 0xc3 + , 0x3d + , 0x1c + , 0xc5 + , 0x96 + , 0x7d + , 0xa5 + , 0x1d + , 0xbb + , 0xb0 + , 0x66 + , 0xb2 + , 0x6c + ] + ) + , + ( B.replicate 24 1 + , B.replicate 16 2 + , B.pack + [ 0xee + , 0x92 + , 0x97 + , 0xc6 + , 0xba + , 0xe8 + , 0x26 + , 0x4d + , 0xff + , 0x08 + , 0x0e + , 0xbb + , 0x1e + , 0x74 + , 0x11 + , 0xc1 + ] + ) + , + ( B.replicate 24 2 + , B.replicate 16 1 + , B.pack + [ 0x49 + , 0x67 + , 0xdf + , 0x70 + , 0xd2 + , 0x9e + , 0x9a + , 0x7f + , 0x5d + , 0x7c + , 0xb9 + , 0xc1 + , 0x20 + , 0xc3 + , 0x8a + , 0x71 + ] + ) + , + ( B.replicate 24 3 + , B.replicate 16 2 + , B.pack + [ 0x74 + , 0x38 + , 0x62 + , 0x42 + , 0x6b + , 0x56 + , 0x7f + , 0xd5 + , 0xf0 + , 0x1d + , 0x1b + , 0x59 + , 0x56 + , 0x01 + , 0x26 + , 0x29 + ] + ) + ] + +vectors_aes256_dec = + [ + ( B.replicate 32 0 + , B.replicate 16 0 + , B.pack + [ 0x67 + , 0x67 + , 0x1c + , 0xe1 + , 0xfa + , 0x91 + , 0xdd + , 0xeb + , 0x0f + , 0x8f + , 0xbb + , 0xb3 + , 0x66 + , 0xb5 + , 0x31 + , 0xb4 + ] + ) + , + ( B.replicate 32 0 + , B.replicate 16 1 + , B.pack + [ 0xcc + , 0x09 + , 0x21 + , 0xa3 + , 0xc5 + , 0xca + , 0x17 + , 0xf7 + , 0x48 + , 0xb7 + , 0xc2 + , 0x7b + , 0x73 + , 0xba + , 0x87 + , 0xa2 + ] + ) + , + ( B.replicate 32 1 + , B.replicate 16 2 + , B.pack + [ 0xc0 + , 0x4b + , 0x27 + , 0x90 + , 0x1a + , 0x50 + , 0xcf + , 0xfa + , 0xf1 + , 0xbb + , 0x88 + , 0x9f + , 0xc0 + , 0x92 + , 0x5e + , 0x14 + ] + ) + , + ( B.replicate 32 2 + , B.replicate 16 1 + , B.pack + [ 0x24 + , 0x61 + , 0x53 + , 0x5d + , 0x16 + , 0x1c + , 0x15 + , 0x39 + , 0x88 + , 0x32 + , 0x77 + , 0x29 + , 0xc5 + , 0x8c + , 0xc0 + , 0x3a + ] + ) + , + ( B.replicate 32 3 + , B.replicate 16 2 + , B.pack + [ 0x30 + , 0xc9 + , 0x1c + , 0xce + , 0xfe + , 0x89 + , 0x30 + , 0xcf + , 0xff + , 0x31 + , 0xdb + , 0xcc + , 0xfc + , 0x11 + , 0xc5 + , 0x23 + ] + ) + ] diff --git a/tests/BlockCipher/AES/GCM.hs b/tests/BlockCipher/AES/GCM.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/GCM.hs @@ -0,0 +1,173 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.AES.GCM where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () + +-- (key, iv, aad, input, out, taglen, tag) +type KATGCM = + ( B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + , Int + , B.ByteString + ) + +vectors_aes128_enc :: [KATGCM] +vectors_aes128_enc = + [ -- vectors 0 + + ( {-key = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-aad = -} "" + , {-input = -} "" + , {-out = -} "" + , {-taglen = -} 16 + , {-tag = -} "\x58\xe2\xfc\xce\xfa\x7e\x30\x61\x36\x7f\x1d\x57\xa4\xe7\x45\x5a" + ) + , -- vectors 1 + + ( {-key = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-aad = -} "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , {-input = -} "\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a" + , {-out = -} "\x09\x82\xd0\xc4\x6a\xbc\xa9\x98\xf9\x22\xc8\xb3\x7b\xb8\xf4\x72\xfd\x9f\xa0\xa1\x43\x41\x53\x29\xfd\xf7\x83\xf5\x9e\x81\xcb\xea" + , {-taglen = -} 16 + , {-tag = -} "\x28\x50\x64\x2f\xa8\x8b\xab\x21\x2a\x67\x1a\x97\x48\x69\xa5\x6c" + ) + , -- vectors 2 + + ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-aad = -} "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , {-input = -} "\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a" + , {-out = -} "\x1c\xa3\xb5\x41\x39\x6f\x19\x7a\x91\x2d\x27\x15\x70\xd1\xf5\x76\xde\xf1\xbe\x84\x42\x2a\xbb\xbe\x0b\x2d\x91\x21\x82\xbf\x7f\x17" + , {-taglen = -} 16 + , {-tag = -} "\x15\x2a\x05\xbb\x7e\x13\x5d\xbe\x93\x7f\xa0\x54\x7a\x8e\x74\xb6" + ) + , -- vectors 3 + + ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-aad = -} "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" + , {-input = -} "\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a" + , {-out = -} "\xda\x35\xf6\x0a\x65\xc2\xa4\x6c\xb6\x6e\xb6\xf8\x1f\x0b\x9c\x74\x53\x4c\x97\x70\x36\xf7\xdf\x05\x6d\x00\xfe\xbf\xb4\xcb\xf5\x27" + , {-taglen = -} 16 + , {-tag = -} "\xb7\x76\x7c\x3b\x9e\xf1\xe2\xcb\xc9\x11\xf1\x9a\xdc\xfa\x35\x0d" + ) + , + ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-aad = -} "\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76" + , {-input = -} "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" + , {-out = -} "\xe4\x42\xf8\xc4\xc6\x67\x84\x86\x4a\x5a\x6e\xc7\xe0\xca\x68\xac\x16\xbc\x5b\xbf\xf7\xd5\xf3\xfa\xf3\xb2\xcb\xb0\xa2\x14\xa1\x81" + , {-taglen = -} 16 + , {-tag = -} "\x5f\x63\xb8\xeb\x1d\x6f\xa8\x7a\xeb\x39\xa5\xf6\xd7\xed\xc3\x13" + ) + , + ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-aad = -} "\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76" + , {-input = -} "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" + , {-out = -} "\xe4\x42\xf8\xc4\xc6\x67\x84\x86\x4a\x5a\x6e\xc7\xe0\xca\x68\xac\x16\xbc\x5b\xbf\xf7\xd5\xf3\xfa\xf3\xb2\xcb\xb0\xa2\x14\xa1" + , {-taglen = -} 16 + , {-tag = -} "\x94\xd1\x47\xc3\xa2\xca\x93\xe9\x66\x93\x1e\x3b\xb3\xbb\x67\x01" + ) + , -- vector 6 tests 32-bit counter wrapping + + ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , {-iv = -} "\xe8\x38\x84\x1d\x75\xae\x33\xb5\x4b\x51\x57\x89\xc9\x5f\xbe\x65" + , {-aad = -} "\x54\x68\x65\x20\x66\x69\x76\x65\x20\x62\x6f\x78\x69\x6e\x67\x20\x77\x69\x7a\x61\x72\x64\x73\x20\x6a\x75\x6d\x70\x20\x71\x75\x69\x63\x6b\x6c\x79\x2e" + , {-input = -} "\x54\x68\x65\x20\x71\x75\x69\x63\x6b\x20\x62\x72\x6f\x77\x6e\x20\x66\x6f\x78\x20\x6a\x75\x6d\x70\x73\x20\x6f\x76\x65\x72\x20\x74\x68\x65\x20\x6c\x61\x7a\x79\x20\x64\x6f\x67" + , {-out = -} "\x82\x31\x9e\x5a\x6a\x7f\x43\xd0\x42\x8c\xf1\x01\xcf\x0c\x75\xf1\x5d\xda\x4f\xa1\x28\x95\xcd\xd7\x7b\xd5\x42\x68\x2f\xcd\x10\x1b\x0c\x75\x05\x54\xf4\x2f\x2b\xf6\x69\x96\x29" + , {-taglen = -} 16 + , {-tag = -} "\x9a\xfa\xf4\xea\xae\x2e\x6f\x40\x00\xf4\x89\x77\xd0\x1e\xd5\x14" + ) + ] + +-- From OpenSSL 3.5, for the AES-192 paths: the suite had no GCM vector +-- at that key size at all. The lengths run over the empty message, a +-- single block, a partial one, exactly the eight blocks the hardware +-- loops take at a time, and nine and a bit. +vectors_aes192_enc :: [KATGCM] +vectors_aes192_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "" + , "" + , "" + , 16 + , "\xc3\x12\x1f\x10\x89\xad\x0b\x07\x2c\xcc\x35\x15\xfd\x13\x76\x47" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c" + , "\xb0\x24\x29\x8e\xb3\x24\x04\x73\x09\x11\x3e\x1b\xe2\x73\xf1\x0c" + , 16 + , "\x46\xd6\x22\x38\x24\x81\x09\xb8\x1f\x72\x32\x99\x2e\x43\x18\xcf" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0" + , "\xb0\x24\x29\x8e\xb3\x24\x04\x73\x09\x11\x3e\x1b\xe2\x73\xf1\x0c\x75\x0b\xb2\xac\x3a\xe3\x6e\x13\x89\x3b\xc4\xc3\xcf\x7d\x54\x5b\x08\xc0\x2d\xd4\xe4\x6f\x6c\x96\xe5\x87\x8f\xa8\x1d\x94\x77\xd9\x6f\x02\xe9\xfa\x93\x9c\x90\x71\x62\xbe\x1b\xb1" + , 16 + , "\xcd\xf8\xad\xc9\xbe\x10\xbd\x38\xff\x0b\xb5\xad\x2a\x97\xd0\x23" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c" + , "\xb0\x24\x29\x8e\xb3\x24\x04\x73\x09\x11\x3e\x1b\xe2\x73\xf1\x0c\x75\x0b\xb2\xac\x3a\xe3\x6e\x13\x89\x3b\xc4\xc3\xcf\x7d\x54\x5b\x08\xc0\x2d\xd4\xe4\x6f\x6c\x96\xe5\x87\x8f\xa8\x1d\x94\x77\xd9\x6f\x02\xe9\xfa\x93\x9c\x90\x71\x62\xbe\x1b\xb1\x99\x07\xc2\xe0\x6f\x29\x8d\xca\x4e\xc1\xb8\x16\x48\xd6\x52\xd7\x58\xb9\xcd\xd6\xee\x16\xbb\x45\x09\x14\x3b\xe0\x5d\x75\xb0\x0a\x33\x78\xa9\x3a\x51\xaf\x1a\x7d\x26\x06\x7d\x0f\x34\x85\x41\x73\xbe\xc6\x0f\x84\xd4\x17\xcb\xab\x16\xbf\x86\x77\xc1\x05\xef\x94\xc6\xc7\x23\x97" + , 16 + , "\x9f\x90\x5a\x7a\x0b\x5f\x37\x20\xf3\xa3\x58\xff\x9b\x5b\xe9\x92" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec\xf3\xfa" + , "\xb0\x24\x29\x8e\xb3\x24\x04\x73\x09\x11\x3e\x1b\xe2\x73\xf1\x0c\x75\x0b\xb2\xac\x3a\xe3\x6e\x13\x89\x3b\xc4\xc3\xcf\x7d\x54\x5b\x08\xc0\x2d\xd4\xe4\x6f\x6c\x96\xe5\x87\x8f\xa8\x1d\x94\x77\xd9\x6f\x02\xe9\xfa\x93\x9c\x90\x71\x62\xbe\x1b\xb1\x99\x07\xc2\xe0\x6f\x29\x8d\xca\x4e\xc1\xb8\x16\x48\xd6\x52\xd7\x58\xb9\xcd\xd6\xee\x16\xbb\x45\x09\x14\x3b\xe0\x5d\x75\xb0\x0a\x33\x78\xa9\x3a\x51\xaf\x1a\x7d\x26\x06\x7d\x0f\x34\x85\x41\x73\xbe\xc6\x0f\x84\xd4\x17\xcb\xab\x16\xbf\x86\x77\xc1\x05\xef\x94\xc6\xc7\x23\x97\x3f\xcf\xce\x87\x9d\x34\x08\xda\x6b\x47\x5d\xff\xa3\xb6\x19\xc7\xdb\x99" + , 16 + , "\xf9\xfd\x33\x7f\xf0\x8f\x5f\xc8\xc0\xe9\x9d\x6d\xe6\xbb\xe2\xec" + ) + ] + +vectors_aes256_enc :: [KATGCM] +vectors_aes256_enc = + [ + ( "\xb5\x2c\x50\x5a\x37\xd7\x8e\xda\x5d\xd3\x4f\x20\xc2\x25\x40\xea\x1b\x58\x96\x3c\xf8\xe5\xbf\x8f\xfa\x85\xf9\xf2\x49\x25\x05\xb4" + , "\x51\x6c\x33\x92\x9d\xf5\xa3\x28\x4f\xf4\x63\xd7" + , "" + , "" + , "" + , 16 + , "\xbd\xc1\xac\x88\x4d\x33\x24\x57\xa1\xd2\x66\x4f\x16\x8c\x76\xf0" + ) + , + ( "\x78\xdc\x4e\x0a\xaf\x52\xd9\x35\xc3\xc0\x1e\xea\x57\x42\x8f\x00\xca\x1f\xd4\x75\xf5\xda\x86\xa4\x9c\x8d\xd7\x3d\x68\xc8\xe2\x23" + , "\xd7\x9c\xf2\x2d\x50\x4c\xc7\x93\xc3\xfb\x6c\x8a" + , "\xb9\x6b\xaa\x8c\x1c\x75\xa6\x71\xbf\xb2\xd0\x8d\x06\xbe\x5f\x36" + , "" + , "" + , 16 + , "\x3e\x5d\x48\x6a\xa2\xe3\x0b\x22\xe0\x40\xb8\x57\x23\xa0\x6e\x76" + ) + , + ( "\xc3\xf1\x05\x86\xf2\x46\xaa\xca\xdc\xce\x37\x01\x44\x17\x70\xc0\x3c\xfe\xc9\x40\xaf\xe1\x90\x8c\x4c\x53\x7d\xf4\xe0\x1c\x50\xa0" + , "\x4f\x52\xfa\xa1\xfa\x67\xa0\xe5\xf4\x19\x64\x52" + , "\x46\xf9\xa2\x2b\x4e\x52\xe1\x52\x65\x13\xa9\x52\xdb\xee\x3b\x91\xf6\x95\x95\x50\x1e\x01\x77\xd5\x0f\xf3\x64\x63\x85\x88\xc0\x8d\x92\xfa\xb8\xc5\x8a\x96\x9b\xdc\xc8\x4c\x46\x8d\x84\x98\xc4\xf0\x63\x92\xb9\x9e\xd5\xe0\xc4\x84\x50\x7f\xc4\x8d\xc1\x8d\x87\xc4\x0e\x2e\xd8\x48\xb4\x31\x50\xbe\x9d\x36\xf1\x4c\xf2\xce\xf1\x31\x0b\xa4\xa7\x45\xad\xcc\x7b\xdc\x41\xf6" + , "\x79\xd9\x7e\xa3\xa2\xed\xd6\x50\x45\x82\x1e\xa7\x45\xa4\x47\x42" + , "\x56\x0c\xf7\x16\xe5\x61\x90\xe9\x39\x7c\x2f\x10\x36\x29\xeb\x1f" + , 16 + , "\xff\x7c\x91\x24\x87\x96\x44\xe8\x05\x55\x68\x7d\x27\x3c\x55\xd8" + ) + ] diff --git a/tests/BlockCipher/AES/GCMLong.hs b/tests/BlockCipher/AES/GCMLong.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/GCMLong.hs @@ -0,0 +1,1138 @@ +{-# LANGUAGE OverloadedStrings #-} + +-- | AES-GCM over messages long enough to reach the bulk loops -- the +-- eight-block group of the AES-NI path, and the assembly beyond it, which +-- take 96 and 288 bytes respectively before they do anything. The lengths +-- sit either side of those boundaries and of the block size. +-- +-- Writing the messages and the ciphertexts out would run to hundreds of +-- kilobytes of literals, so both are given by a rule instead: the input is +-- a fixed pattern of the stated length and what is recorded is the tag and +-- the SHA-256 of the ciphertext. The expected values come from OpenSSL +-- 3.6.4 through EVP. +module BlockCipher.AES.GCMLong ( + KATGCMLong, + gcmKey, + gcmIV, + gcmAAD, + gcmPlaintext, + vectors, +) where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () +import Data.Word (Word8) + +-- (key length, AAD length, message length, tag, SHA-256 of the ciphertext) +type KATGCMLong = (Int, Int, Int, B.ByteString, B.ByteString) + +pattern_ :: Int -> Int -> Int -> B.ByteString +pattern_ a b n = B.pack [fromIntegral (i * a + b) :: Word8 | i <- [0 .. n - 1]] + +gcmKey :: Int -> B.ByteString +gcmKey n = B.pack [fromIntegral (i + 0x40) :: Word8 | i <- [0 .. n - 1]] + +gcmIV :: B.ByteString +gcmIV = B.pack [fromIntegral (0xf0 - i) :: Word8 | i <- [0 .. 11 :: Int]] + +gcmAAD :: Int -> B.ByteString +gcmAAD = pattern_ 3 1 + +gcmPlaintext :: Int -> B.ByteString +gcmPlaintext = pattern_ 7 3 + +vectors :: [KATGCMLong] +vectors = + [ + ( 16 + , 0 + , 0 + , "\xb7\x4b\x23\x40\x36\x14\xec\x89\x6e\x0c\xbe\x06\xb8\x72\x79\x80" + , "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24\x27\xae\x41\xe4\x64\x9b\x93\x4c\xa4\x95\x99\x1b\x78\x52\xb8\x55" + ) + , + ( 16 + , 0 + , 1 + , "\x21\x9e\x2e\x1d\x60\x79\xae\xae\x73\x79\xef\x6a\x59\x1c\xaf\x3c" + , "\x8a\x8d\xe8\x23\xd5\xed\x3e\x12\x74\x6a\x62\xef\x16\x9b\xcf\x37\x2b\xe0\xca\x44\xf0\xa1\x23\x6a\xbc\x35\xdf\x05\xd9\x69\x28\xe1" + ) + , + ( 16 + , 0 + , 15 + , "\x75\x13\x81\x33\xc3\x3a\xff\x75\x60\x5a\x8c\xc6\x38\x27\x18\x40" + , "\xa3\xc9\xf6\x0d\x54\x49\xb3\x48\x83\xba\x20\xc4\x89\x74\x46\xaf\x1e\x27\x86\xff\x18\x40\x31\xa1\xa9\x6a\xd7\x0a\x58\x1a\x38\xfb" + ) + , + ( 16 + , 0 + , 16 + , "\xf5\xe5\x3d\xd3\x2e\x2f\x98\x47\xb5\xed\x40\x0e\x75\x18\xae\x71" + , "\xeb\xe5\xf7\xf7\x43\x47\x3e\x4a\x3b\x82\xbb\xdd\xd4\xee\x4a\xaf\xbe\x73\xbb\x2c\xa4\xc8\xf0\xa5\xc7\xb6\xd5\x97\x4b\xaf\xcd\x0e" + ) + , + ( 16 + , 0 + , 17 + , "\xcc\x69\x37\x31\xc5\x15\xd3\x13\xf3\xf1\x71\x4e\x19\x74\xf9\x32" + , "\xe7\xd1\xc1\xa8\xb6\xc6\xaf\x32\x2f\xe3\xeb\xae\x6f\xb7\x27\xde\x3d\xde\xe0\x28\x7f\x8f\x43\xe5\xc0\x2a\xb4\x95\xae\xd6\xd4\x74" + ) + , + ( 16 + , 0 + , 95 + , "\x6e\xac\x93\xfa\x12\x70\x0f\xda\x86\x8f\x26\xa7\xe7\xe6\xaf\x2f" + , "\x05\xbf\xb1\x40\x93\xea\x41\xe2\xe3\xac\x14\xf0\x41\x6d\xc0\xc8\xf6\x29\x25\xbd\xdc\x16\x57\x63\x28\x11\x1a\xc7\x71\x1f\x04\xa0" + ) + , + ( 16 + , 0 + , 96 + , "\xe6\xd8\xb3\xd9\x12\xea\x15\x92\xc5\x5c\x04\xf3\x39\x01\x3a\xcd" + , "\x61\x6d\x16\x55\x7e\x6c\x2a\x43\xe1\x61\x92\x7c\x64\xf1\x0c\x70\x5c\xc0\x25\xbe\x47\xa3\x28\xa9\x9a\xe3\xab\x86\x83\x96\x5b\x0c" + ) + , + ( 16 + , 0 + , 97 + , "\xda\x50\xec\xfa\xed\xe5\xe4\xca\x51\x9d\x42\xe3\x0b\xf7\x6b\x96" + , "\x54\x91\xf4\xcc\x30\x36\xa1\x7e\xbe\xeb\x16\xa3\x59\x74\xe0\xac\xa7\x51\x53\x09\x1b\x12\xcc\xd2\x01\x70\x9f\xc1\xa1\x3b\x03\x31" + ) + , + ( 16 + , 0 + , 112 + , "\x55\x5f\x7c\x75\xb8\xa1\x9b\xd6\x85\xc8\x11\xd3\xca\x97\x3e\x9e" + , "\x34\x3c\x8d\xf0\x33\x39\xfb\x89\x5e\xa2\x49\xd5\xa5\x45\x24\x8c\x33\x11\x52\xbb\x76\x04\xa9\xa2\x01\x10\xb5\x44\x49\x51\xae\x80" + ) + , + ( 16 + , 0 + , 159 + , "\xcb\xb3\x51\xbe\xfa\x99\xf4\x53\x22\xaa\xf6\x80\xdf\xa1\xb2\xc0" + , "\x9a\x12\xd7\x0e\xe8\xdd\x96\xac\xbc\x22\x4d\x71\x0b\xd3\x53\x49\x07\xfd\x54\x95\xf1\xb2\xf8\xf8\x47\x6e\x13\x91\x2a\xe0\xb5\x45" + ) + , + ( 16 + , 0 + , 160 + , "\x47\x4c\xd6\x6c\x93\xcb\xb3\x1c\x4a\x50\x71\xdd\xff\xbc\xfc\x41" + , "\x1b\x3e\xf6\x7f\xfa\xe8\x40\xbe\xc9\x2d\xc6\xa0\x13\x80\x47\xd5\x1c\x39\xf1\xe7\x71\x33\x57\x9b\xea\xec\x35\xfa\x50\x29\x51\xa1" + ) + , + ( 16 + , 0 + , 191 + , "\xc8\xfc\x72\x46\xd6\xe6\x73\x3a\xe9\x93\x2e\x11\xb9\x32\x43\x53" + , "\x6e\x35\x69\x18\x73\x5e\x0e\x3a\xa0\x0d\x28\x0b\x07\x13\xde\xc3\x68\x97\xbd\xa5\x32\xd8\xb0\x60\x27\x61\x58\x8e\x71\xb0\xcb\x34" + ) + , + ( 16 + , 0 + , 192 + , "\x2c\x74\xd1\x29\x67\xa7\x94\x18\x7d\x49\x36\xcf\x2d\x20\xcd\xdd" + , "\x6b\x55\x0d\x71\x75\xba\xd4\x2c\x0f\x3d\x44\x24\xad\x19\xd5\x66\xbc\xa6\x5d\xcc\xc8\xc7\x81\x4f\x6e\x33\x2d\xb8\x67\x67\x02\x87" + ) + , + ( 16 + , 0 + , 287 + , "\x1b\xc9\xe8\x6b\x95\x95\x1b\x34\x4a\x16\x0c\x01\x4f\x46\x7a\xbe" + , "\x1f\xa9\x9b\x5e\x49\xfa\xe4\x99\x3f\x90\x95\x04\x7f\x92\xf6\xba\xa3\x8e\x79\xae\x7b\xdd\x2f\x3d\xba\x88\xe6\x18\xa9\xed\xc6\x78" + ) + , + ( 16 + , 0 + , 288 + , "\x33\x1b\x69\x63\x64\x24\x46\x87\x8c\x8a\x7c\x3c\xa8\x29\x78\x77" + , "\x35\x84\x4c\x16\xd6\x16\x31\x04\xd1\x43\xc7\xcd\x10\x90\x41\x03\x32\x48\xe9\x5c\xfd\xc0\x4c\xef\x42\x52\xc0\xe0\x45\xec\x8e\x0f" + ) + , + ( 16 + , 0 + , 289 + , "\x35\x45\xd6\x40\xaf\x6c\x5d\x5a\xc2\x73\x93\x8e\x4f\x9e\x6f\x08" + , "\xce\x08\x82\x6c\x10\x30\xd8\xda\xf8\x01\xd7\x47\x85\xd3\x4b\xbe\x25\xb5\x43\x52\x50\x90\xa9\x29\x8f\x48\xc4\x1c\x16\x90\x0c\xee" + ) + , + ( 16 + , 0 + , 304 + , "\x87\xa4\x6b\x61\xf7\xd8\x19\x5a\x82\x6c\xba\x18\x21\xb7\x40\x43" + , "\x9f\x74\x66\xcf\xdd\x78\xaf\x93\x36\x0e\xe2\xf2\xc5\x74\xed\xdf\xb3\xb6\x40\xcf\x3a\xa5\xd0\x61\x12\x3e\x09\x07\xae\x4a\x0f\xef" + ) + , + ( 16 + , 0 + , 383 + , "\xfc\x60\x06\xbc\xba\x02\x4c\xaa\x9e\xbf\xe6\xc2\xe8\x1a\xc3\xd3" + , "\x23\xdb\xda\x5f\xd3\xf1\x66\x1b\xc9\xb6\xb4\x88\x21\x3e\x8c\xc5\xc9\x7f\xd7\x6e\x8a\x26\x4b\x09\xc8\x43\x64\x1a\xb0\x41\xb2\xb5" + ) + , + ( 16 + , 0 + , 384 + , "\x63\xf4\x5e\xd3\xb8\xd1\x23\x58\x94\x2e\x33\xb9\xc9\xa7\xf3\xab" + , "\x67\xec\x7b\x0e\x61\x15\xbb\xc5\xfa\x00\x43\xad\x07\x89\x6e\xff\x77\xe9\xe8\x7e\x6f\x76\xb1\xd8\xb5\x08\x36\xdc\xde\x19\xb9\xb8" + ) + , + ( 16 + , 0 + , 385 + , "\xaf\xf8\x9c\xe4\x5e\x33\x16\x0d\x83\xc1\x0d\x80\xcf\x5d\x15\x3e" + , "\x8e\xf0\x14\x46\x53\x7d\x9c\x16\xb0\xd8\x7e\xfa\xff\xf1\xa9\x9a\x83\x76\xe2\x1b\xd3\x9b\xc8\x61\xb7\xf8\x13\xa4\xed\x06\xfd\xce" + ) + , + ( 16 + , 0 + , 480 + , "\xfc\x11\x26\x7d\xb7\xfa\xc2\xe4\x40\x27\xad\x9b\x21\xdf\xf0\x9e" + , "\x73\x56\xb8\xf7\x72\x11\x76\xc4\x0b\x51\xd2\xa6\x01\xe9\x2a\x2e\x09\x7e\x42\xfa\x3e\x53\xd1\x79\x46\xb3\x3a\x20\x17\x1a\x90\xa3" + ) + , + ( 16 + , 0 + , 576 + , "\xbb\xca\x37\xa6\xa7\x6d\x38\x2d\x27\xc5\x12\x35\xf3\xe5\x76\x9b" + , "\x74\x93\xe2\xe2\x80\x8d\xb2\xc9\xc8\xe1\x99\x54\x63\xc5\x3f\xce\xea\x9d\x54\x97\x94\x08\x99\xce\xdf\x8c\xb9\x2f\x9d\xf7\xe1\xb8" + ) + , + ( 16 + , 0 + , 1023 + , "\x97\x07\xe2\x3a\x86\xac\xf2\x25\xbf\x9b\x04\x87\x1f\xe2\x31\xfb" + , "\xe4\xf8\x93\x21\x93\x1d\x07\x3c\x3a\x7b\xf1\xf5\x81\x7f\x86\xbd\xf5\x4f\xdb\xed\xa5\x28\x67\xba\x32\x91\xe0\xe5\x06\xe6\x35\x6d" + ) + , + ( 16 + , 0 + , 1024 + , "\x83\x5e\xdf\x28\x20\x2e\x9f\x21\xee\xd1\xdc\x5c\x99\x3d\x51\xa5" + , "\x47\x29\x52\xfa\xa4\xe7\xeb\xea\x8c\x57\x95\xa6\x8b\x1b\xf0\xdf\x04\x8e\xa6\xd7\x12\x0c\xb1\x7d\x25\x9f\x00\xfe\xab\x67\x4f\x22" + ) + , + ( 16 + , 0 + , 1025 + , "\x2b\xad\xe2\x48\xfe\x36\x12\xdb\x39\x9a\x80\xac\x3c\xfa\xd8\x75" + , "\xf3\x7a\xf7\xec\xd0\x99\x32\xe8\xdd\xb7\xc7\x96\x01\x44\xba\xe0\x28\x3c\xe3\x7a\x25\x87\x5e\xd9\xd7\x42\x45\xcb\x04\x01\x92\x5d" + ) + , + ( 16 + , 0 + , 4099 + , "\xb5\xbd\x54\x91\xe7\xd7\x24\xe0\x32\x7b\x99\x89\x1a\x1f\x57\x06" + , "\x91\xae\x42\x66\xa7\x07\x98\x08\xaa\x85\x75\xdc\x4a\xa2\x29\x7b\x89\x44\x67\xbd\x4d\x40\x2d\x15\x62\xeb\x3f\xd6\x66\x9b\x3b\x32" + ) + , + ( 16 + , 20 + , 0 + , "\x28\xf8\xfa\xef\x7c\xf9\x67\x84\x7b\x7d\x04\xa6\x3b\x20\x60\x5c" + , "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24\x27\xae\x41\xe4\x64\x9b\x93\x4c\xa4\x95\x99\x1b\x78\x52\xb8\x55" + ) + , + ( 16 + , 20 + , 1 + , "\x05\xbe\x76\x39\x63\xe0\x05\xe3\xca\xf6\x12\x41\x8b\x22\xb7\xc5" + , "\x8a\x8d\xe8\x23\xd5\xed\x3e\x12\x74\x6a\x62\xef\x16\x9b\xcf\x37\x2b\xe0\xca\x44\xf0\xa1\x23\x6a\xbc\x35\xdf\x05\xd9\x69\x28\xe1" + ) + , + ( 16 + , 20 + , 15 + , "\x51\x33\xd9\x17\xc0\xa3\x54\x38\xd9\xd5\x71\xed\xea\x19\x00\xb9" + , "\xa3\xc9\xf6\x0d\x54\x49\xb3\x48\x83\xba\x20\xc4\x89\x74\x46\xaf\x1e\x27\x86\xff\x18\x40\x31\xa1\xa9\x6a\xd7\x0a\x58\x1a\x38\xfb" + ) + , + ( 16 + , 20 + , 16 + , "\xd1\xc5\x65\xf7\x2d\xb6\x33\x0a\x0c\x62\xbd\x25\xa7\x26\xb6\x88" + , "\xeb\xe5\xf7\xf7\x43\x47\x3e\x4a\x3b\x82\xbb\xdd\xd4\xee\x4a\xaf\xbe\x73\xbb\x2c\xa4\xc8\xf0\xa5\xc7\xb6\xd5\x97\x4b\xaf\xcd\x0e" + ) + , + ( 16 + , 20 + , 17 + , "\x1a\x92\xfe\x2d\xf9\x3b\xdd\xa3\x64\xce\x15\xf6\x96\x89\x41\xe8" + , "\xe7\xd1\xc1\xa8\xb6\xc6\xaf\x32\x2f\xe3\xeb\xae\x6f\xb7\x27\xde\x3d\xde\xe0\x28\x7f\x8f\x43\xe5\xc0\x2a\xb4\x95\xae\xd6\xd4\x74" + ) + , + ( 16 + , 20 + , 95 + , "\x06\xb3\x0c\xca\xb5\xbe\xf0\xba\x89\xeb\x10\xe4\x65\xb8\x88\xb0" + , "\x05\xbf\xb1\x40\x93\xea\x41\xe2\xe3\xac\x14\xf0\x41\x6d\xc0\xc8\xf6\x29\x25\xbd\xdc\x16\x57\x63\x28\x11\x1a\xc7\x71\x1f\x04\xa0" + ) + , + ( 16 + , 20 + , 96 + , "\x8e\xc7\x2c\xe9\xb5\x24\xea\xf2\xca\x38\x32\xb0\xbb\x5f\x1d\x52" + , "\x61\x6d\x16\x55\x7e\x6c\x2a\x43\xe1\x61\x92\x7c\x64\xf1\x0c\x70\x5c\xc0\x25\xbe\x47\xa3\x28\xa9\x9a\xe3\xab\x86\x83\x96\x5b\x0c" + ) + , + ( 16 + , 20 + , 97 + , "\x24\x28\x51\x23\xbc\x1e\x28\xec\x9d\x4f\x98\x97\x85\xb4\x8c\x33" + , "\x54\x91\xf4\xcc\x30\x36\xa1\x7e\xbe\xeb\x16\xa3\x59\x74\xe0\xac\xa7\x51\x53\x09\x1b\x12\xcc\xd2\x01\x70\x9f\xc1\xa1\x3b\x03\x31" + ) + , + ( 16 + , 20 + , 112 + , "\xab\x27\xc1\xac\xe9\x5a\x57\xf0\x49\x1a\xcb\xa7\x44\xd4\xd9\x3b" + , "\x34\x3c\x8d\xf0\x33\x39\xfb\x89\x5e\xa2\x49\xd5\xa5\x45\x24\x8c\x33\x11\x52\xbb\x76\x04\xa9\xa2\x01\x10\xb5\x44\x49\x51\xae\x80" + ) + , + ( 16 + , 20 + , 159 + , "\xe2\x61\xff\x5c\x16\x63\x29\xab\x90\xf7\x88\x56\x6c\x52\xc5\xd4" + , "\x9a\x12\xd7\x0e\xe8\xdd\x96\xac\xbc\x22\x4d\x71\x0b\xd3\x53\x49\x07\xfd\x54\x95\xf1\xb2\xf8\xf8\x47\x6e\x13\x91\x2a\xe0\xb5\x45" + ) + , + ( 16 + , 20 + , 160 + , "\x6e\x9e\x78\x8e\x7f\x31\x6e\xe4\xf8\x0d\x0f\x0b\x4c\x4f\x8b\x55" + , "\x1b\x3e\xf6\x7f\xfa\xe8\x40\xbe\xc9\x2d\xc6\xa0\x13\x80\x47\xd5\x1c\x39\xf1\xe7\x71\x33\x57\x9b\xea\xec\x35\xfa\x50\x29\x51\xa1" + ) + , + ( 16 + , 20 + , 191 + , "\x8c\xa1\x85\xb7\xd9\xde\x69\xa0\x5e\x4d\xe6\xcc\xc2\x2e\x20\x3b" + , "\x6e\x35\x69\x18\x73\x5e\x0e\x3a\xa0\x0d\x28\x0b\x07\x13\xde\xc3\x68\x97\xbd\xa5\x32\xd8\xb0\x60\x27\x61\x58\x8e\x71\xb0\xcb\x34" + ) + , + ( 16 + , 20 + , 192 + , "\x68\x29\x26\xd8\x68\x9f\x8e\x82\xca\x97\xfe\x12\x56\x3c\xae\xb5" + , "\x6b\x55\x0d\x71\x75\xba\xd4\x2c\x0f\x3d\x44\x24\xad\x19\xd5\x66\xbc\xa6\x5d\xcc\xc8\xc7\x81\x4f\x6e\x33\x2d\xb8\x67\x67\x02\x87" + ) + , + ( 16 + , 20 + , 287 + , "\xd9\xf4\xe4\xf2\xf8\x3d\x83\xf1\x5e\x84\x5d\x3f\x91\x23\x3a\xde" + , "\x1f\xa9\x9b\x5e\x49\xfa\xe4\x99\x3f\x90\x95\x04\x7f\x92\xf6\xba\xa3\x8e\x79\xae\x7b\xdd\x2f\x3d\xba\x88\xe6\x18\xa9\xed\xc6\x78" + ) + , + ( 16 + , 20 + , 288 + , "\xf1\x26\x65\xfa\x09\x8c\xde\x42\x98\x18\x2d\x02\x76\x4c\x38\x17" + , "\x35\x84\x4c\x16\xd6\x16\x31\x04\xd1\x43\xc7\xcd\x10\x90\x41\x03\x32\x48\xe9\x5c\xfd\xc0\x4c\xef\x42\x52\xc0\xe0\x45\xec\x8e\x0f" + ) + , + ( 16 + , 20 + , 289 + , "\x4e\x47\x7c\x8c\x3c\x62\xd6\xba\xe0\xb7\x62\x17\x61\x85\x3a\x78" + , "\xce\x08\x82\x6c\x10\x30\xd8\xda\xf8\x01\xd7\x47\x85\xd3\x4b\xbe\x25\xb5\x43\x52\x50\x90\xa9\x29\x8f\x48\xc4\x1c\x16\x90\x0c\xee" + ) + , + ( 16 + , 20 + , 304 + , "\xfc\xa6\xc1\xad\x64\xd6\x92\xba\xa0\xa8\x4b\x81\x0f\xac\x15\x33" + , "\x9f\x74\x66\xcf\xdd\x78\xaf\x93\x36\x0e\xe2\xf2\xc5\x74\xed\xdf\xb3\xb6\x40\xcf\x3a\xa5\xd0\x61\x12\x3e\x09\x07\xae\x4a\x0f\xef" + ) + , + ( 16 + , 20 + , 383 + , "\xa0\x31\x4b\xd9\x85\x83\xe7\x9e\x8b\xfc\xbb\xe1\x01\x18\x44\x17" + , "\x23\xdb\xda\x5f\xd3\xf1\x66\x1b\xc9\xb6\xb4\x88\x21\x3e\x8c\xc5\xc9\x7f\xd7\x6e\x8a\x26\x4b\x09\xc8\x43\x64\x1a\xb0\x41\xb2\xb5" + ) + , + ( 16 + , 20 + , 384 + , "\x3f\xa5\x13\xb6\x87\x50\x88\x6c\x81\x6d\x6e\x9a\x20\xa5\x74\x6f" + , "\x67\xec\x7b\x0e\x61\x15\xbb\xc5\xfa\x00\x43\xad\x07\x89\x6e\xff\x77\xe9\xe8\x7e\x6f\x76\xb1\xd8\xb5\x08\x36\xdc\xde\x19\xb9\xb8" + ) + , + ( 16 + , 20 + , 385 + , "\xf4\x22\xff\x5f\x0e\x3f\xa1\xfa\x44\xa2\x78\x5d\xf0\xb9\x48\x11" + , "\x8e\xf0\x14\x46\x53\x7d\x9c\x16\xb0\xd8\x7e\xfa\xff\xf1\xa9\x9a\x83\x76\xe2\x1b\xd3\x9b\xc8\x61\xb7\xf8\x13\xa4\xed\x06\xfd\xce" + ) + , + ( 16 + , 20 + , 480 + , "\x56\xb6\xac\xdb\x6f\x9b\xf9\x69\xc0\x01\x21\x93\xf5\xe0\xab\xd6" + , "\x73\x56\xb8\xf7\x72\x11\x76\xc4\x0b\x51\xd2\xa6\x01\xe9\x2a\x2e\x09\x7e\x42\xfa\x3e\x53\xd1\x79\x46\xb3\x3a\x20\x17\x1a\x90\xa3" + ) + , + ( 16 + , 20 + , 576 + , "\xec\x0f\x49\x67\xaf\x67\x63\x78\x13\xe3\x23\xa4\x36\x76\xcc\xe2" + , "\x74\x93\xe2\xe2\x80\x8d\xb2\xc9\xc8\xe1\x99\x54\x63\xc5\x3f\xce\xea\x9d\x54\x97\x94\x08\x99\xce\xdf\x8c\xb9\x2f\x9d\xf7\xe1\xb8" + ) + , + ( 16 + , 20 + , 1023 + , "\x7a\xe2\x50\x10\xe9\x53\x9d\x2e\xd7\xa4\x2e\x66\x72\x38\xfd\x7f" + , "\xe4\xf8\x93\x21\x93\x1d\x07\x3c\x3a\x7b\xf1\xf5\x81\x7f\x86\xbd\xf5\x4f\xdb\xed\xa5\x28\x67\xba\x32\x91\xe0\xe5\x06\xe6\x35\x6d" + ) + , + ( 16 + , 20 + , 1024 + , "\x6e\xbb\x6d\x02\x4f\xd1\xf0\x2a\x86\xee\xf6\xbd\xf4\xe7\x9d\x21" + , "\x47\x29\x52\xfa\xa4\xe7\xeb\xea\x8c\x57\x95\xa6\x8b\x1b\xf0\xdf\x04\x8e\xa6\xd7\x12\x0c\xb1\x7d\x25\x9f\x00\xfe\xab\x67\x4f\x22" + ) + , + ( 16 + , 20 + , 1025 + , "\x4c\xfe\x1e\xf9\xcc\x4e\x5d\xc0\x80\x79\xa5\x58\xf2\x27\xfa\xa6" + , "\xf3\x7a\xf7\xec\xd0\x99\x32\xe8\xdd\xb7\xc7\x96\x01\x44\xba\xe0\x28\x3c\xe3\x7a\x25\x87\x5e\xd9\xd7\x42\x45\xcb\x04\x01\x92\x5d" + ) + , + ( 16 + , 20 + , 4099 + , "\x4b\x37\x40\xe3\x59\xd6\x2f\x1d\x40\xb9\x14\xb0\xf1\xd7\x6f\xf7" + , "\x91\xae\x42\x66\xa7\x07\x98\x08\xaa\x85\x75\xdc\x4a\xa2\x29\x7b\x89\x44\x67\xbd\x4d\x40\x2d\x15\x62\xeb\x3f\xd6\x66\x9b\x3b\x32" + ) + , + ( 24 + , 0 + , 0 + , "\xde\xec\xf4\x23\xc2\x6e\x84\xa0\x91\x2b\x1d\xb2\x32\x47\x0b\xfa" + , "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24\x27\xae\x41\xe4\x64\x9b\x93\x4c\xa4\x95\x99\x1b\x78\x52\xb8\x55" + ) + , + ( 24 + , 0 + , 1 + , "\xe0\x1e\x9d\x0a\x98\xd0\x02\x27\xf8\x68\x4c\x8e\x97\xd6\x9c\xc0" + , "\x65\xc7\x4c\x15\xa6\x86\x18\x7b\xb6\xbb\xf9\x95\x8f\x49\x4f\xc6\xb8\x00\x68\x03\x4a\x65\x9a\x9a\xd4\x49\x91\xb0\x8c\x58\xf2\xd2" + ) + , + ( 24 + , 0 + , 15 + , "\x19\x32\x52\x67\x67\xbb\xee\x27\xec\xd4\xef\xe1\x15\x5b\x94\x4f" + , "\xd9\x1f\x32\x05\x01\xb1\xb6\xd1\xf7\x18\x68\xbe\x66\x57\x31\xa9\xa2\x04\x8a\xbe\xf3\xaf\xf4\x56\x78\xb6\x79\xee\xb3\x49\xae\xf9" + ) + , + ( 24 + , 0 + , 16 + , "\x02\xa9\x6d\x98\x17\x40\x13\xa1\x32\x9c\xd9\x7d\xfc\x3f\x56\x28" + , "\x46\x9f\xd3\x7f\x52\x32\x1f\xd3\x01\xfe\x91\x3f\xa5\xb0\xec\xfe\xf8\x29\x25\x2e\x54\x08\xe1\xe8\x9b\x28\x1d\xba\xe4\x6b\x0d\x5e" + ) + , + ( 24 + , 0 + , 17 + , "\x9d\x65\xb0\x7e\xea\x52\xb9\xc7\x30\x6f\xff\x13\x66\x78\x5f\x0c" + , "\xed\xcf\x38\x1d\x7b\x81\xb9\x0e\xf0\x70\x40\xe4\x32\x20\x9b\x63\x83\x2f\x64\x3a\x21\x63\xb3\x13\x10\x2a\x27\x48\xca\xd9\x46\xf1" + ) + , + ( 24 + , 0 + , 95 + , "\x0c\x44\x17\x11\x74\x65\x4f\xe7\xdc\x3a\x42\x21\xde\x23\xeb\x7b" + , "\xaa\x05\xb9\x9d\x9f\xfa\x15\x33\x76\xf8\x57\x57\xa7\xa5\x35\x37\xb6\xde\x4d\xa4\x0c\x7e\xfd\xbf\xb1\x18\x05\x99\xdc\xd0\xce\x6a" + ) + , + ( 24 + , 0 + , 96 + , "\x37\x49\x2f\xd9\xbd\x45\xf8\x92\x3d\x91\x7c\x69\x6a\xef\xae\xd2" + , "\x2e\xa4\x70\xf5\xc1\x71\xdd\x63\x9b\x7e\xec\x37\x97\xbf\xb3\x07\xd2\x8a\xd1\x4b\xc9\xf0\xbd\xa2\xe9\xb1\xf2\x47\xee\x9f\x33\x59" + ) + , + ( 24 + , 0 + , 97 + , "\x94\x61\x1a\x75\xb0\x1f\x65\xa5\x80\xce\x0d\x60\x35\xe9\x11\x3c" + , "\x87\xfb\x52\x87\xcb\xba\xe9\x93\xd1\x58\x0e\x3c\x00\x13\xe5\x22\xc3\x4a\x2b\x5f\xd3\xd2\xfe\x88\xb9\x1e\x6f\x33\xe7\x30\xf4\x9a" + ) + , + ( 24 + , 0 + , 112 + , "\xc7\x4e\x6c\x7c\x40\x68\xec\x0c\xc9\x39\x65\x68\x2f\xcb\x2d\xf0" + , "\x41\x21\x02\x79\x9c\x39\xba\xd7\x69\x82\x43\x72\xe0\x0d\x58\x83\x0b\x8f\x20\x60\xfa\xd0\xdf\xdc\x1e\xa8\x82\xbc\xc4\x45\x15\x66" + ) + , + ( 24 + , 0 + , 159 + , "\xc1\x39\x4c\x2c\xd7\xa9\x24\x04\xab\x16\x5f\xa4\xed\xbe\x58\x63" + , "\xe5\x30\x31\x97\xf8\x41\x02\xc6\x01\x96\xa6\x07\x63\x03\x5c\x3a\x99\xd9\xcc\xe3\x7b\xe2\x79\x5e\x30\x5d\x3f\xde\xdd\x4a\xea\xc4" + ) + , + ( 24 + , 0 + , 160 + , "\xae\xf5\x38\xe8\xd5\x76\xe4\xc9\x41\x12\xb9\x7f\x1b\xeb\x61\xf7" + , "\x06\xc3\x15\x57\x3b\x25\xd1\x46\x44\x50\x90\xac\xd0\xe9\x3f\xc1\xbe\x70\x87\x81\x50\x24\xfe\xc1\x74\xa2\xb2\x56\xdf\x70\x56\x5c" + ) + , + ( 24 + , 0 + , 191 + , "\xd7\x76\x00\x3f\x3b\x10\x8d\x9a\xe0\xdf\x75\x7a\x8e\x67\x8f\xb9" + , "\x8a\xa2\xdc\x1a\xaf\x64\x4d\x50\xab\x7a\xce\x06\x88\x81\x8d\x8f\x51\xf3\x50\xe1\x6a\xdc\x22\xd5\x6c\x4b\xde\x65\xe0\x02\xa4\x33" + ) + , + ( 24 + , 0 + , 192 + , "\x2f\x2a\x65\xc9\x66\x26\xe0\x86\x1d\xf7\x0d\xc3\x70\x26\xfa\xc7" + , "\x35\xeb\xd6\xe5\xb4\x71\x04\x81\xc4\xcb\xc3\x19\xb1\x4e\x66\xe1\x3b\xf4\xeb\xc5\x89\x47\xff\xc6\xe2\xba\x86\x90\x65\x64\xc4\x18" + ) + , + ( 24 + , 0 + , 287 + , "\x4d\xfa\xff\xd1\xef\x7a\x7d\x38\xb5\x79\xf1\xe4\x50\x5b\xc3\xfa" + , "\xa2\xab\x3f\xb9\x12\x71\x69\xf1\xc8\xd3\x57\x32\x28\x0f\x16\x37\x67\xdf\x3c\x72\x8d\x1d\xc9\x83\x5f\x73\x19\xca\x3f\x4d\x09\xe3" + ) + , + ( 24 + , 0 + , 288 + , "\x0f\xfc\x69\xc3\x34\x22\x07\x45\xfc\x17\x85\x08\xf2\x5c\x09\xa2" + , "\xac\x5f\x41\x3a\xf8\xee\x4d\xb9\x32\xc5\x93\x98\x8b\xc1\xae\x2e\x01\x0b\x03\x1f\xdf\x2c\x64\x25\xf7\x23\x02\x38\x22\xe1\x6e\xfe" + ) + , + ( 24 + , 0 + , 289 + , "\xcb\xc8\x53\x8f\x9e\x35\xf5\x09\xc3\xc4\x9d\x02\x57\xfc\x92\x70" + , "\x87\x8a\x09\x45\xdd\x6b\xff\xc0\xe0\x35\x90\x04\x39\xba\x1b\xf3\xa2\xf0\x8e\x33\x42\x25\x01\x94\x32\xec\x98\x98\xaa\x52\xe0\x01" + ) + , + ( 24 + , 0 + , 304 + , "\x24\x22\x57\xcc\x04\x75\xe0\x03\xd6\x27\xea\xbd\xe9\x9b\xdf\xd4" + , "\x74\x2a\x92\x83\x4c\x30\x1c\x6a\xe4\x86\xbc\x6d\x2b\x06\xfc\x44\x6c\x4c\x3e\x8a\x40\xb4\x75\xee\xbc\xde\x23\xb1\x81\x3f\xe4\x5d" + ) + , + ( 24 + , 0 + , 383 + , "\xb8\x0f\x07\x13\xf1\x56\x88\xa6\x19\xa3\xb7\x9a\xa0\x04\x5c\x10" + , "\xe9\x6d\x7c\x77\x8d\xb2\xc5\x46\x58\x61\xb2\x8c\x91\xe9\xf8\x51\x5b\xdf\xd1\x60\x9d\x7b\x53\x82\xd0\x93\xac\x7e\x99\xf0\x76\x85" + ) + , + ( 24 + , 0 + , 384 + , "\xd0\x2c\x80\x49\x2d\x59\xbf\x70\xe0\x2b\x15\x2a\xd1\xa9\xc8\x4a" + , "\xf1\x56\x3c\x1f\x6f\x66\xa0\x20\xe3\xbc\xe2\x0b\xd6\x84\xd1\xe8\xca\xb3\xde\x9c\xb2\xfb\xeb\x12\x50\x9e\x00\x7d\x65\x3c\xbc\xdf" + ) + , + ( 24 + , 0 + , 385 + , "\xbd\x25\xd2\xde\x68\xde\x95\xee\x2f\xaa\x33\xf3\x15\xf0\x80\xf2" + , "\x29\x52\x4a\xb6\x53\xb5\x5e\x79\xf5\xb3\x0d\x26\xf7\x4b\xa9\xa6\x91\xd7\xda\xd6\x44\xd4\x91\x0b\x5f\x31\xd0\x68\xfa\x80\x0e\x61" + ) + , + ( 24 + , 0 + , 480 + , "\x57\x1d\xc8\x25\x4e\xb6\x05\x4a\xa0\x72\x97\x82\x63\x66\xe1\xb2" + , "\x06\x26\x73\x50\x9a\x02\x1b\xeb\x8c\x0c\x91\x8e\x9d\xc8\xfe\x2e\xf5\x0f\xfc\x14\xf8\x85\xa2\xd5\xde\xc3\xa4\x92\x60\x2b\xc5\x74" + ) + , + ( 24 + , 0 + , 576 + , "\x26\x24\x08\xa8\x37\x64\xb1\x10\xf8\xdd\xd7\x7c\x8d\x6c\xff\x1f" + , "\x6d\x6a\x95\xc9\xef\xb7\x58\xfe\xf3\x20\xc5\x6c\x9c\x18\xf8\x37\x29\x96\xed\xc6\xa8\xac\x63\x42\xce\xc4\x5d\xf4\xe9\x66\xaa\x50" + ) + , + ( 24 + , 0 + , 1023 + , "\xd7\x2c\xff\x00\xc9\x7a\x62\x4f\x17\xea\x4f\x8a\xcc\x3d\x37\xc7" + , "\x59\x73\x52\x3c\x98\x5c\x8c\xd0\xff\x86\x73\x52\x6a\xce\xa9\xba\x1a\x6f\xd2\x99\x00\xd5\xcb\x6d\xb3\x29\x5d\x40\xad\xe4\x30\x63" + ) + , + ( 24 + , 0 + , 1024 + , "\xba\x02\xe9\xb9\x06\x78\xd4\x85\x1f\xdf\x6d\x84\x06\x10\xe6\xc0" + , "\xb7\xcb\x38\xbe\xc7\x5a\x07\x7f\x49\x5d\xbd\x3b\xdd\xfe\x0f\x74\xe9\x58\x3b\x69\x8f\x0a\xed\xc4\x2a\x8b\x9e\x69\x8d\xc1\x35\xec" + ) + , + ( 24 + , 0 + , 1025 + , "\x2e\xb4\x24\xa2\x19\x2c\x65\x62\x10\xaf\x17\x86\xf5\xc5\x9d\xbf" + , "\xe9\x00\xd8\x3e\x27\x37\xbe\xc8\x37\x67\xe1\x0e\x56\xf2\x64\x7e\xa8\xd3\x0b\x1a\x5b\x2a\xc5\xb3\x9f\x96\x11\x2a\x0c\xd7\x6d\xca" + ) + , + ( 24 + , 0 + , 4099 + , "\x52\x69\x7a\xa8\x59\x50\x2a\xb5\x57\x43\x1f\xdb\x5b\xe8\xcf\x68" + , "\x32\x46\x97\xc8\x6d\xe2\x6f\x89\x4e\xe0\x93\xd4\x3a\xd0\x80\xbf\x76\x1c\xa0\xd1\xd1\x5e\xb5\x69\x38\xb0\xae\x06\xf3\x0b\x58\xb7" + ) + , + ( 24 + , 20 + , 0 + , "\xe0\xa2\x25\x4c\x39\x7e\xbb\xe0\x2c\x66\x72\xe1\xb4\xe6\x3a\xac" + , "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24\x27\xae\x41\xe4\x64\x9b\x93\x4c\xa4\x95\x99\x1b\x78\x52\xb8\x55" + ) + , + ( 24 + , 20 + , 1 + , "\xb7\x6a\x1c\xcf\x2b\x91\xfb\x80\xf0\x42\x3a\x80\xbc\x7a\xb2\x14" + , "\x65\xc7\x4c\x15\xa6\x86\x18\x7b\xb6\xbb\xf9\x95\x8f\x49\x4f\xc6\xb8\x00\x68\x03\x4a\x65\x9a\x9a\xd4\x49\x91\xb0\x8c\x58\xf2\xd2" + ) + , + ( 24 + , 20 + , 15 + , "\x4e\x46\xd3\xa2\xd4\xfa\x17\x80\xe4\xfe\x99\xef\x3e\xf7\xba\x9b" + , "\xd9\x1f\x32\x05\x01\xb1\xb6\xd1\xf7\x18\x68\xbe\x66\x57\x31\xa9\xa2\x04\x8a\xbe\xf3\xaf\xf4\x56\x78\xb6\x79\xee\xb3\x49\xae\xf9" + ) + , + ( 24 + , 20 + , 16 + , "\x55\xdd\xec\x5d\xa4\x01\xea\x06\x3a\xb6\xaf\x73\xd7\x93\x78\xfc" + , "\x46\x9f\xd3\x7f\x52\x32\x1f\xd3\x01\xfe\x91\x3f\xa5\xb0\xec\xfe\xf8\x29\x25\x2e\x54\x08\xe1\xe8\x9b\x28\x1d\xba\xe4\x6b\x0d\x5e" + ) + , + ( 24 + , 20 + , 17 + , "\xda\x17\xad\xe5\xb6\x78\x02\x7f\xd8\x84\x1b\x0a\x9d\x63\x93\xd8" + , "\xed\xcf\x38\x1d\x7b\x81\xb9\x0e\xf0\x70\x40\xe4\x32\x20\x9b\x63\x83\x2f\x64\x3a\x21\x63\xb3\x13\x10\x2a\x27\x48\xca\xd9\x46\xf1" + ) + , + ( 24 + , 20 + , 95 + , "\x4e\xff\xd3\x66\x1b\xa8\x92\x22\x92\xf8\x95\x72\xed\xb8\x55\xee" + , "\xaa\x05\xb9\x9d\x9f\xfa\x15\x33\x76\xf8\x57\x57\xa7\xa5\x35\x37\xb6\xde\x4d\xa4\x0c\x7e\xfd\xbf\xb1\x18\x05\x99\xdc\xd0\xce\x6a" + ) + , + ( 24 + , 20 + , 96 + , "\x75\xf2\xeb\xae\xd2\x88\x25\x57\x73\x53\xab\x3a\x59\x74\x10\x47" + , "\x2e\xa4\x70\xf5\xc1\x71\xdd\x63\x9b\x7e\xec\x37\x97\xbf\xb3\x07\xd2\x8a\xd1\x4b\xc9\xf0\xbd\xa2\xe9\xb1\xf2\x47\xee\x9f\x33\x59" + ) + , + ( 24 + , 20 + , 97 + , "\x2d\xd5\x1a\xd4\x39\x81\xe0\xec\x05\xc2\x4d\x5b\xea\xda\xc7\x28" + , "\x87\xfb\x52\x87\xcb\xba\xe9\x93\xd1\x58\x0e\x3c\x00\x13\xe5\x22\xc3\x4a\x2b\x5f\xd3\xd2\xfe\x88\xb9\x1e\x6f\x33\xe7\x30\xf4\x9a" + ) + , + ( 24 + , 20 + , 112 + , "\x7e\xfa\x6c\xdd\xc9\xf6\x69\x45\x4c\x35\x25\x53\xf0\xf8\xfb\xe4" + , "\x41\x21\x02\x79\x9c\x39\xba\xd7\x69\x82\x43\x72\xe0\x0d\x58\x83\x0b\x8f\x20\x60\xfa\xd0\xdf\xdc\x1e\xa8\x82\xbc\xc4\x45\x15\x66" + ) + , + ( 24 + , 20 + , 159 + , "\x1b\x6d\x91\x8b\x64\x0f\xd0\xb2\xb1\xe1\xef\xa2\xad\xab\xf5\x8c" + , "\xe5\x30\x31\x97\xf8\x41\x02\xc6\x01\x96\xa6\x07\x63\x03\x5c\x3a\x99\xd9\xcc\xe3\x7b\xe2\x79\x5e\x30\x5d\x3f\xde\xdd\x4a\xea\xc4" + ) + , + ( 24 + , 20 + , 160 + , "\x74\xa1\xe5\x4f\x66\xd0\x10\x7f\x5b\xe5\x09\x79\x5b\xfe\xcc\x18" + , "\x06\xc3\x15\x57\x3b\x25\xd1\x46\x44\x50\x90\xac\xd0\xe9\x3f\xc1\xbe\x70\x87\x81\x50\x24\xfe\xc1\x74\xa2\xb2\x56\xdf\x70\x56\x5c" + ) + , + ( 24 + , 20 + , 191 + , "\xaa\x66\x04\x97\xba\x14\x6f\xd4\xf1\xf4\xdf\x59\xc0\x51\x68\xf2" + , "\x8a\xa2\xdc\x1a\xaf\x64\x4d\x50\xab\x7a\xce\x06\x88\x81\x8d\x8f\x51\xf3\x50\xe1\x6a\xdc\x22\xd5\x6c\x4b\xde\x65\xe0\x02\xa4\x33" + ) + , + ( 24 + , 20 + , 192 + , "\x52\x3a\x61\x61\xe7\x22\x02\xc8\x0c\xdc\xa7\xe0\x3e\x10\x1d\x8c" + , "\x35\xeb\xd6\xe5\xb4\x71\x04\x81\xc4\xcb\xc3\x19\xb1\x4e\x66\xe1\x3b\xf4\xeb\xc5\x89\x47\xff\xc6\xe2\xba\x86\x90\x65\x64\xc4\x18" + ) + , + ( 24 + , 20 + , 287 + , "\x67\x5b\x82\x18\xe9\x49\xda\x5f\x0d\x61\x73\x51\x23\x5b\xd7\xde" + , "\xa2\xab\x3f\xb9\x12\x71\x69\xf1\xc8\xd3\x57\x32\x28\x0f\x16\x37\x67\xdf\x3c\x72\x8d\x1d\xc9\x83\x5f\x73\x19\xca\x3f\x4d\x09\xe3" + ) + , + ( 24 + , 20 + , 288 + , "\x25\x5d\x14\x0a\x32\x11\xa0\x22\x44\x0f\x07\xbd\x81\x5c\x1d\x86" + , "\xac\x5f\x41\x3a\xf8\xee\x4d\xb9\x32\xc5\x93\x98\x8b\xc1\xae\x2e\x01\x0b\x03\x1f\xdf\x2c\x64\x25\xf7\x23\x02\x38\x22\xe1\x6e\xfe" + ) + , + ( 24 + , 20 + , 289 + , "\xd3\xd3\x52\x46\x22\x31\xff\x27\xcc\x5c\x7d\xcc\x41\xae\x4a\xcd" + , "\x87\x8a\x09\x45\xdd\x6b\xff\xc0\xe0\x35\x90\x04\x39\xba\x1b\xf3\xa2\xf0\x8e\x33\x42\x25\x01\x94\x32\xec\x98\x98\xaa\x52\xe0\x01" + ) + , + ( 24 + , 20 + , 304 + , "\x3c\x39\x56\x05\xb8\x71\xea\x2d\xd9\xbf\x0a\x73\xff\xc9\x07\x69" + , "\x74\x2a\x92\x83\x4c\x30\x1c\x6a\xe4\x86\xbc\x6d\x2b\x06\xfc\x44\x6c\x4c\x3e\x8a\x40\xb4\x75\xee\xbc\xde\x23\xb1\x81\x3f\xe4\x5d" + ) + , + ( 24 + , 20 + , 383 + , "\xae\xd0\x96\xb8\x91\x86\xea\xca\xa2\x82\x17\xee\xdc\x1e\x7b\x26" + , "\xe9\x6d\x7c\x77\x8d\xb2\xc5\x46\x58\x61\xb2\x8c\x91\xe9\xf8\x51\x5b\xdf\xd1\x60\x9d\x7b\x53\x82\xd0\x93\xac\x7e\x99\xf0\x76\x85" + ) + , + ( 24 + , 20 + , 384 + , "\xc6\xf3\x11\xe2\x4d\x89\xdd\x1c\x5b\x0a\xb5\x5e\xad\xb3\xef\x7c" + , "\xf1\x56\x3c\x1f\x6f\x66\xa0\x20\xe3\xbc\xe2\x0b\xd6\x84\xd1\xe8\xca\xb3\xde\x9c\xb2\xfb\xeb\x12\x50\x9e\x00\x7d\x65\x3c\xbc\xdf" + ) + , + ( 24 + , 20 + , 385 + , "\xfa\x94\xce\xc6\x97\x91\xa9\x80\x15\x59\x28\xea\xe6\xdf\xe3\xd4" + , "\x29\x52\x4a\xb6\x53\xb5\x5e\x79\xf5\xb3\x0d\x26\xf7\x4b\xa9\xa6\x91\xd7\xda\xd6\x44\xd4\x91\x0b\x5f\x31\xd0\x68\xfa\x80\x0e\x61" + ) + , + ( 24 + , 20 + , 480 + , "\x13\x29\x88\xa0\x26\x7d\x2b\x23\xa8\x6c\xf3\xb4\x09\xa3\x0b\xef" + , "\x06\x26\x73\x50\x9a\x02\x1b\xeb\x8c\x0c\x91\x8e\x9d\xc8\xfe\x2e\xf5\x0f\xfc\x14\xf8\x85\xa2\xd5\xde\xc3\xa4\x92\x60\x2b\xc5\x74" + ) + , + ( 24 + , 20 + , 576 + , "\x2f\x00\x96\x82\x57\xc1\xfc\x88\xe0\x91\x30\x5b\x70\xba\x20\x9a" + , "\x6d\x6a\x95\xc9\xef\xb7\x58\xfe\xf3\x20\xc5\x6c\x9c\x18\xf8\x37\x29\x96\xed\xc6\xa8\xac\x63\x42\xce\xc4\x5d\xf4\xe9\x66\xaa\x50" + ) + , + ( 24 + , 20 + , 1023 + , "\x2a\x08\x3e\x74\x74\xf9\x58\x5b\x19\xd4\x24\x9e\xfe\x05\x63\x39" + , "\x59\x73\x52\x3c\x98\x5c\x8c\xd0\xff\x86\x73\x52\x6a\xce\xa9\xba\x1a\x6f\xd2\x99\x00\xd5\xcb\x6d\xb3\x29\x5d\x40\xad\xe4\x30\x63" + ) + , + ( 24 + , 20 + , 1024 + , "\x47\x26\x28\xcd\xbb\xfb\xee\x91\x11\xe1\x06\x90\x34\x28\xb2\x3e" + , "\xb7\xcb\x38\xbe\xc7\x5a\x07\x7f\x49\x5d\xbd\x3b\xdd\xfe\x0f\x74\xe9\x58\x3b\x69\x8f\x0a\xed\xc4\x2a\x8b\x9e\x69\x8d\xc1\x35\xec" + ) + , + ( 24 + , 20 + , 1025 + , "\xc8\x2a\xd6\x14\xfd\xfa\xae\x3b\x0f\x3c\xb7\xf7\x41\x3e\xf2\x62" + , "\xe9\x00\xd8\x3e\x27\x37\xbe\xc8\x37\x67\xe1\x0e\x56\xf2\x64\x7e\xa8\xd3\x0b\x1a\x5b\x2a\xc5\xb3\x9f\x96\x11\x2a\x0c\xd7\x6d\xca" + ) + , + ( 24 + , 20 + , 4099 + , "\x9e\x5b\x7c\xc7\xf3\xdc\x07\x4f\x8c\x71\xaa\xf4\x64\x6b\xe9\x57" + , "\x32\x46\x97\xc8\x6d\xe2\x6f\x89\x4e\xe0\x93\xd4\x3a\xd0\x80\xbf\x76\x1c\xa0\xd1\xd1\x5e\xb5\x69\x38\xb0\xae\x06\xf3\x0b\x58\xb7" + ) + , + ( 32 + , 0 + , 0 + , "\xb5\xc4\xd9\x83\x2d\x9d\x6e\xef\x47\xaf\xb7\xaf\x4b\x0b\x0e\xc0" + , "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24\x27\xae\x41\xe4\x64\x9b\x93\x4c\xa4\x95\x99\x1b\x78\x52\xb8\x55" + ) + , + ( 32 + , 0 + , 1 + , "\xc2\xd0\x6f\x2a\x01\x8b\xce\x09\xa9\xac\xb1\xe7\x53\xc3\x89\x7f" + , "\x01\xba\x47\x19\xc8\x0b\x6f\xe9\x11\xb0\x91\xa7\xc0\x51\x24\xb6\x4e\xee\xce\x96\x4e\x09\xc0\x58\xef\x8f\x98\x05\xda\xca\x54\x6b" + ) + , + ( 32 + , 0 + , 15 + , "\xc6\xea\xfc\xe0\xc8\x78\x79\xfc\xa2\xd7\xe4\xc7\x0c\xcb\x6d\xbb" + , "\xb3\xcd\xe8\x23\xf3\xf5\xb3\xa9\x2e\x3d\xe2\x2f\xc6\xd6\x3a\x7f\xf2\x0b\x61\xfe\x89\xd7\x34\xd4\xb1\x24\xb8\x42\x50\x63\xc5\xf5" + ) + , + ( 32 + , 0 + , 16 + , "\xa0\xf8\x35\x8c\x3e\xac\xd1\xcb\xab\x1f\xe1\x4a\x74\xb1\x37\xa3" + , "\x9b\x8e\x57\x27\x94\x78\xbe\xbc\x9b\xb8\x79\xd1\xf8\xf4\x32\x45\x60\xa4\x1e\xf2\xae\x6d\x06\x33\x2f\x6d\xd7\x28\x14\x19\x12\x53" + ) + , + ( 32 + , 0 + , 17 + , "\x6f\xc5\x38\x71\x9a\xdd\x1a\xc9\x63\x82\x7c\x32\x02\x4a\x32\x77" + , "\x00\x56\xd7\x84\x80\xe4\x6a\x70\xbb\xee\xb2\xf3\xe4\x7e\x2c\x32\x9c\xa1\xba\xb6\xb0\x45\x04\x34\x86\xca\xf8\xe7\x1f\x18\xd0\x8b" + ) + , + ( 32 + , 0 + , 95 + , "\x52\x3a\xd6\xf4\x0b\x01\xe1\xfd\x01\xb9\xb0\xfa\x38\xc4\xb4\x03" + , "\x1f\x84\x11\xd8\xcb\x11\x28\x70\x33\x0f\x63\x73\xcd\x70\xf4\xa5\x6d\xd4\x53\x57\x25\x96\xb1\x42\x78\xb4\x54\xde\x0b\x37\x4a\x03" + ) + , + ( 32 + , 0 + , 96 + , "\xad\x90\x66\x28\xb1\xbb\x7d\x91\x63\x9d\x87\xc8\x04\x10\xb3\x1f" + , "\x4c\xad\x61\x9d\x98\x48\xc5\xbc\x0d\x3c\xe1\x5d\xaf\x01\x6a\x70\x85\xe0\xc9\x40\x9e\xb9\x07\xb9\x96\x2b\x26\x46\x11\xae\x9f\x32" + ) + , + ( 32 + , 0 + , 97 + , "\xa2\xbf\x4c\x3a\x0b\x45\x8c\x92\x68\x6c\x69\x98\x04\x56\xcc\x9d" + , "\x5e\x52\xc6\x8b\x0c\xac\xd6\x4b\xaf\x46\x8c\x96\xe5\xc3\x9a\xaa\x5e\x45\x1e\x22\xc0\x09\xb9\x72\x06\x33\xc2\xab\x18\x0d\x03\x85" + ) + , + ( 32 + , 0 + , 112 + , "\xad\x3e\x50\x9b\x71\x98\x5d\x9f\xe2\x54\x73\x5f\x8d\xa0\x27\xbf" + , "\x70\xad\x3d\xad\x9a\x1e\xc4\x5c\x3c\xea\xe1\x95\x15\xb8\x19\xd2\x83\xbd\x01\x62\xe5\xb7\x42\xd8\x45\xdd\xbe\x51\xc9\xca\x85\x0c" + ) + , + ( 32 + , 0 + , 159 + , "\x9e\xcb\xa1\x29\x19\x43\x66\x2f\xd1\x3a\xf5\x71\xfb\x02\x96\x3b" + , "\xf8\x6d\xd2\x2a\x72\x18\x2d\xde\x32\xa6\x77\xd9\xbd\x0f\x8f\x07\x32\x68\x00\xa9\xd2\x9c\x2b\x52\x91\x7a\x37\x0f\xe4\xae\x1f\xc7" + ) + , + ( 32 + , 0 + , 160 + , "\x8c\xf8\xc8\xc8\x92\xc6\x56\x45\xeb\x05\x5d\xd4\xc2\x78\x1c\xed" + , "\x90\xd2\xe7\xf0\x9c\x18\xcd\x98\xb9\x92\xdd\xc2\x02\xd6\x23\x9c\x31\xab\x14\x7d\x32\x38\x0f\x97\x87\x3a\xec\x2a\x59\x2a\x70\x1f" + ) + , + ( 32 + , 0 + , 191 + , "\xc8\xe7\xa5\xcf\x44\xf6\xd9\x81\xa2\x82\x54\x7e\x16\x7f\xe5\x4e" + , "\xc7\x39\xa4\xd7\x2f\xe5\x4b\x57\x24\x42\x56\xdf\xc7\x59\x74\x93\x70\x2c\x8a\xea\x19\xe6\x27\x66\xe4\x03\x74\xc2\x29\x23\x72\x2b" + ) + , + ( 32 + , 0 + , 192 + , "\x52\xdd\x33\x17\xb8\x66\x97\x63\x79\x17\x21\x59\xee\xdf\x2a\x4f" + , "\x5c\xdf\x56\xa2\xe4\xe5\x00\xf5\xbe\xce\x4c\x9f\x69\xed\x5d\x6d\x71\x5f\x6d\x96\x16\x5c\x55\x3c\xc1\x2f\xe3\x86\x11\x9d\x8e\x26" + ) + , + ( 32 + , 0 + , 287 + , "\xe7\x5a\x6f\x25\x4f\x39\xde\xe0\x43\x49\xb0\x5c\x66\x94\x1e\x70" + , "\xa4\xd8\x0b\x4a\x6a\xc9\xf1\xed\x0d\x47\x1d\x4b\xd9\x7d\xc6\x3d\xae\xcf\x2a\x57\x9b\xeb\xab\x95\x61\x70\x22\x4b\xa7\x58\x28\x71" + ) + , + ( 32 + , 0 + , 288 + , "\x19\x9e\xdd\xe6\xb1\xcf\xb9\xb7\x40\x45\xd2\x7b\xa2\xe8\xe8\x92" + , "\xc4\xf5\x66\x65\x76\xfd\x49\x60\xee\xe9\x1f\xde\x3f\x36\x99\xd8\x5b\xcb\xf9\x4e\xbc\xb8\x31\xb6\xcb\x9e\x7e\x48\x68\xb5\xcb\xed" + ) + , + ( 32 + , 0 + , 289 + , "\x24\x58\x17\x9c\x3c\xf2\x9e\x1c\xfe\xc3\x6c\xf5\x98\xc8\xb4\x2b" + , "\xaa\xca\xe3\xe9\x0c\x7d\xf6\x3f\xc8\x8a\x45\x43\xd1\x69\xac\x5a\xe7\x43\x41\x77\x3c\x33\xf8\x86\xc8\x0c\xdb\x87\x55\x1d\xb2\x94" + ) + , + ( 32 + , 0 + , 304 + , "\x26\x10\xbb\xaa\x1c\xe1\xcd\x5e\x9c\x1a\xe9\xbf\x7d\xda\xdb\xf4" + , "\x63\x92\x47\x6e\x1e\x12\x07\x26\xe6\xfb\x9c\x33\x9d\xc4\xfd\x6c\x94\xd3\x6f\xbd\xa0\x78\x5a\xad\xdf\x05\xa9\x80\xde\xc3\x4b\x1e" + ) + , + ( 32 + , 0 + , 383 + , "\xa3\x47\x90\x62\xb2\x99\x1c\x0f\x0a\x84\x67\x39\x52\x18\x12\x98" + , "\xc4\xe4\x1d\x5d\xd2\x06\x56\x59\x77\x14\x57\x08\x71\x87\xd1\x92\x89\x83\x34\x11\xfa\x00\x24\x79\x11\x8d\x27\x49\x8b\xa7\xb7\xc0" + ) + , + ( 32 + , 0 + , 384 + , "\x15\x2c\x07\x92\x66\x14\x33\x6d\x77\x32\x7a\x60\x87\xfd\x7a\x4c" + , "\x96\x9a\x67\xf1\xb6\xb2\x25\x6a\x72\xd6\x51\xd7\xe7\x72\xe9\x2f\xe3\xf0\x04\x59\x05\x9c\xc0\x93\x12\x17\x39\x2b\xe7\x91\x35\x95" + ) + , + ( 32 + , 0 + , 385 + , "\x21\x39\xf9\x5e\x4c\x91\x46\x9b\x47\x3c\xfa\xcb\x2e\x30\x79\x61" + , "\x78\xa2\x0c\xc8\xee\xc4\x58\xed\x65\x2d\x22\xf7\xee\x40\x2e\x29\xbf\x53\x48\x7f\xa2\xea\x6b\xa6\xd9\x6c\x6a\xc5\x1a\x7d\xe1\xcc" + ) + , + ( 32 + , 0 + , 480 + , "\x51\x41\x4b\x4c\xb6\x29\x27\x8e\xab\xc9\x1b\xb0\xa0\x95\xb3\x58" + , "\x76\x69\x2c\x2b\xd3\x4f\x5d\xb6\x75\xb2\xe6\x25\x7b\x49\xd8\x19\x56\x9d\xc8\x59\xd8\x91\xa6\x00\x37\x92\x50\x78\xac\x7b\x77\x3f" + ) + , + ( 32 + , 0 + , 576 + , "\x56\x6e\x52\x8a\xfd\x1e\xa3\x38\x02\x9f\xbb\xb8\x00\x5d\x28\xe0" + , "\x89\xa0\x91\x43\x01\x0b\xf7\x98\x88\x48\x53\x0b\x72\x0a\xd5\x1f\x47\xfa\x6c\x5c\xb4\xd5\x21\xff\x40\xff\xcd\xe1\x3c\x91\x5c\xaf" + ) + , + ( 32 + , 0 + , 1023 + , "\x56\xb6\x10\xc2\x6a\xb1\xe4\xac\xeb\xa8\xd8\x64\x8f\xfd\x2c\xd0" + , "\x0c\x76\xaf\x87\x66\xcc\x2f\xe0\xcc\x8b\x89\x78\x34\xf2\xc3\x5e\x58\x7f\x13\x06\x33\x48\xca\xd9\x8e\x2b\x31\xf5\x90\x57\x14\xaf" + ) + , + ( 32 + , 0 + , 1024 + , "\x0c\x13\x10\xfa\x4a\xc1\xe1\x44\x06\x61\x8c\x76\xb7\x16\xee\xaa" + , "\x3f\x94\xb6\xd1\xbd\x12\xc6\xe2\xcf\xfc\x08\xd3\x1e\x49\xb0\x82\x6d\xd5\xb8\x17\x1e\xf4\x77\x26\xbf\x1e\x4c\x92\x7e\x6a\xf9\x9d" + ) + , + ( 32 + , 0 + , 1025 + , "\xf1\x2f\xd0\xe5\xe6\x80\x27\x57\x54\xca\x7d\xc3\x0b\x55\x1b\x5f" + , "\x8f\x11\x28\xff\xee\x67\xc7\x29\xb5\x83\x64\x53\x7e\x90\x07\x58\xe3\x7b\x33\xbe\xf5\x60\xfb\x89\x57\xfc\x9c\xea\xbd\x3c\x46\x9e" + ) + , + ( 32 + , 0 + , 4099 + , "\x1a\x12\x1e\x32\xbf\xe4\x3c\xcd\xd5\x10\x72\x5c\x34\xdd\xd4\x80" + , "\xbd\x20\x02\x9b\x6c\xd4\x69\xdd\x67\x25\x9e\xaf\x27\xac\x84\x99\xeb\x57\x51\x59\x54\x88\x56\x85\x73\xee\xc3\x37\x94\xd2\xf8\x4d" + ) + , + ( 32 + , 20 + , 0 + , "\xbe\x05\xa0\xc2\x96\x98\xbe\x3d\xf3\xb8\x08\x22\x52\x98\x1d\x54" + , "\xe3\xb0\xc4\x42\x98\xfc\x1c\x14\x9a\xfb\xf4\xc8\x99\x6f\xb9\x24\x27\xae\x41\xe4\x64\x9b\x93\x4c\xa4\x95\x99\x1b\x78\x52\xb8\x55" + ) + , + ( 32 + , 20 + , 1 + , "\x3b\x22\x82\x62\x05\x1b\xfa\x69\x13\x7f\x18\x3f\xe5\x87\x3f\xf4" + , "\x01\xba\x47\x19\xc8\x0b\x6f\xe9\x11\xb0\x91\xa7\xc0\x51\x24\xb6\x4e\xee\xce\x96\x4e\x09\xc0\x58\xef\x8f\x98\x05\xda\xca\x54\x6b" + ) + , + ( 32 + , 20 + , 15 + , "\x3f\x18\x11\xa8\xcc\xe8\x4d\x9c\x18\x04\x4d\x1f\xba\x8f\xdb\x30" + , "\xb3\xcd\xe8\x23\xf3\xf5\xb3\xa9\x2e\x3d\xe2\x2f\xc6\xd6\x3a\x7f\xf2\x0b\x61\xfe\x89\xd7\x34\xd4\xb1\x24\xb8\x42\x50\x63\xc5\xf5" + ) + , + ( 32 + , 20 + , 16 + , "\x59\x0a\xd8\xc4\x3a\x3c\xe5\xab\x11\xcc\x48\x92\xc2\xf5\x81\x28" + , "\x9b\x8e\x57\x27\x94\x78\xbe\xbc\x9b\xb8\x79\xd1\xf8\xf4\x32\x45\x60\xa4\x1e\xf2\xae\x6d\x06\x33\x2f\x6d\xd7\x28\x14\x19\x12\x53" + ) + , + ( 32 + , 20 + , 17 + , "\x39\xe8\xcc\x85\x85\x90\x88\x3e\x1b\x84\xcf\xbc\x72\xfc\xe1\xca" + , "\x00\x56\xd7\x84\x80\xe4\x6a\x70\xbb\xee\xb2\xf3\xe4\x7e\x2c\x32\x9c\xa1\xba\xb6\xb0\x45\x04\x34\x86\xca\xf8\xe7\x1f\x18\xd0\x8b" + ) + , + ( 32 + , 20 + , 95 + , "\x0e\x30\xb3\x0b\x36\x3c\x2c\x08\xb1\x6a\xbf\xc5\xfc\x90\xcc\xe8" + , "\x1f\x84\x11\xd8\xcb\x11\x28\x70\x33\x0f\x63\x73\xcd\x70\xf4\xa5\x6d\xd4\x53\x57\x25\x96\xb1\x42\x78\xb4\x54\xde\x0b\x37\x4a\x03" + ) + , + ( 32 + , 20 + , 96 + , "\xf1\x9a\x03\xd7\x8c\x86\xb0\x64\xd3\x4e\x88\xf7\xc0\x44\xcb\xf4" + , "\x4c\xad\x61\x9d\x98\x48\xc5\xbc\x0d\x3c\xe1\x5d\xaf\x01\x6a\x70\x85\xe0\xc9\x40\x9e\xb9\x07\xb9\x96\x2b\x26\x46\x11\xae\x9f\x32" + ) + , + ( 32 + , 20 + , 97 + , "\xce\xa0\xd8\xdc\x79\x73\x12\x68\x77\xe6\x40\xdb\x24\x48\x1d\x47" + , "\x5e\x52\xc6\x8b\x0c\xac\xd6\x4b\xaf\x46\x8c\x96\xe5\xc3\x9a\xaa\x5e\x45\x1e\x22\xc0\x09\xb9\x72\x06\x33\xc2\xab\x18\x0d\x03\x85" + ) + , + ( 32 + , 20 + , 112 + , "\xc1\x21\xc4\x7d\x03\xae\xc3\x65\xfd\xde\x5a\x1c\xad\xbe\xf6\x65" + , "\x70\xad\x3d\xad\x9a\x1e\xc4\x5c\x3c\xea\xe1\x95\x15\xb8\x19\xd2\x83\xbd\x01\x62\xe5\xb7\x42\xd8\x45\xdd\xbe\x51\xc9\xca\x85\x0c" + ) + , + ( 32 + , 20 + , 159 + , "\x9b\x6e\xa6\x70\xe7\x96\x66\xe2\x94\x23\xe0\xa6\x2b\xbd\x1c\xb0" + , "\xf8\x6d\xd2\x2a\x72\x18\x2d\xde\x32\xa6\x77\xd9\xbd\x0f\x8f\x07\x32\x68\x00\xa9\xd2\x9c\x2b\x52\x91\x7a\x37\x0f\xe4\xae\x1f\xc7" + ) + , + ( 32 + , 20 + , 160 + , "\x89\x5d\xcf\x91\x6c\x13\x56\x88\xae\x1c\x48\x03\x12\xc7\x96\x66" + , "\x90\xd2\xe7\xf0\x9c\x18\xcd\x98\xb9\x92\xdd\xc2\x02\xd6\x23\x9c\x31\xab\x14\x7d\x32\x38\x0f\x97\x87\x3a\xec\x2a\x59\x2a\x70\x1f" + ) + , + ( 32 + , 20 + , 191 + , "\x06\xa9\x19\x90\xe9\x31\xe1\x3d\x5c\x84\xb4\x6d\xc9\x5a\xad\xdb" + , "\xc7\x39\xa4\xd7\x2f\xe5\x4b\x57\x24\x42\x56\xdf\xc7\x59\x74\x93\x70\x2c\x8a\xea\x19\xe6\x27\x66\xe4\x03\x74\xc2\x29\x23\x72\x2b" + ) + , + ( 32 + , 20 + , 192 + , "\x9c\x93\x8f\x48\x15\xa1\xaf\xdf\x87\x11\xc1\x4a\x31\xfa\x62\xda" + , "\x5c\xdf\x56\xa2\xe4\xe5\x00\xf5\xbe\xce\x4c\x9f\x69\xed\x5d\x6d\x71\x5f\x6d\x96\x16\x5c\x55\x3c\xc1\x2f\xe3\x86\x11\x9d\x8e\x26" + ) + , + ( 32 + , 20 + , 287 + , "\x0e\x81\xdc\x74\x57\xa6\xfc\x5a\x90\x59\x32\x82\x75\xfb\x6e\x37" + , "\xa4\xd8\x0b\x4a\x6a\xc9\xf1\xed\x0d\x47\x1d\x4b\xd9\x7d\xc6\x3d\xae\xcf\x2a\x57\x9b\xeb\xab\x95\x61\x70\x22\x4b\xa7\x58\x28\x71" + ) + , + ( 32 + , 20 + , 288 + , "\xf0\x45\x6e\xb7\xa9\x50\x9b\x0d\x93\x55\x50\xa5\xb1\x87\x98\xd5" + , "\xc4\xf5\x66\x65\x76\xfd\x49\x60\xee\xe9\x1f\xde\x3f\x36\x99\xd8\x5b\xcb\xf9\x4e\xbc\xb8\x31\xb6\xcb\x9e\x7e\x48\x68\xb5\xcb\xed" + ) + , + ( 32 + , 20 + , 289 + , "\xc0\x2b\xf5\xa1\x7f\xd0\x14\xc4\xe1\x81\x80\x63\x1f\x60\x06\x9d" + , "\xaa\xca\xe3\xe9\x0c\x7d\xf6\x3f\xc8\x8a\x45\x43\xd1\x69\xac\x5a\xe7\x43\x41\x77\x3c\x33\xf8\x86\xc8\x0c\xdb\x87\x55\x1d\xb2\x94" + ) + , + ( 32 + , 20 + , 304 + , "\xc2\x63\x59\x97\x5f\xc3\x47\x86\x83\x58\x05\x29\xfa\x72\x69\x42" + , "\x63\x92\x47\x6e\x1e\x12\x07\x26\xe6\xfb\x9c\x33\x9d\xc4\xfd\x6c\x94\xd3\x6f\xbd\xa0\x78\x5a\xad\xdf\x05\xa9\x80\xde\xc3\x4b\x1e" + ) + , + ( 32 + , 20 + , 383 + , "\xd8\x7a\x9c\x06\x53\xbb\xe3\x46\xd7\x84\xaa\x11\xd1\x01\x79\x65" + , "\xc4\xe4\x1d\x5d\xd2\x06\x56\x59\x77\x14\x57\x08\x71\x87\xd1\x92\x89\x83\x34\x11\xfa\x00\x24\x79\x11\x8d\x27\x49\x8b\xa7\xb7\xc0" + ) + , + ( 32 + , 20 + , 384 + , "\x6e\x11\x0b\xf6\x87\x36\xcc\x24\xaa\x32\xb7\x48\x04\xe4\x11\xb1" + , "\x96\x9a\x67\xf1\xb6\xb2\x25\x6a\x72\xd6\x51\xd7\xe7\x72\xe9\x2f\xe3\xf0\x04\x59\x05\x9c\xc0\x93\x12\x17\x39\x2b\xe7\x91\x35\x95" + ) + , + ( 32 + , 20 + , 385 + , "\xf2\xf6\x71\x5c\x6d\xad\xdb\xfa\xc5\x43\x5f\x71\x40\x13\xbe\xb4" + , "\x78\xa2\x0c\xc8\xee\xc4\x58\xed\x65\x2d\x22\xf7\xee\x40\x2e\x29\xbf\x53\x48\x7f\xa2\xea\x6b\xa6\xd9\x6c\x6a\xc5\x1a\x7d\xe1\xcc" + ) + , + ( 32 + , 20 + , 480 + , "\x9f\x29\xb2\x5b\xa9\x99\x2b\xdd\x34\x3a\x39\xc9\xb5\x15\x31\x3e" + , "\x76\x69\x2c\x2b\xd3\x4f\x5d\xb6\x75\xb2\xe6\x25\x7b\x49\xd8\x19\x56\x9d\xc8\x59\xd8\x91\xa6\x00\x37\x92\x50\x78\xac\x7b\x77\x3f" + ) + , + ( 32 + , 20 + , 576 + , "\x42\xfa\x24\x87\x53\xf4\xf6\xf9\x88\x61\x38\x77\x64\x66\x82\x2c" + , "\x89\xa0\x91\x43\x01\x0b\xf7\x98\x88\x48\x53\x0b\x72\x0a\xd5\x1f\x47\xfa\x6c\x5c\xb4\xd5\x21\xff\x40\xff\xcd\xe1\x3c\x91\x5c\xaf" + ) + , + ( 32 + , 20 + , 1023 + , "\x55\x79\xfd\xdf\x43\x31\xe2\xe2\x5e\x05\x18\xe4\xa6\x77\x05\x14" + , "\x0c\x76\xaf\x87\x66\xcc\x2f\xe0\xcc\x8b\x89\x78\x34\xf2\xc3\x5e\x58\x7f\x13\x06\x33\x48\xca\xd9\x8e\x2b\x31\xf5\x90\x57\x14\xaf" + ) + , + ( 32 + , 20 + , 1024 + , "\x0f\xdc\xfd\xe7\x63\x41\xe7\x0a\xb3\xcc\x4c\xf6\x9e\x9c\xc7\x6e" + , "\x3f\x94\xb6\xd1\xbd\x12\xc6\xe2\xcf\xfc\x08\xd3\x1e\x49\xb0\x82\x6d\xd5\xb8\x17\x1e\xf4\x77\x26\xbf\x1e\x4c\x92\x7e\x6a\xf9\x9d" + ) + , + ( 32 + , 20 + , 1025 + , "\xb2\x25\x27\x55\xb1\x57\x78\x93\x73\xa2\x65\x07\x88\x8d\x45\xa0" + , "\x8f\x11\x28\xff\xee\x67\xc7\x29\xb5\x83\x64\x53\x7e\x90\x07\x58\xe3\x7b\x33\xbe\xf5\x60\xfb\x89\x57\xfc\x9c\xea\xbd\x3c\x46\x9e" + ) + , + ( 32 + , 20 + , 4099 + , "\x0a\x93\x20\x79\x0c\x19\xe7\x51\x4b\x61\x4f\x5b\xd8\xf3\x67\x9a" + , "\xbd\x20\x02\x9b\x6c\xd4\x69\xdd\x67\x25\x9e\xaf\x27\xac\x84\x99\xeb\x57\x51\x59\x54\x88\x56\x85\x73\xee\xc3\x37\x94\xd2\xf8\x4d" + ) + ] diff --git a/tests/BlockCipher/AES/OCB3.hs b/tests/BlockCipher/AES/OCB3.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/OCB3.hs @@ -0,0 +1,513 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.AES.OCB3 where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () + +-- (key, iv, aad, input, out, taglen, tag) +type KATOCB3 = + ( B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + , Int + , B.ByteString + ) + +key1 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" +nonce1 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" +key2, nonce2 :: B.ByteString +key2 = "\x0f\x0e\x0d\x0c\x0b\x0a\x09\x08\x07\x06\x05\x04\x03\x02\x01\x00" +nonce2 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0d" +nonce_rfc7253_00 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x00" +nonce_rfc7253_01 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x01" +nonce_rfc7253_02 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x02" +nonce_rfc7253_03 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x03" +nonce_rfc7253_04 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x04" +nonce_rfc7253_05 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x05" +nonce_rfc7253_06 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x06" +nonce_rfc7253_07 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x07" +nonce_rfc7253_08 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x08" +nonce_rfc7253_09 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x09" +nonce_rfc7253_0a = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0a" +nonce_rfc7253_0b = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0b" +nonce_rfc7253_0c = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0c" +nonce_rfc7253_0d = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0d" +nonce_rfc7253_0e = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0e" +nonce_rfc7253_0f = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0f" +nonce_dkg_120_00 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x00" +nonce_dkg_120_01 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x01" +nonce_dkg_120_02 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x02" +nonce_dkg_120_03 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x03" +nonce_dkg_120_04 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x04" +nonce_dkg_120_05 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x05" +nonce_dkg_120_06 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x06" +nonce_dkg_120_07 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x07" +nonce_dkg_120_08 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x08" +nonce_dkg_120_09 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x09" +nonce_dkg_120_0a = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0a" +nonce_dkg_120_0b = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0b" +nonce_dkg_120_0c = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0c" +nonce_dkg_120_0d = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0d" +nonce_dkg_120_0e = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0e" +nonce_dkg_120_0f = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0f" + +bytes8 = "\x00\x01\x02\x03\x04\x05\x06\x07" +bytes16 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" +bytes24 = + "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17" +bytes32 = + "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" +bytes40 = + "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + +vectors_aes128_enc :: [KATOCB3] +vectors_aes128_enc = + [ + ( {-key = -} key1 + , {-iv = -} nonce1 + , {-aad = -} "" + , {-input = -} "" + , {-out = -} "" + , {-taglen = -} 16 + , {-tag = -} "\x19\x7b\x9c\x3c\x44\x1d\x3c\x83\xea\xfb\x2b\xef\x63\x3b\x91\x82" + ) + , + ( key1 + , nonce1 + , "\x00\x01\x02\x03\x04\x05\x06\x07" + , "\x00\x01\x02\x03\x04\x05\x06\x07" + , "\x92\xb6\x57\x13\x0a\x74\xb8\x5a" + , 16 + , "\x16\xdc\x76\xa4\x6d\x47\xe1\xea\xd5\x37\x20\x9e\x8a\x96\xd1\x4e" + ) + , + ( key1 + , nonce1 + , "\x00\x01\x02\x03\x04\x05\x06\x07" + , "" + , "" + , 16 + , "\x98\xb9\x15\x52\xc8\xc0\x09\x18\x50\x44\xe3\x0a\x6e\xb2\xfe\x21" + ) + , + ( key1 + , nonce1 + , "" + , "\x00\x01\x02\x03\x04\x05\x06\x07" + , "\x92\xb6\x57\x13\x0a\x74\xb8\x5a" + , 16 + , "\x97\x1e\xff\xca\xe1\x9a\xd4\x71\x6f\x88\xe8\x7b\x87\x1f\xbe\xed" + ) + , + ( key1 + , nonce1 + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" + , "\xbe\xa5\xe8\x79\x8d\xbe\x71\x10\x03\x1c\x14\x4d\xa0\xb2\x61\x22" + , 16 + , "\x77\x6c\x99\x24\xd6\x72\x3a\x1f\xc4\x52\x45\x32\xac\x3e\x5b\xeb" + ) + {- Disabled: 96-bit tag vector + , ( key2 + , nonce2 + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\x17\x92\xa4\xe3\x1e\x07\x55\xfb\x03\xe3\x1b\x22\x11\x6e\x6c\x2d\xdf\x9e\xfd\x6e\x33\xd5\x36\xf1\xa0\x12\x4b\x0a\x55\xba\xe8\x84\xed\x93\x48\x15\x29\xc7\x6b\x6a" + , 12 + , "\xd0\xc5\x15\xf4\xd1\xcd\xd4\xfd\xac\x4f\x02\xaa" + ) + -} + ] + ++ vectors_rfc7253_aes128_tag128 + ++ vectors_dkg_nonce120_aes128 + +-- From OpenSSL 3.5: the suite had OCB vectors at 128 bits only, and the +-- mode runs a block at a time through whatever the key size dispatches to. +vectors_aes192_enc :: [KATOCB3] +vectors_aes192_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "" + , "" + , "" + , 16 + , "\x6a\xaf\x71\x1d\xef\xf8\xc9\x45\xa0\xf6\x19\x79\x56\x53\x44\x5c" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c" + , "\xd2\xe9\x10\x26\xf2\x07\xc8\xd9\x21\x36\x43\xc4\xdc\x32\xa2\xaa" + , 16 + , "\x4c\x79\xf5\x7b\x3b\x99\xd4\xb0\x7e\x41\xe9\x4e\x0b\xb2\x54\xda" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0" + , "\xd2\xe9\x10\x26\xf2\x07\xc8\xd9\x21\x36\x43\xc4\xdc\x32\xa2\xaa\xcf\x50\xd3\xd3\x40\x79\xef\xe6\x91\xcc\x4d\xd0\x5c\x33\x3f\x2e\x36\xe2\x00\x0f\x2a\x50\xa5\xcc\xf2\x46\x39\x4f\x36\xee\x2b\x22\xa5\x3e\x8a\xb2\x0b\xa1\x1b\x33\x94\xca\x61\x2e" + , 16 + , "\x89\x79\x7c\xc8\xae\xdc\x11\xbc\x16\x90\xfa\x80\x9a\xb7\xb4\xad" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec\xf3\xfa" + , "\xd2\xe9\x10\x26\xf2\x07\xc8\xd9\x21\x36\x43\xc4\xdc\x32\xa2\xaa\xcf\x50\xd3\xd3\x40\x79\xef\xe6\x91\xcc\x4d\xd0\x5c\x33\x3f\x2e\x36\xe2\x00\x0f\x2a\x50\xa5\xcc\xf2\x46\x39\x4f\x36\xee\x2b\x22\xd2\xd6\x45\x6c\x07\x54\xd0\xde\x9a\x0e\xf5\x79\x34\xa2\x51\xa1\x1b\xe0\xdd\x47\xe5\x26\x81\x03\xdb\x20\x17\x0c\x4c\x77\x1a\x96\x87\x9a\x4d\xe4\x06\x07\xd6\x25\xdd\x85\x7d\xf4\x05\x42\x74\xc3\xca\x9d\x85\x7a\xe1\x9d\xfc\xab\xe2\x35\x8a\x53\xd6\x9a\x0d\x14\x6d\x81\xb2\x05\xbe\x28\x68\xf0\xa6\xbc\xfb\xb4\x61\x6d\xe2\x2c\xdd\xfb\x3f\xf6\xb3\x49\x45\xb2\xfd\xe6\x60\x55\x9f\xd3\x89\xee\xee\x6e" + , 16 + , "\x9e\x65\x4d\xe8\x97\x81\x6a\x00\x20\x11\xd0\x89\xdf\x6e\x1c\xfc" + ) + ] + +vectors_aes256_enc :: [KATOCB3] +vectors_aes256_enc = + [ + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "" + , "" + , "" + , 16 + , "\x96\x46\xd7\x6e\x09\xaf\x82\x36\x7c\x05\xe8\x37\x26\x9c\x28\xa2" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c" + , "\xb6\xd6\x04\x43\x79\x1b\x06\xba\xbb\x66\xe9\xda\x14\xbb\x4b\x07" + , 16 + , "\x92\x64\x4d\x1a\x5c\x28\x39\x79\x17\x14\x7f\x97\x66\xf9\x36\x94" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0" + , "\xb6\xd6\x04\x43\x79\x1b\x06\xba\xbb\x66\xe9\xda\x14\xbb\x4b\x07\xb4\xa7\xea\x1a\xc1\x12\x12\x4e\xcc\x81\x3f\x3b\x89\x02\xaa\xf5\xaa\x21\xdf\xa5\xe1\x1a\xe9\x73\x11\x4e\x89\x20\xf6\x32\xfa\xdb\x25\x68\x31\x2d\x7c\x2c\x68\x0c\xb9\x7f\x6d\xc5" + , 16 + , "\x09\x35\x78\xfd\x9d\xdb\xc6\x62\x1d\x54\x52\x80\x6d\xe7\x46\x6b" + ) + , + ( "\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f" + , "\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab" + , "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c" + , "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc\xe3\xea\xf1\xf8\xff\x06\x0d\x14\x1b\x22\x29\x30\x37\x3e\x45\x4c\x53\x5a\x61\x68\x6f\x76\x7d\x84\x8b\x92\x99\xa0\xa7\xae\xb5\xbc\xc3\xca\xd1\xd8\xdf\xe6\xed\xf4\xfb\x02\x09\x10\x17\x1e\x25\x2c\x33\x3a\x41\x48\x4f\x56\x5d\x64\x6b\x72\x79\x80\x87\x8e\x95\x9c\xa3\xaa\xb1\xb8\xbf\xc6\xcd\xd4\xdb\xe2\xe9\xf0\xf7\xfe\x05\x0c\x13\x1a\x21\x28\x2f\x36\x3d\x44\x4b\x52\x59\x60\x67\x6e\x75\x7c\x83\x8a\x91\x98\x9f\xa6\xad\xb4\xbb\xc2\xc9\xd0\xd7\xde\xe5\xec\xf3\xfa" + , "\xb6\xd6\x04\x43\x79\x1b\x06\xba\xbb\x66\xe9\xda\x14\xbb\x4b\x07\xb4\xa7\xea\x1a\xc1\x12\x12\x4e\xcc\x81\x3f\x3b\x89\x02\xaa\xf5\xaa\x21\xdf\xa5\xe1\x1a\xe9\x73\x11\x4e\x89\x20\xf6\x32\xfa\xdb\x96\xe1\x52\x26\xb5\x0c\xc7\xf4\xce\x6c\x77\xd9\x3f\xbb\x7d\x37\x3d\xb2\xa0\xe3\x48\x2a\xff\xab\xac\x92\x32\x1b\xde\xe6\x74\xd4\x15\xdf\xe7\x71\x8f\x32\x0c\xcf\x83\x37\x5f\xd2\xf0\x6b\xfa\xa6\x7a\x98\x5b\x64\xb8\x57\x48\x8d\x31\xa0\x1b\x41\xd2\x4f\x59\x98\xbf\xa0\x49\xb6\xd9\x7a\xf6\xb8\x20\xef\x85\x9c\x4c\xd8\xb8\xe6\x14\x0e\x46\x24\x6f\x19\x7c\x88\x4c\x02\x20\x25\x3c\x89\x73\xa4\x2c\xd0" + , 16 + , "\x12\xd9\x0f\x54\x89\xcf\x64\xcc\x21\xe9\x4a\x6e\x9c\xf7\x0b\x31" + ) + ] + +vectors_rfc7253_aes128_tag128 :: [KATOCB3] +vectors_rfc7253_aes128_tag128 = + [ + ( key1 + , nonce_rfc7253_00 + , "" + , "" + , "" + , 16 + , "\x78\x54\x07\xbf\xff\xc8\xad\x9e\xdc\xc5\x52\x0a\xc9\x11\x1e\xe6" + ) + , + ( key1 + , nonce_rfc7253_01 + , bytes8 + , bytes8 + , "\x68\x20\xb3\x65\x7b\x6f\x61\x5a" + , 16 + , "\x57\x25\xbd\xa0\xd3\xb4\xeb\x3a\x25\x7c\x9a\xf1\xf8\xf0\x30\x09" + ) + , + ( key1 + , nonce_rfc7253_02 + , bytes8 + , "" + , "" + , 16 + , "\x81\x01\x7f\x82\x03\xf0\x81\x27\x71\x52\xfa\xde\x69\x4a\x0a\x00" + ) + , + ( key1 + , nonce_rfc7253_03 + , "" + , bytes8 + , "\x45\xdd\x69\xf8\xf5\xaa\xe7\x24" + , 16 + , "\x14\x05\x4c\xd1\xf3\x5d\x82\x76\x0b\x2c\xd0\x0d\x2f\x99\xbf\xa9" + ) + , + ( key1 + , nonce_rfc7253_04 + , bytes16 + , bytes16 + , "\x57\x1d\x53\x5b\x60\xb2\x77\x18\x8b\xe5\x14\x71\x70\xa9\xa2\x2c" + , 16 + , "\x3a\xd7\xa4\xff\x38\x35\xb8\xc5\x70\x1c\x1c\xce\xc8\xfc\x33\x58" + ) + , + ( key1 + , nonce_rfc7253_05 + , bytes16 + , "" + , "" + , 16 + , "\x8c\xf7\x61\xb6\x90\x2e\xf7\x64\x46\x2a\xd8\x64\x98\xca\x6b\x97" + ) + , + ( key1 + , nonce_rfc7253_06 + , "" + , bytes16 + , "\x5c\xe8\x8e\xc2\xe0\x69\x27\x06\xa9\x15\xc0\x0a\xeb\x8b\x23\x96" + , 16 + , "\xf4\x0e\x1c\x74\x3f\x52\x43\x6b\xdf\x06\xd8\xfa\x1e\xca\x34\x3d" + ) + , + ( key1 + , nonce_rfc7253_07 + , bytes24 + , bytes24 + , "\x1c\xa2\x20\x73\x08\xc8\x7c\x01\x07\x56\x10\x4d\x88\x40\xce\x19\x52\xf0\x96\x73\xa4\x48\xa1\x22" + , 16 + , "\xc9\x2c\x62\x24\x10\x51\xf5\x73\x56\xd7\xf3\xc9\x0b\xb0\xe0\x7f" + ) + , + ( key1 + , nonce_rfc7253_08 + , bytes24 + , "" + , "" + , 16 + , "\x6d\xc2\x25\xa0\x71\xfc\x1b\x9f\x7c\x69\xf9\x3b\x0f\x1e\x10\xde" + ) + , + ( key1 + , nonce_rfc7253_09 + , "" + , bytes24 + , "\x22\x1b\xd0\xde\x7f\xa6\xfe\x99\x3e\xcc\xd7\x69\x46\x0a\x0a\xf2\xd6\xcd\xed\x0c\x39\x5b\x1c\x3c" + , 16 + , "\xe7\x25\xf3\x24\x94\xb9\xf9\x14\xd8\x5c\x0b\x1e\xb3\x83\x57\xff" + ) + , + ( key1 + , nonce_rfc7253_0a + , bytes32 + , bytes32 + , "\xbd\x6f\x6c\x49\x62\x01\xc6\x92\x96\xc1\x1e\xfd\x13\x8a\x46\x7a\xbd\x3c\x70\x79\x24\xb9\x64\xde\xaf\xfc\x40\x31\x9a\xf5\xa4\x85" + , 16 + , "\x40\xfb\xba\x18\x6c\x55\x53\xc6\x8a\xd9\xf5\x92\xa7\x9a\x42\x40" + ) + , + ( key1 + , nonce_rfc7253_0b + , bytes32 + , "" + , "" + , 16 + , "\xfe\x80\x69\x0b\xee\x8a\x48\x5d\x11\xf3\x29\x65\xbc\x9d\x2a\x32" + ) + , + ( key1 + , nonce_rfc7253_0c + , "" + , bytes32 + , "\x29\x42\xbf\xc7\x73\xbd\xa2\x3c\xab\xc6\xac\xfd\x9b\xfd\x58\x35\xbd\x30\x0f\x09\x73\x79\x2e\xf4\x60\x40\xc5\x3f\x14\x32\xbc\xdf" + , 16 + , "\xb5\xe1\xdd\xe3\xbc\x18\xa5\xf8\x40\xb5\x2e\x65\x34\x44\xd5\xdf" + ) + , + ( key1 + , nonce_rfc7253_0d + , bytes40 + , bytes40 + , "\xd5\xca\x91\x74\x84\x10\xc1\x75\x1f\xf8\xa2\xf6\x18\x25\x5b\x68\xa0\xa1\x2e\x09\x3f\xf4\x54\x60\x6e\x59\xf9\xc1\xd0\xdd\xc5\x4b\x65\xe8\x62\x8e\x56\x8b\xad\x7a" + , 16 + , "\xed\x07\xba\x06\xa4\xa6\x94\x83\xa7\x03\x54\x90\xc5\x76\x9e\x60" + ) + , + ( key1 + , nonce_rfc7253_0e + , bytes40 + , "" + , "" + , 16 + , "\xc5\xcd\x9d\x18\x50\xc1\x41\xe3\x58\x64\x99\x94\xee\x70\x1b\x68" + ) + , + ( key1 + , nonce_rfc7253_0f + , "" + , bytes40 + , "\x44\x12\x92\x34\x93\xc5\x7d\x5d\xe0\xd7\x00\xf7\x53\xcc\xe0\xd1\xd2\xd9\x50\x60\x12\x2e\x9f\x15\xa5\xdd\xbf\xc5\x78\x7e\x50\xb5\xcc\x55\xee\x50\x7b\xcb\x08\x4e" + , 16 + , "\x47\x9a\xd3\x63\xac\x36\x6b\x95\xa9\x8c\xa5\xf3\x00\x0b\x14\x79" + ) + ] + +vectors_dkg_nonce120_aes128 :: [KATOCB3] +vectors_dkg_nonce120_aes128 = + [ + ( key1 + , nonce_dkg_120_00 + , "" + , "" + , "" + , 16 + , "\x75\x2a\xcd\x21\x32\xc4\x1e\x02\x0e\x41\xfb\x22\x3e\xfd\x77\xb6" + ) + , + ( key1 + , nonce_dkg_120_01 + , bytes8 + , bytes8 + , "\x20\x1f\xe4\xd8\x9e\xa7\xbd\x1e" + , 16 + , "\xb5\xb1\x57\x7d\xb1\x62\x83\xb8\xae\xd1\x71\x5a\xd6\xbe\x51\x49" + ) + , + ( key1 + , nonce_dkg_120_02 + , bytes8 + , "" + , "" + , 16 + , "\x71\x09\x60\xb9\xee\x00\xb8\xf4\x4d\x2e\x81\x20\xaa\xba\x63\xae" + ) + , + ( key1 + , nonce_dkg_120_03 + , "" + , bytes8 + , "\x08\x4e\x86\x95\x70\x19\x4b\xd2" + , 16 + , "\x50\x32\xfe\x9e\x53\x28\xe4\x5d\x50\x7e\x74\xf3\x36\x6e\x20\xd2" + ) + , + ( key1 + , nonce_dkg_120_04 + , bytes16 + , bytes16 + , "\x96\x76\xee\x37\xfd\x64\x5c\x07\xc0\xd4\xf7\x0a\xab\xf6\x86\x68" + , 16 + , "\x8e\x39\xb2\xfb\x3f\xc4\xff\x30\xdc\xd1\x82\x7b\x36\xa2\x98\xd3" + ) + , + ( key1 + , nonce_dkg_120_05 + , bytes16 + , "" + , "" + , 16 + , "\x9d\x51\x0f\x56\xed\xf7\x2f\xfa\x34\x96\x9b\xce\xf9\x1e\x6d\xe9" + ) + , + ( key1 + , nonce_dkg_120_06 + , "" + , bytes16 + , "\xd5\xe1\x5a\xa1\xd2\x32\xab\x57\xf2\x34\x36\x6d\xff\xb2\x55\x74" + , 16 + , "\xa3\x63\x6a\x5f\x3e\x34\x33\xea\x45\x90\xcb\xf4\xf9\xac\x1f\x4d" + ) + , + ( key1 + , nonce_dkg_120_07 + , bytes24 + , bytes24 + , "\x1c\x4b\x67\x77\xb7\xf1\x37\xc3\x09\x71\xa9\x3d\xe3\xc5\x6c\xc7\x35\x68\x6a\x6f\x77\x03\x14\x2f" + , 16 + , "\xab\x8a\xcc\x98\x7c\x14\x06\xdf\xf9\x62\x73\xc5\x37\x6e\x62\x10" + ) + , + ( key1 + , nonce_dkg_120_08 + , bytes24 + , "" + , "" + , 16 + , "\x96\xe6\x70\xc0\x23\x8f\xb9\x69\xb7\xac\xe4\xab\xaf\x74\x38\xc7" + ) + , + ( key1 + , nonce_dkg_120_09 + , "" + , bytes24 + , "\x12\x90\xa6\x86\xd8\x25\xf7\x12\xe5\x94\xbe\x40\x39\xc0\x4d\x3e\x44\xf7\xd1\x34\x2b\x84\xff\xca" + , 16 + , "\xd6\x8b\xbd\xfa\x04\xb5\x80\xea\x9a\x01\xe2\xf4\x56\x53\x99\xc3" + ) + , + ( key1 + , nonce_dkg_120_0a + , bytes32 + , bytes32 + , "\xfb\xdf\xc1\x1f\x74\x92\x17\xbb\x7f\xae\x5d\x40\x36\xb8\xf2\x28\x03\x71\x2e\xff\x9e\xf9\x43\x42\xfe\x1b\x68\x49\x68\xd0\xe3\xe3" + , 16 + , "\x81\xa2\x77\xda\xab\x83\x57\x94\x06\xa0\x1e\x26\x75\xa0\x82\xc9" + ) + , + ( key1 + , nonce_dkg_120_0b + , bytes32 + , "" + , "" + , 16 + , "\x90\xcd\xa8\xa0\x51\x61\xd2\x87\x33\x61\x37\x4b\x76\xf9\x54\x30" + ) + , + ( key1 + , nonce_dkg_120_0c + , "" + , bytes32 + , "\xd1\x32\x0a\xf4\xb6\xff\x8a\xfe\xec\xee\x79\x21\x39\x5d\x4e\x86\x92\x71\x77\x53\xee\x15\xf5\x03\x8e\xb6\x74\xda\x43\xd6\xea\x8d" + , 16 + , "\xbe\x78\x31\xe7\x23\xbe\x47\x1f\x62\xd9\xe7\xf4\x9a\x7d\x3b\x32" + ) + , + ( key1 + , nonce_dkg_120_0d + , bytes40 + , bytes40 + , "\x5c\x79\xf1\xc4\xb9\xa2\x04\xed\x33\x23\x61\x6d\x57\x6f\xc5\x00\xe4\xa7\x19\x39\xf0\x3a\x3c\x3d\xe2\xc0\x97\xaf\x2c\x6c\x81\xdc\x3f\x03\x09\xe7\x60\x82\xb1\xf5" + , 16 + , "\x0f\xf8\x52\x29\x59\xff\xe4\x1f\x37\xef\x50\x7e\x90\x76\xd3\x2c" + ) + , + ( key1 + , nonce_dkg_120_0e + , bytes40 + , "" + , "" + , 16 + , "\x3b\xf1\x58\xb7\xde\x76\xc5\x15\x1e\xf6\x08\x6a\x82\x5d\x0c\xc4" + ) + , + ( key1 + , nonce_dkg_120_0f + , "" + , bytes40 + , "\x34\xda\x59\xd2\xeb\x08\xf4\x78\x22\xd4\x8c\x85\xb6\xa1\xd2\x36\x94\xe1\xd3\xde\x68\x0d\x61\x6d\x7b\x1b\x59\x47\x2c\x13\xe3\x69\xc6\x8d\xca\x69\x9d\xa1\x68\x6a" + , 16 + , "\x33\x9d\x54\x52\x80\x36\x32\x81\x0b\x08\x40\xe6\x80\x4a\xb0\x20" + ) + {- Disabled: 96-bit tag vector + , ( key2 + , nonce_dkg_120_0d + , bytes40 + , bytes40 + , "\x07\xe9\x03\xbf\xc4\x95\x52\x41\x1a\xbc\x86\x5f\x5e\xce\x60\xf6\xfa\xd1\xf5\xa9\xf1\x4d\x30\x70\xfa\x2f\x13\x08\xa5\x63\x20\x7f\xfe\x14\xc1\xee\xa4\x4b\x22\x05" + , 12 + , "\x9c\x74\x84\x31\x9d\x8a\x2c\x53\xc2\x36\xa7\xb3" + ) + -} + ] diff --git a/tests/BlockCipher/AES/XTS.hs b/tests/BlockCipher/AES/XTS.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AES/XTS.hs @@ -0,0 +1,68 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.AES.XTS where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () + +type KATXTS = + ( B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + , B.ByteString + ) + +vectors_aes128_enc + , vectors_aes128_dec + , vectors_aes256_enc + , vectors_aes256_dec + :: [KATXTS] +vectors_aes128_enc = + [ + ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x66\xe9\x4b\xd4\xef\x8a\x2c\x3b\x88\x4c\xfa\x59\xca\x34\x2b\x2e\xcc\xd2\x97\xa8\xdf\x15\x59\x76\x10\x99\xf4\xb3\x94\x69\x56\x5c" + , "\x91\x7c\xf6\x9e\xbd\x68\xb2\xec\x9b\x9f\xe9\xa3\xea\xdd\xa6\x92\xcd\x43\xd2\xf5\x95\x98\xed\x85\x8c\x02\xc2\x65\x2f\xbf\x92\x2e" + ) + , + ( "\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11" + , "\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22" + , "\x33\x33\x33\x33\x33\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44" + , "\x3f\x80\x3b\xcd\x0d\x7f\xd2\xb3\x75\x58\x41\x9f\x59\xd5\xcd\xa6\xf9\x00\x77\x9a\x1b\xfe\xa4\x67\xeb\xb0\x82\x3e\xb3\xaa\x9b\x4d" + , "\xc4\x54\x18\x5e\x6a\x16\x93\x6e\x39\x33\x40\x38\xac\xef\x83\x8b\xfb\x18\x6f\xff\x74\x80\xad\xc4\x28\x93\x82\xec\xd6\xd3\x94\xf0" + ) + , + ( "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0" + , "\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22" + , "\x33\x33\x33\x33\x33\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44" + , "\x3f\x80\x3b\xcd\x0d\x7f\xd2\xb3\x75\x58\x41\x9f\x59\xd5\xcd\xa6\xf9\x00\x77\x9a\x1b\xfe\xa4\x67\xeb\xb0\x82\x3e\xb3\xaa\x9b\x4d" + , "\xaf\x85\x33\x6b\x59\x7a\xfc\x1a\x90\x0b\x2e\xb2\x1e\xc9\x49\xd2\x92\xdf\x4c\x04\x7e\x0b\x21\x53\x21\x86\xa5\x97\x1a\x22\x7a\x89" + ) + , + ( "\x27\x18\x28\x18\x28\x45\x90\x45\x23\x53\x60\x28\x74\x71\x35\x26" + , "\x31\x41\x59\x26\x53\x58\x97\x93\x23\x84\x62\x64\x33\x83\x27\x95" + , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff" + , "" + , "\x27\xa7\x47\x9b\xef\xa1\xd4\x76\x48\x9f\x30\x8c\xd4\xcf\xa6\xe2\xa9\x6e\x4b\xbe\x32\x08\xff\x25\x28\x7d\xd3\x81\x96\x16\xe8\x9c\xc7\x8c\xf7\xf5\xe5\x43\x44\x5f\x83\x33\xd8\xfa\x7f\x56\x00\x00\x05\x27\x9f\xa5\xd8\xb5\xe4\xad\x40\xe7\x36\xdd\xb4\xd3\x54\x12\x32\x80\x63\xfd\x2a\xab\x53\xe5\xea\x1e\x0a\x9f\x33\x25\x00\xa5\xdf\x94\x87\xd0\x7a\x5c\x92\xcc\x51\x2c\x88\x66\xc7\xe8\x60\xce\x93\xfd\xf1\x66\xa2\x49\x12\xb4\x22\x97\x61\x46\xae\x20\xce\x84\x6b\xb7\xdc\x9b\xa9\x4a\x76\x7a\xae\xf2\x0c\x0d\x61\xad\x02\x65\x5e\xa9\x2d\xc4\xc4\xe4\x1a\x89\x52\xc6\x51\xd3\x31\x74\xbe\x51\xa1\x0c\x42\x11\x10\xe6\xd8\x15\x88\xed\xe8\x21\x03\xa2\x52\xd8\xa7\x50\xe8\x76\x8d\xef\xff\xed\x91\x22\x81\x0a\xae\xb9\x9f\x91\x72\xaf\x82\xb6\x04\xdc\x4b\x8e\x51\xbc\xb0\x82\x35\xa6\xf4\x34\x13\x32\xe4\xca\x60\x48\x2a\x4b\xa1\xa0\x3b\x3e\x65\x00\x8f\xc5\xda\x76\xb7\x0b\xf1\x69\x0d\xb4\xea\xe2\x9c\x5f\x1b\xad\xd0\x3c\x5c\xcf\x2a\x55\xd7\x05\xdd\xcd\x86\xd4\x49\x51\x1c\xeb\x7e\xc3\x0b\xf1\x2b\x1f\xa3\x5b\x91\x3f\x9f\x74\x7a\x8a\xfd\x1b\x13\x0e\x94\xbf\xf9\x4e\xff\xd0\x1a\x91\x73\x5c\xa1\x72\x6a\xcd\x0b\x19\x7c\x4e\x5b\x03\x39\x36\x97\xe1\x26\x82\x6f\xb6\xbb\xde\x8e\xcc\x1e\x08\x29\x85\x16\xe2\xc9\xed\x03\xff\x3c\x1b\x78\x60\xf6\xde\x76\xd4\xce\xcd\x94\xc8\x11\x98\x55\xef\x52\x97\xca\x67\xe9\xf3\xe7\xff\x72\xb1\xe9\x97\x85\xca\x0a\x7e\x77\x20\xc5\xb3\x6d\xc6\xd7\x2c\xac\x95\x74\xc8\xcb\xbc\x2f\x80\x1e\x23\xe5\x6f\xd3\x44\xb0\x7f\x22\x15\x4b\xeb\xa0\xf0\x8c\xe8\x89\x1e\x64\x3e\xd9\x95\xc9\x4d\x9a\x69\xc9\xf1\xb5\xf4\x99\x02\x7a\x78\x57\x2a\xee\xbd\x74\xd2\x0c\xc3\x98\x81\xc2\x13\xee\x77\x0b\x10\x10\xe4\xbe\xa7\x18\x84\x69\x77\xae\x11\x9f\x7a\x02\x3a\xb5\x8c\xca\x0a\xd7\x52\xaf\xe6\x56\xbb\x3c\x17\x25\x6a\x9f\x6e\x9b\xf1\x9f\xdd\x5a\x38\xfc\x82\xbb\xe8\x72\xc5\x53\x9e\xdb\x60\x9e\xf4\xf7\x9c\x20\x3e\xbb\x14\x0f\x2e\x58\x3c\xb2\xad\x15\xb4\xaa\x5b\x65\x50\x16\xa8\x44\x92\x77\xdb\xd4\x77\xef\x2c\x8d\x6c\x01\x7d\xb7\x38\xb1\x8d\xeb\x4a\x42\x7d\x19\x23\xce\x3f\xf2\x62\x73\x57\x79\xa4\x18\xf2\x0a\x28\x2d\xf9\x20\x14\x7b\xea\xbe\x42\x1e\xe5\x31\x9d\x05\x68" + ) + ] +vectors_aes128_dec = + [] +vectors_aes256_enc = + [ + ( "\x27\x18\x28\x18\x28\x45\x90\x45\x23\x53\x60\x28\x74\x71\x35\x26\x62\x49\x77\x57\x24\x70\x93\x69\x99\x59\x57\x49\x66\x96\x76\x27" + , "\x31\x41\x59\x26\x53\x58\x97\x93\x23\x84\x62\x64\x33\x83\x27\x95\x02\x88\x41\x97\x16\x93\x99\x37\x51\x05\x82\x09\x74\x94\x45\x92" + , "\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff" + , "" + , "\x1c\x3b\x3a\x10\x2f\x77\x03\x86\xe4\x83\x6c\x99\xe3\x70\xcf\x9b\xea\x00\x80\x3f\x5e\x48\x23\x57\xa4\xae\x12\xd4\x14\xa3\xe6\x3b\x5d\x31\xe2\x76\xf8\xfe\x4a\x8d\x66\xb3\x17\xf9\xac\x68\x3f\x44\x68\x0a\x86\xac\x35\xad\xfc\x33\x45\xbe\xfe\xcb\x4b\xb1\x88\xfd\x57\x76\x92\x6c\x49\xa3\x09\x5e\xb1\x08\xfd\x10\x98\xba\xec\x70\xaa\xa6\x69\x99\xa7\x2a\x82\xf2\x7d\x84\x8b\x21\xd4\xa7\x41\xb0\xc5\xcd\x4d\x5f\xff\x9d\xac\x89\xae\xba\x12\x29\x61\xd0\x3a\x75\x71\x23\xe9\x87\x0f\x8a\xcf\x10\x00\x02\x08\x87\x89\x14\x29\xca\x2a\x3e\x7a\x7d\x7d\xf7\xb1\x03\x55\x16\x5c\x8b\x9a\x6d\x0a\x7d\xe8\xb0\x62\xc4\x50\x0d\xc4\xcd\x12\x0c\x0f\x74\x18\xda\xe3\xd0\xb5\x78\x1c\x34\x80\x3f\xa7\x54\x21\xc7\x90\xdf\xe1\xde\x18\x34\xf2\x80\xd7\x66\x7b\x32\x7f\x6c\x8c\xd7\x55\x7e\x12\xac\x3a\x0f\x93\xec\x05\xc5\x2e\x04\x93\xef\x31\xa1\x2d\x3d\x92\x60\xf7\x9a\x28\x9d\x6a\x37\x9b\xc7\x0c\x50\x84\x14\x73\xd1\xa8\xcc\x81\xec\x58\x3e\x96\x45\xe0\x7b\x8d\x96\x70\x65\x5b\xa5\xbb\xcf\xec\xc6\xdc\x39\x66\x38\x0a\xd8\xfe\xcb\x17\xb6\xba\x02\x46\x9a\x02\x0a\x84\xe1\x8e\x8f\x84\x25\x20\x70\xc1\x3e\x9f\x1f\x28\x9b\xe5\x4f\xbc\x48\x14\x57\x77\x8f\x61\x60\x15\xe1\x32\x7a\x02\xb1\x40\xf1\x50\x5e\xb3\x09\x32\x6d\x68\x37\x8f\x83\x74\x59\x5c\x84\x9d\x84\xf4\xc3\x33\xec\x44\x23\x88\x51\x43\xcb\x47\xbd\x71\xc5\xed\xae\x9b\xe6\x9a\x2f\xfe\xce\xb1\xbe\xc9\xde\x24\x4f\xbe\x15\x99\x2b\x11\xb7\x7c\x04\x0f\x12\xbd\x8f\x6a\x97\x5a\x44\xa0\xf9\x0c\x29\xa9\xab\xc3\xd4\xd8\x93\x92\x72\x84\xc5\x87\x54\xcc\xe2\x94\x52\x9f\x86\x14\xdc\xd2\xab\xa9\x91\x92\x5f\xed\xc4\xae\x74\xff\xac\x6e\x33\x3b\x93\xeb\x4a\xff\x04\x79\xda\x9a\x41\x0e\x44\x50\xe0\xdd\x7a\xe4\xc6\xe2\x91\x09\x00\x57\x5d\xa4\x01\xfc\x07\x05\x9f\x64\x5e\x8b\x7e\x9b\xfd\xef\x33\x94\x30\x54\xff\x84\x01\x14\x93\xc2\x7b\x34\x29\xea\xed\xb4\xed\x53\x76\x44\x1a\x77\xed\x43\x85\x1a\xd7\x7f\x16\xf5\x41\xdf\xd2\x69\xd5\x0d\x6a\x5f\x14\xfb\x0a\xab\x1c\xbb\x4c\x15\x50\xbe\x97\xf7\xab\x40\x66\x19\x3c\x4c\xaa\x77\x3d\xad\x38\x01\x4b\xd2\x09\x2f\xa7\x55\xc8\x24\xbb\x5e\x54\xc4\xf3\x6f\xfd\xa9\xfc\xea\x70\xb9\xc6\xe6\x93\xe1\x48\xc1\x51" + ) + ] +vectors_aes256_dec = [] diff --git a/tests/BlockCipher/AESGCMSIVSpec.hs b/tests/BlockCipher/AESGCMSIVSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AESGCMSIVSpec.hs @@ -0,0 +1,665 @@ +{-# LANGUAGE FlexibleInstances #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE Rank2Types #-} +{-# LANGUAGE RecordWildCards #-} + +module BlockCipher.AESGCMSIVSpec (spec) where + +import Imports + +import qualified Data.ByteArray as B +import Data.Proxy + +import Crypto.Cipher.AES +import Crypto.Cipher.AESGCMSIV +import Crypto.Cipher.Types +import Crypto.Error + +data Vector c = Vector + { vecPlaintext :: ByteString + , vecAAD :: ByteString + , vecKey :: ByteString + , vecNonce :: ByteString + , vecTag :: ByteString + , vecCiphertext :: ByteString + } + +vecCipher :: Cipher c => Vector c -> c +vecCipher = throwCryptoError . cipherInit . vecKey + +vectors128 :: [Vector AES128] +vectors128 = + [ Vector + { vecPlaintext = "" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xdc\x20\xe2\xd8\x3f\x25\x70\x5b\xb4\x9e\x43\x9e\xca\x56\xde\x25" + , vecCiphertext = "" + } + , Vector + { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x57\x87\x82\xff\xf6\x01\x3b\x81\x5b\x28\x7c\x22\x49\x3a\x36\x4c" + , vecCiphertext = "\xb5\xd8\x39\x33\x0a\xc7\xb7\x86" + } + , Vector + { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xa4\x97\x8d\xb3\x57\x39\x1a\x0b\xc4\xfd\xec\x8b\x0d\x10\x66\x39" + , vecCiphertext = "\x73\x23\xea\x61\xd0\x59\x32\x26\x00\x47\xd9\x42" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x30\x3a\xaf\x90\xf6\xfe\x21\x19\x9c\x60\x68\x57\x74\x37\xa0\xc4" + , vecCiphertext = + "\x74\x3f\x7c\x80\x77\xab\x25\xf8\x62\x4e\x2e\x94\x85\x79\xcf\x77" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x1a\x8e\x45\xdc\xd4\x57\x8c\x66\x7c\xd8\x68\x47\xbf\x61\x55\xff" + , vecCiphertext = + "\x84\xe0\x7e\x62\xba\x83\xa6\x58\x54\x17\x24\x5d\x7e\xc4\x13\xa9\xfe\x42\x7d\x63\x15\xc0\x9b\x57\xce\x45\xf2\xe3\x93\x6a\x94\x45" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x5e\x6e\x31\x1d\xbf\x39\x5d\x35\xb0\xfe\x39\xc2\x71\x43\x88\xf8" + , vecCiphertext = + "\x3f\xd2\x4c\xe1\xf5\xa6\x7b\x75\xbf\x23\x51\xf1\x81\xa4\x75\xc7\xb8\x00\xa5\xb4\xd3\xdc\xf7\x01\x06\xb1\xee\xa8\x2f\xa1\xd6\x4d\xf4\x2b\xf7\x22\x61\x22\xfa\x92\xe1\x7a\x40\xee\xaa\xc1\x20\x1b" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x8a\x26\x3d\xd3\x17\xaa\x88\xd5\x6b\xdf\x39\x36\xdb\xa7\x5b\xb8" + , vecCiphertext = + "\x24\x33\x66\x8f\x10\x58\x19\x0f\x6d\x43\xe3\x60\xf4\xf3\x5c\xd8\xe4\x75\x12\x7c\xfc\xa7\x02\x8e\xa8\xab\x5c\x20\xf7\xab\x2a\xf0\x25\x16\xa2\xbd\xcb\xc0\x8d\x52\x1b\xe3\x7f\xf2\x8c\x15\x2b\xba\x36\x69\x7f\x25\xb4\xcd\x16\x9c\x65\x90\xd1\xdd\x39\x56\x6d\x3f" + } + , Vector + { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x3b\x0a\x1a\x25\x60\x96\x9c\xdf\x79\x0d\x99\x75\x9a\xbd\x15\x08" + , vecCiphertext = "\x1e\x6d\xab\xa3\x56\x69\xf4\x27" + } + , Vector + { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x08\x29\x9c\x51\x02\x74\x5a\xaa\x3a\x0c\x46\x9f\xad\x9e\x07\x5a" + , vecCiphertext = "\x29\x6c\x78\x89\xfd\x99\xf4\x19\x17\xf4\x46\x20" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x8f\x89\x36\xec\x03\x9e\x4e\x4b\xb9\x7e\xbd\x8c\x44\x57\x44\x1f" + , vecCiphertext = + "\xe2\xb0\xc5\xda\x79\xa9\x01\xc1\x74\x5f\x70\x05\x25\xcb\x33\x5b" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xe6\xaf\x6a\x7f\x87\x28\x7d\xa0\x59\xa7\x16\x84\xed\x34\x98\xe1" + , vecCiphertext = + "\x62\x00\x48\xef\x3c\x1e\x73\xe5\x7e\x02\xbb\x85\x62\xc4\x16\xa3\x19\xe7\x3e\x4c\xaa\xc8\xe9\x6a\x1e\xcb\x29\x33\x14\x5a\x1d\x71" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x6a\x8c\xc3\x86\x5f\x76\x89\x7c\x2e\x4b\x24\x5c\xf3\x1c\x51\xf2" + , vecCiphertext = + "\x50\xc8\x30\x3e\xa9\x39\x25\xd6\x40\x90\xd0\x7b\xd1\x09\xdf\xd9\x51\x5a\x5a\x33\x43\x10\x19\xc1\x7d\x93\x46\x59\x99\xa8\xb0\x05\x32\x01\xd7\x23\x12\x0a\x85\x62\xb8\x38\xcd\xff\x25\xbf\x9d\x1e" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xcd\xc4\x6a\xe4\x75\x56\x3d\xe0\x37\x00\x1e\xf8\x4a\xe2\x17\x44" + , vecCiphertext = + "\x2f\x5c\x64\x05\x9d\xb5\x5e\xe0\xfb\x84\x7e\xd5\x13\x00\x37\x46\xac\xa4\xe6\x1c\x71\x1b\x5d\xe2\xe7\xa7\x7f\xfd\x02\xda\x42\xfe\xec\x60\x19\x10\xd3\x46\x7b\xb8\xb3\x6e\xbb\xae\xbc\xe5\xfb\xa3\x0d\x36\xc9\x5f\x48\xa3\xe7\x98\x0f\x0e\x7a\xc2\x99\x33\x2a\x80" + } + , Vector + { vecPlaintext = "\x02\x00\x00\x00" + , vecAAD = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x07\xeb\x1f\x84\xfb\x28\xf8\xcb\x73\xde\x8e\x99\xe2\xf4\x8a\x14" + , vecCiphertext = "\xa8\xfe\x3e\x87" + } + , Vector + { vecPlaintext = + "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00" + , vecAAD = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x24\xaf\xc9\x80\x5e\x97\x6f\x45\x1e\x6d\x87\xf6\xfe\x10\x65\x14" + , vecCiphertext = + "\x6b\xb0\xfe\xcf\x5d\xed\x9b\x77\xf9\x02\xc7\xd5\xda\x23\x6a\x43\x91\xdd\x02\x97" + } + , Vector + { vecPlaintext = + "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00" + , vecAAD = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xbf\xf9\xb2\xef\x00\xfb\x47\x92\x0c\xc7\x2a\x0c\x0f\x13\xb9\xfd" + , vecCiphertext = + "\x44\xd0\xaa\xf6\xfb\x2f\x1f\x34\xad\xd5\xe8\x06\x4e\x83\xe1\x2a\x2a\xda" + } + , Vector + { vecPlaintext = "" + , vecAAD = "" + , vecKey = + "\xe6\x60\x21\xd5\xeb\x8e\x4f\x40\x66\xd4\xad\xb9\xc3\x35\x60\xe4" + , vecNonce = "\xf4\x6e\x44\xbb\x3d\xa0\x01\x5c\x94\xf7\x08\x87" + , vecTag = + "\xa4\x19\x4b\x79\x07\x1b\x01\xa8\x7d\x65\xf7\x06\xe3\x94\x95\x78" + , vecCiphertext = "" + } + , Vector + { vecPlaintext = "\x7a\x80\x6c" + , vecAAD = "\x46\xbb\x91\xc3\xc5" + , vecKey = + "\x36\x86\x42\x00\xe0\xea\xf5\x28\x4d\x88\x4a\x0e\x77\xd3\x16\x46" + , vecNonce = "\xba\xe8\xe3\x7f\xc8\x34\x41\xb1\x60\x34\x56\x6b" + , vecTag = + "\x71\x1b\xd8\x5b\xc1\xe4\xd3\xe0\xa4\x62\xe0\x74\xee\xa4\x28\xa8" + , vecCiphertext = "\xaf\x60\xeb" + } + , Vector + { vecPlaintext = "\xbd\xc6\x6f\x14\x65\x45" + , vecAAD = "\xfc\x88\x0c\x94\xa9\x51\x98\x87\x42\x96" + , vecKey = + "\xae\xdb\x64\xa6\xc5\x90\xbc\x84\xd1\xa5\xe2\x69\xe4\xb4\x78\x01" + , vecNonce = "\xaf\xc0\x57\x7e\x34\x69\x9b\x9e\x67\x1f\xdd\x4f" + , vecTag = + "\xd6\xa9\xc4\x55\x45\xcf\xc1\x1f\x03\xad\x74\x3d\xba\x20\xf9\x66" + , vecCiphertext = "\xbb\x93\xa3\xe3\x4d\x3c" + } + , Vector + { vecPlaintext = "\x11\x77\x44\x1f\x19\x54\x95\x86\x0f" + , vecAAD = "\x04\x67\x87\xf3\xea\x22\xc1\x27\xaa\xf1\x95\xd1\x89\x47\x28" + , vecKey = + "\xd5\xcc\x1f\xd1\x61\x32\x0b\x69\x20\xce\x07\x78\x7f\x86\x74\x3b" + , vecNonce = "\x27\x5d\x1a\xb3\x2f\x6d\x1f\x04\x34\xd8\x84\x8c" + , vecTag = + "\x1d\x02\xfd\x0c\xd1\x74\xc8\x4f\xc5\xda\xe2\xf6\x0f\x52\xfd\x2b" + , vecCiphertext = "\x4f\x37\x28\x1f\x7a\xd1\x29\x49\xd0" + } + , Vector + { vecPlaintext = "\x9f\x57\x2c\x61\x4b\x47\x45\x91\x44\x74\xe7\xc7" + , vecAAD = + "\xc9\x88\x2e\x53\x86\xfd\x9f\x92\xec\x48\x9c\x8f\xde\x2b\xe2\xcf\x97\xe7\x4e\x93" + , vecKey = + "\xb3\xfe\xd1\x47\x3c\x52\x8b\x84\x26\xa5\x82\x99\x59\x29\xa1\x49" + , vecNonce = "\x9e\x9a\xd8\x78\x0c\x8d\x63\xd0\xab\x41\x49\xc0" + , vecTag = + "\xc1\xdc\x2f\x87\x1f\xb7\x56\x1d\xa1\x28\x6e\x65\x5e\x24\xb7\xb0" + , vecCiphertext = "\xf5\x46\x73\xc5\xdd\xf7\x10\xc7\x45\x64\x1c\x8b" + } + , Vector + { vecPlaintext = "\x0d\x8c\x84\x51\x17\x80\x82\x35\x5c\x9e\x94\x0f\xea\x2f\x58" + , vecAAD = + "\x29\x50\xa7\x0d\x5a\x1d\xb2\x31\x6f\xd5\x68\x37\x8d\xa1\x07\xb5\x2b\x0d\xa5\x52\x10\xcc\x1c\x1b\x0a" + , vecKey = + "\x2d\x4e\xd8\x7d\xa4\x41\x02\x95\x2e\xf9\x4b\x02\xb8\x05\x24\x9b" + , vecNonce = "\xac\x80\xe6\xf6\x14\x55\xbf\xac\x83\x08\xa2\xd4" + , vecTag = + "\x83\xb3\x44\x9b\x9f\x39\x55\x2d\xe9\x9d\xc2\x14\xa1\x19\x0b\x0b" + , vecCiphertext = "\xc9\xff\x54\x5e\x07\xb8\x8a\x01\x5f\x05\xb2\x74\x54\x0a\xa1" + } + , Vector + { vecPlaintext = + "\x6b\x3d\xb4\xda\x3d\x57\xaa\x94\x84\x2b\x98\x03\xa9\x6e\x07\xfb\x6d\xe7" + , vecAAD = + "\x18\x60\xf7\x62\xeb\xfb\xd0\x82\x84\xe4\x21\x70\x2d\xe0\xde\x18\xba\xa9\xc9\x59\x62\x91\xb0\x84\x66\xf3\x7d\xe2\x1c\x7f" + , vecKey = + "\xbd\xe3\xb2\xf2\x04\xd1\xe9\xf8\xb0\x6b\xc4\x7f\x97\x45\xb3\xd1" + , vecNonce = "\xae\x06\x55\x6f\xb6\xaa\x78\x90\xbe\xbc\x18\xfe" + , vecTag = + "\x3e\x37\x70\x94\xf0\x47\x09\xf6\x4d\x7b\x98\x53\x10\xa4\xdb\x84" + , vecCiphertext = + "\x62\x98\xb2\x96\xe2\x4e\x8c\xc3\x5d\xce\x0b\xed\x48\x4b\x7f\x30\xd5\x80" + } + , Vector + { vecPlaintext = + "\xe4\x2a\x3c\x02\xc2\x5b\x64\x86\x9e\x14\x6d\x7b\x23\x39\x87\xbd\xdf\xc2\x40\x87\x1d" + , vecAAD = + "\x75\x76\xf7\x02\x8e\xc6\xeb\x5e\xa7\xe2\x98\x34\x2a\x94\xd4\xb2\x02\xb3\x70\xef\x97\x68\xec\x65\x61\xc4\xfe\x6b\x7e\x72\x96\xfa\x85\x9c\x21" + , vecKey = + "\xf9\x01\xcf\xe8\xa6\x96\x15\xa9\x3f\xdf\x7a\x98\xca\xd4\x81\x79" + , vecNonce = "\x62\x45\x70\x9f\xb1\x88\x53\xf6\x8d\x83\x36\x40" + , vecTag = + "\x2d\x15\x50\x6c\x84\xa9\xed\xd6\x5e\x13\xe9\xd2\x4a\x2a\x6e\x70" + , vecCiphertext = + "\x39\x1c\xc3\x28\xd4\x84\xa4\xf4\x64\x06\x18\x1b\xcd\x62\xef\xd9\xb3\xee\x19\x7d\x05" + } + ] + +vectors256 :: [Vector AES256] +vectors256 = + [ Vector + { vecPlaintext = "" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x07\xf5\xf4\x16\x9b\xbf\x55\xa8\x40\x0c\xd4\x7e\xa6\xfd\x40\x0f" + , vecCiphertext = "" + } + , Vector + { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x84\x31\x22\x13\x0f\x73\x64\xb7\x61\xe0\xb9\x74\x27\xe3\xdf\x28" + , vecCiphertext = "\xc2\xef\x32\x8e\x5c\x71\xc8\x3b" + } + , Vector + { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x8c\xa5\x0d\xa9\xae\x65\x59\xe4\x8f\xd1\x0f\x6e\x5c\x9c\xa1\x7e" + , vecCiphertext = "\x9a\xab\x2a\xeb\x3f\xaa\x0a\x34\xae\xa8\xe2\xb1" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xc9\xea\xc6\xfa\x70\x09\x42\x70\x2e\x90\x86\x23\x83\xc6\xc3\x66" + , vecCiphertext = + "\x85\xa0\x1b\x63\x02\x5b\xa1\x9b\x7f\xd3\xdd\xfc\x03\x3b\x3e\x76" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xe8\x19\xe6\x3a\xbc\xd0\x20\xb0\x06\xa9\x76\x39\x76\x32\xeb\x5d" + , vecCiphertext = + "\x4a\x6a\x9d\xb4\xc8\xc6\x54\x92\x01\xb9\xed\xb5\x30\x06\xcb\xa8\x21\xec\x9c\xf8\x50\x94\x8a\x7c\x86\xc6\x8a\xc7\x53\x9d\x02\x7f" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x79\x0b\xc9\x68\x80\xa9\x9b\xa8\x04\xbd\x12\xc0\xe6\xa2\x2c\xc4" + , vecCiphertext = + "\xc0\x0d\x12\x18\x93\xa9\xfa\x60\x3f\x48\xcc\xc1\xca\x3c\x57\xce\x74\x99\x24\x5e\xa0\x04\x6d\xb1\x6c\x53\xc7\xc6\x6f\xe7\x17\xe3\x9c\xf6\xc7\x48\x83\x7b\x61\xf6\xee\x3a\xdc\xee\x17\x53\x4e\xd5" + } + , Vector + { vecPlaintext = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x11\x28\x64\xc2\x69\xfc\x0d\x9d\x88\xc6\x1f\xa4\x7e\x39\xaa\x08" + , vecCiphertext = + "\xc2\xd5\x16\x0a\x1f\x86\x83\x83\x49\x10\xac\xda\xfc\x41\xfb\xb1\x63\x2d\x4a\x35\x3e\x8b\x90\x5e\xc9\xa5\x49\x9a\xc3\x4f\x96\xc7\xe1\x04\x9e\xb0\x80\x88\x38\x91\xa4\xdb\x8c\xaa\xa1\xf9\x9d\xd0\x04\xd8\x04\x87\x54\x07\x35\x23\x4e\x37\x44\x51\x2c\x6f\x90\xce" + } + , Vector + { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x91\x21\x3f\x26\x7e\x3b\x45\x2f\x02\xd0\x1a\xe3\x3e\x4e\xc8\x54" + , vecCiphertext = "\x1d\xe2\x29\x67\x23\x7a\x81\x32" + } + , Vector + { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xc1\xa4\xa1\x9a\xe8\x00\x94\x1c\xcd\xc5\x7c\xc8\x41\x3c\x27\x7f" + , vecCiphertext = "\x16\x3d\x6f\x9c\xc1\xb3\x46\xcd\x45\x3a\x2e\x4c" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xb2\x92\xd2\x8f\xf6\x11\x89\xe8\xe4\x9f\x38\x75\xef\x91\xaf\xf7" + , vecCiphertext = + "\xc9\x15\x45\x82\x3c\xc2\x4f\x17\xdb\xb0\xe9\xe8\x07\xd5\xec\x17" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xae\xa1\xba\xd1\x27\x02\xe1\x96\x56\x04\x37\x4a\xab\x96\xdb\xbc" + , vecCiphertext = + "\x07\xda\xd3\x64\xbf\xc2\xb9\xda\x89\x11\x6d\x7b\xef\x6d\xaa\xaf\x6f\x25\x55\x10\xaa\x65\x4f\x92\x0a\xc8\x1b\x94\xe8\xba\xd3\x65" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x03\x33\x27\x42\xb2\x28\xc6\x47\x17\x36\x16\xcf\xd4\x4c\x54\xeb" + , vecCiphertext = + "\xc6\x7a\x1f\x0f\x56\x7a\x51\x98\xaa\x1f\xcc\x8e\x3f\x21\x31\x43\x36\xf7\xf5\x1c\xa8\xb1\xaf\x61\xfe\xac\x35\xa8\x64\x16\xfa\x47\xfb\xca\x3b\x5f\x74\x9c\xdf\x56\x45\x27\xf2\x31\x4f\x42\xfe\x25" + } + , Vector + { vecPlaintext = + "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "\x01" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x5b\xde\x02\x85\x03\x7c\x5d\xe8\x1e\x5b\x57\x0a\x04\x9b\x62\xa0" + , vecCiphertext = + "\x67\xfd\x45\xe1\x26\xbf\xb9\xa7\x99\x30\xc4\x3a\xad\x2d\x36\x96\x7d\x3f\x0e\x4d\x21\x7c\x1e\x55\x1f\x59\x72\x78\x70\xbe\xef\xc9\x8c\xb9\x33\xa8\xfc\xe9\xde\x88\x7b\x1e\x40\x79\x99\x88\xdb\x1f\xc3\xf9\x18\x80\xed\x40\x5b\x2d\xd2\x98\x31\x88\x58\x46\x7c\x89" + } + , Vector + { vecPlaintext = "\x02\x00\x00\x00" + , vecAAD = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\x18\x35\xe5\x17\x74\x1d\xfd\xdc\xcf\xa0\x7f\xa4\x66\x1b\x74\xcf" + , vecCiphertext = "\x22\xb3\xf4\xcd" + } + , Vector + { vecPlaintext = + "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00" + , vecAAD = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xb8\x79\xad\x97\x6d\x82\x42\xac\xc1\x88\xab\x59\xca\xbf\xe3\x07" + , vecCiphertext = + "\x43\xdd\x01\x63\xcd\xb4\x8f\x9f\xe3\x21\x2b\xf6\x1b\x20\x19\x76\x06\x7f\x34\x2b" + } + , Vector + { vecPlaintext = + "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00" + , vecAAD = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00" + , vecKey = + "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xcf\xcd\xf5\x04\x21\x12\xaa\x29\x68\x5c\x91\x2f\xc2\x05\x65\x43" + , vecCiphertext = + "\x46\x24\x01\x72\x4b\x5c\xe6\x58\x8d\x5a\x54\xaa\xe5\x37\x55\x13\xa0\x75" + } + , Vector + { vecPlaintext = "" + , vecAAD = "" + , vecKey = + "\xe6\x60\x21\xd5\xeb\x8e\x4f\x40\x66\xd4\xad\xb9\xc3\x35\x60\xe4\xf4\x6e\x44\xbb\x3d\xa0\x01\x5c\x94\xf7\x08\x87\x36\x86\x42\x00" + , vecNonce = "\xe0\xea\xf5\x28\x4d\x88\x4a\x0e\x77\xd3\x16\x46" + , vecTag = + "\x16\x9f\xbb\x2f\xbf\x38\x9a\x99\x5f\x63\x90\xaf\x22\x22\x8a\x62" + , vecCiphertext = "" + } + , Vector + { vecPlaintext = "\x67\x1f\xdd" + , vecAAD = "\x4f\xbd\xc6\x6f\x14" + , vecKey = + "\xba\xe8\xe3\x7f\xc8\x34\x41\xb1\x60\x34\x56\x6b\x7a\x80\x6c\x46\xbb\x91\xc3\xc5\xae\xdb\x64\xa6\xc5\x90\xbc\x84\xd1\xa5\xe2\x69" + , vecNonce = "\xe4\xb4\x78\x01\xaf\xc0\x57\x7e\x34\x69\x9b\x9e" + , vecTag = + "\x93\xda\x9b\xb8\x13\x33\xae\xe0\xc7\x85\xb2\x40\xd3\x19\x71\x9d" + , vecCiphertext = "\x0e\xac\xcb" + } + , Vector + { vecPlaintext = "\x19\x54\x95\x86\x0f\x04" + , vecAAD = "\x67\x87\xf3\xea\x22\xc1\x27\xaa\xf1\x95" + , vecKey = + "\x65\x45\xfc\x88\x0c\x94\xa9\x51\x98\x87\x42\x96\xd5\xcc\x1f\xd1\x61\x32\x0b\x69\x20\xce\x07\x78\x7f\x86\x74\x3b\x27\x5d\x1a\xb3" + , vecNonce = "\x2f\x6d\x1f\x04\x34\xd8\x84\x8c\x11\x77\x44\x1f" + , vecTag = + "\x6b\x62\xb8\x4d\xc4\x0c\x84\x63\x6a\x5e\xc1\x20\x20\xec\x8c\x2c" + , vecCiphertext = "\xa2\x54\xda\xd4\xf3\xf9" + } + , Vector + { vecPlaintext = "\xc9\x88\x2e\x53\x86\xfd\x9f\x92\xec" + , vecAAD = "\x48\x9c\x8f\xde\x2b\xe2\xcf\x97\xe7\x4e\x93\x2d\x4e\xd8\x7d" + , vecKey = + "\xd1\x89\x47\x28\xb3\xfe\xd1\x47\x3c\x52\x8b\x84\x26\xa5\x82\x99\x59\x29\xa1\x49\x9e\x9a\xd8\x78\x0c\x8d\x63\xd0\xab\x41\x49\xc0" + , vecNonce = "\x9f\x57\x2c\x61\x4b\x47\x45\x91\x44\x74\xe7\xc7" + , vecTag = + "\xc0\xfd\x3d\xc6\x62\x8d\xfe\x55\xeb\xb0\xb9\xfb\x22\x95\xc8\xc2" + , vecCiphertext = "\x0d\xf9\xe3\x08\x67\x82\x44\xc4\x4b" + } + , Vector + { vecPlaintext = "\x1d\xb2\x31\x6f\xd5\x68\x37\x8d\xa1\x07\xb5\x2b" + , vecAAD = + "\x0d\xa5\x52\x10\xcc\x1c\x1b\x0a\xbd\xe3\xb2\xf2\x04\xd1\xe9\xf8\xb0\x6b\xc4\x7f" + , vecKey = + "\xa4\x41\x02\x95\x2e\xf9\x4b\x02\xb8\x05\x24\x9b\xac\x80\xe6\xf6\x14\x55\xbf\xac\x83\x08\xa2\xd4\x0d\x8c\x84\x51\x17\x80\x82\x35" + , vecNonce = "\x5c\x9e\x94\x0f\xea\x2f\x58\x29\x50\xa7\x0d\x5a" + , vecTag = + "\x40\x40\x99\xc2\x58\x7f\x64\x97\x9f\x21\x82\x67\x06\xd4\x97\xd5" + , vecCiphertext = "\x8d\xbe\xb9\xf7\x25\x5b\xf5\x76\x9d\xd5\x66\x92" + } + , Vector + { vecPlaintext = "\x21\x70\x2d\xe0\xde\x18\xba\xa9\xc9\x59\x62\x91\xb0\x84\x66" + , vecAAD = + "\xf3\x7d\xe2\x1c\x7f\xf9\x01\xcf\xe8\xa6\x96\x15\xa9\x3f\xdf\x7a\x98\xca\xd4\x81\x79\x62\x45\x70\x9f" + , vecKey = + "\x97\x45\xb3\xd1\xae\x06\x55\x6f\xb6\xaa\x78\x90\xbe\xbc\x18\xfe\x6b\x3d\xb4\xda\x3d\x57\xaa\x94\x84\x2b\x98\x03\xa9\x6e\x07\xfb" + , vecNonce = "\x6d\xe7\x18\x60\xf7\x62\xeb\xfb\xd0\x82\x84\xe4" + , vecTag = + "\xb3\x08\x0d\x28\xf6\xeb\xb5\xd3\x64\x8c\xe9\x7b\xd5\xba\x67\xfd" + , vecCiphertext = "\x79\x35\x76\xdf\xa5\xc0\xf8\x87\x29\xa7\xed\x3c\x2f\x1b\xff" + } + , Vector + { vecPlaintext = + "\xb2\x02\xb3\x70\xef\x97\x68\xec\x65\x61\xc4\xfe\x6b\x7e\x72\x96\xfa\x85" + , vecAAD = + "\x9c\x21\x59\x05\x8b\x1f\x0f\xe9\x14\x33\xa5\xbd\xc2\x0e\x21\x4e\xab\x7f\xec\xef\x44\x54\xa1\x0e\xf0\x65\x7d\xf2\x1a\xc7" + , vecKey = + "\xb1\x88\x53\xf6\x8d\x83\x36\x40\xe4\x2a\x3c\x02\xc2\x5b\x64\x86\x9e\x14\x6d\x7b\x23\x39\x87\xbd\xdf\xc2\x40\x87\x1d\x75\x76\xf7" + , vecNonce = "\x02\x8e\xc6\xeb\x5e\xa7\xe2\x98\x34\x2a\x94\xd4" + , vecTag = + "\x45\x4f\xc2\xa1\x54\xfe\xa9\x1f\x83\x63\xa3\x9f\xec\x7d\x0a\x49" + , vecCiphertext = + "\x85\x7e\x16\xa6\x49\x15\xa7\x87\x63\x76\x87\xdb\x4a\x95\x19\x63\x5c\xdd" + } + , Vector + { vecPlaintext = + "\xce\xd5\x32\xce\x41\x59\xb0\x35\x27\x7d\x4d\xfb\xb7\xdb\x62\x96\x8b\x13\xcd\x4e\xec" + , vecAAD = + "\x73\x43\x20\xcc\xc9\xd9\xbb\xbb\x19\xcb\x81\xb2\xaf\x4e\xcb\xc3\xe7\x28\x34\x32\x1f\x7a\xa0\xf7\x0b\x72\x82\xb4\xf3\x3d\xf2\x3f\x16\x75\x41" + , vecKey = + "\x3c\x53\x5d\xe1\x92\xea\xed\x38\x22\xa2\xfb\xbe\x2c\xa9\xdf\xc8\x82\x55\xe1\x4a\x66\x1b\x8a\xa8\x2c\xc5\x42\x36\x09\x3b\xbc\x23" + , vecNonce = "\x68\x80\x89\xe5\x55\x40\xdb\x18\x72\x50\x4e\x1c" + , vecTag = + "\x9d\x6c\x70\x29\x67\x5b\x89\xea\xf4\xba\x1d\xed\x1a\x28\x65\x94" + , vecCiphertext = + "\x62\x66\x60\xc2\x6e\xa6\x61\x2f\xb1\x7a\xd9\x1e\x8e\x76\x76\x39\xed\xd6\xc9\xfa\xee" + } + ] + +vectorsWrap256 :: [Vector AES256] +vectorsWrap256 = + [ Vector + { vecPlaintext = + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x4d\xb9\x23\xdc\x79\x3e\xe6\x49\x7c\x76\xdc\xc0\x3a\x98\xe1\x08" + , vecAAD = "" + , vecKey = + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecCiphertext = + "\xf3\xf8\x0f\x2c\xf0\xcb\x2d\xd9\xc5\x98\x4f\xcd\xa9\x08\x45\x6c\xc5\x37\x70\x3b\x5b\xa7\x03\x24\xa6\x79\x3a\x7b\xf2\x18\xd3\xea" + } + , Vector + { vecPlaintext = + "\xeb\x36\x40\x27\x7c\x7f\xfd\x13\x03\xc7\xa5\x42\xd0\x2d\x3e\x4c\x00\x00\x00\x00\x00\x00\x00\x00" + , vecAAD = "" + , vecKey = + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecNonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecTag = + "\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + , vecCiphertext = + "\x18\xce\x4f\x0b\x8c\xb4\xd0\xca\xc6\x5f\xea\x8f\x79\x25\x7b\x20\x88\x8e\x53\xe7\x22\x99\xe5\x6d" + } + ] + +makeEncryptionTest :: BlockCipher128 aes => Int -> Vector aes -> Spec +makeEncryptionTest i vec@Vector{..} = + it (show i) $ + encrypt (vecCipher vec) n vecAAD vecPlaintext `shouldBe` (t, vecCiphertext) + where + t = AuthTag (B.convert vecTag) + n = throwCryptoError (nonce vecNonce) + +makeDecryptionTest :: BlockCipher128 aes => Int -> Vector aes -> Spec +makeDecryptionTest i vec@Vector{..} = + it (show i) $ + decrypt (vecCipher vec) n vecAAD vecCiphertext t `shouldBe` Just vecPlaintext + where + t = AuthTag (B.convert vecTag) + n = throwCryptoError (nonce vecNonce) + +katTests + :: String + -> (forall c. BlockCipher128 c => Int -> Vector c -> Spec) + -> Spec +katTests name makeTest = + describe name $ do + describe "AES128" $ zipWithM_ makeTest [1 ..] vectors128 + describe "AES256" $ zipWithM_ makeTest [1 ..] vectors256 + describe "CounterWrap" $ zipWithM_ makeTest [1 ..] vectorsWrap256 + +newtype Key c = Key ByteString + deriving (Show, Eq) + +instance Arbitrary (Key AES128) where + arbitrary = Key <$> arbitraryBS 16 + +instance Arbitrary (Key AES256) where + arbitrary = Key <$> arbitraryBS 32 + +instance Arbitrary Nonce where + arbitrary = throwCryptoError . nonce <$> arbitraryBS 12 + +encDecTest + :: BlockCipher128 c + => Proxy c + -> Key c + -> Nonce + -> ArbitraryBS0_2901 + -> ArbitraryBS0_2901 + -> Property +encDecTest prx (Key key) iv (ArbitraryBS0_2901 aad) (ArbitraryBS0_2901 input) = + let c = throwCryptoError (cipherInit key) `asProxyTypeOf` prx + (tag, ciphertext) = encrypt c iv aad input + in decrypt c iv aad ciphertext tag === Just input + +spec :: Spec +spec = + describe "AES-GCM-SIV" $ do + describe "KATs" $ do + katTests "encrypt" makeEncryptionTest + katTests "decrypt" makeDecryptionTest + describe "properties" $ do + prop "AES128" $ encDecTest (Proxy :: Proxy AES128) + prop "AES256" $ encDecTest (Proxy :: Proxy AES256) diff --git a/tests/BlockCipher/AESSpec.hs b/tests/BlockCipher/AESSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/AESSpec.hs @@ -0,0 +1,232 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.AESSpec (spec) where + +import BlockCipher +import qualified Crypto.Cipher.AES as AES +import Crypto.Cipher.Types +import Crypto.Error +import Crypto.Hash (Digest, SHA256, hash) +import qualified Data.ByteArray as BA +import qualified Data.ByteString as B +import Data.Maybe +import Imports + +import qualified BlockCipher.AES.CBC as KATCBC +import qualified BlockCipher.AES.CCM as KATCCM +import qualified BlockCipher.AES.CTR as KATCTR +import qualified BlockCipher.AES.ECB as KATECB +import qualified BlockCipher.AES.GCM as KATGCM +import qualified BlockCipher.AES.GCMLong as KATGCMLong +import qualified BlockCipher.AES.OCB3 as KATOCB3 +import qualified BlockCipher.AES.XTS as KATXTS + +{- +instance Show AES.AES where + show _ = "AES" +instance Arbitrary AES.AESIV where + arbitrary = AES.aesIV_ . B.pack <$> replicateM 16 arbitrary +instance Arbitrary AES.AES where + arbitrary = AES.initAES . B.pack <$> replicateM 16 arbitrary +-} + +toKatECB (k, p, c) = KAT_ECB{ecbKey = k, ecbPlaintext = p, ecbCiphertext = c} +toKatCBC (k, iv, p, c) = KAT_CBC{cbcKey = k, cbcIV = iv, cbcPlaintext = p, cbcCiphertext = c} +toKatCTR (k, iv, p, c) = KAT_CTR{ctrKey = k, ctrIV = iv, ctrPlaintext = p, ctrCiphertext = c} +toKatXTS (k1, k2, iv, p, _, c) = + KAT_XTS + { xtsKey1 = k1 + , xtsKey2 = k2 + , xtsIV = iv + , xtsPlaintext = p + , xtsCiphertext = c + } +toKatAEAD mode (k, iv, h, p, c, taglen, tag) = + KAT_AEAD + { aeadMode = mode + , aeadKey = k + , aeadIV = iv + , aeadHeader = h + , aeadPlaintext = p + , aeadCiphertext = c + , aeadTaglen = taglen + , aeadTag = tag + } +toKatGCM = toKatAEAD AEAD_GCM +toKatOCB = toKatAEAD AEAD_OCB + +toKatCCM (k, iv, h, i, o, m) = + KAT_AEAD + { aeadMode = AEAD_CCM (B.length i) (ccmMVal m) CCM_L2 + , aeadKey = k + , aeadIV = iv + , aeadHeader = h + , aeadPlaintext = i + , aeadCiphertext = ct + , aeadTaglen = m + , aeadTag = at + } + where + ccmMVal x = + fromMaybe (error $ "unsupported CCM tag length: " ++ show x) $ + lookup + x + [ (4, CCM_M4) + , (6, CCM_M6) + , (8, CCM_M8) + , (10, CCM_M10) + , (12, CCM_M12) + , (14, CCM_M14) + , (16, CCM_M16) + ] + ctWithTag = B.drop (B.length h) o + (ct, at) = B.splitAt (B.length ctWithTag - m) ctWithTag + +kats128 = + defaultKATs + { kat_ECB = map toKatECB KATECB.vectors_aes128_enc + , kat_CBC = map toKatCBC KATCBC.vectors_aes128_enc + , kat_CTR = map toKatCTR KATCTR.vectors_aes128_enc + , kat_CFB = + [ KAT_CFB + { cfbKey = + "\x2b\x7e\x15\x16\x28\xae\xd2\xa6\xab\xf7\x15\x88\x09\xcf\x4f\x3c" + , cfbIV = + "\xC8\xA6\x45\x37\xA0\xB3\xA9\x3F\xCD\xE3\xCD\xAD\x9F\x1C\xE5\x8B" + , cfbPlaintext = + "\x30\xc8\x1c\x46\xa3\x5c\xe4\x11\xe5\xfb\xc1\x19\x1a\x0a\x52\xef" + , cfbCiphertext = + "\x26\x75\x1f\x67\xa3\xcb\xb1\x40\xb1\x80\x8c\xf1\x87\xa4\xf4\xdf" + } + ] + , kat_XTS = map toKatXTS KATXTS.vectors_aes128_enc + , kat_AEAD = + map toKatGCM KATGCM.vectors_aes128_enc + ++ map toKatOCB KATOCB3.vectors_aes128_enc + ++ map toKatCCM KATCCM.vectors_aes128_enc + } + +kats192 = + defaultKATs + { kat_ECB = map toKatECB KATECB.vectors_aes192_enc + , kat_CBC = map toKatCBC KATCBC.vectors_aes192_enc + , kat_CTR = map toKatCTR KATCTR.vectors_aes192_enc + , kat_AEAD = + map toKatGCM KATGCM.vectors_aes192_enc + ++ map toKatOCB KATOCB3.vectors_aes192_enc + ++ map toKatCCM KATCCM.vectors_aes192_enc + } + +kats256 = + defaultKATs + { kat_ECB = map toKatECB KATECB.vectors_aes256_enc + , kat_CBC = map toKatCBC KATCBC.vectors_aes256_enc + , kat_CTR = map toKatCTR KATCTR.vectors_aes256_enc + , kat_XTS = map toKatXTS KATXTS.vectors_aes256_enc + , kat_AEAD = + map toKatGCM KATGCM.vectors_aes256_enc + ++ map toKatOCB KATOCB3.vectors_aes256_enc + ++ map toKatCCM KATCCM.vectors_aes256_enc + } + +-- SP 800-38D 5.2.1.1: 1 <= len(IV) <= 2^64 - 1. A zero-length IV makes +-- J0 the GHASH of the empty string, which leaks the authentication key. +aeadIVLengthTests :: Spec +aeadIVLengthTests = + describe "AEAD IV length" $ do + it "96-bit IV accepted" $ + isRight (initWith (B.replicate 12 0)) `shouldBe` True + it "8-bit IV accepted" $ + isRight (initWith (B.replicate 1 0)) `shouldBe` True + it "empty IV rejected" $ + initWith B.empty `shouldBe` Left CryptoError_IvSizeInvalid + where + ctx = throwCryptoError (cipherInit (B.replicate 16 0)) :: AES.AES128 + initWith iv = + eitherCryptoError (() <$ aeadInit AEAD_GCM ctx (iv :: ByteString)) + isRight = either (const False) (const True) + +aeadTagLengthTests :: Spec +aeadTagLengthTests = + describe "AEAD tag length" $ do + it "full tag verifies" $ openWith fullTag `shouldBe` Just message + it "empty tag rejected" $ openWith B.empty `shouldBe` Nothing + it "1-byte tag rejected" $ openWith (B.take 1 fullTag) `shouldBe` Nothing + it "3-byte tag rejected" $ openWith (B.take 3 fullTag) `shouldBe` Nothing + it "wrong tag rejected" $ + openWith (B.map (+ 1) fullTag) `shouldBe` Nothing + -- a truncated tag is still at or above the minimum, so the length + -- taken from the tag is the peer's choice of how much to verify + it "4-byte tag accepted, since the tag sets the length" $ + openWith (B.take 4 fullTag) `shouldBe` Just message + it "tryAeadSimpleDecrypt verifies the full tag" $ + openWith' 16 fullTag `shouldBe` Just message + it "tryAeadSimpleDecrypt refuses a truncated tag" $ + openWith' 16 (B.take 4 fullTag) `shouldBe` Nothing + it "tryAeadSimpleDecrypt refuses an overlong tag" $ + openWith' 16 (fullTag `B.append` B.singleton 0) `shouldBe` Nothing + it "tryAeadSimpleDecrypt refuses a length below the minimum" $ + openWith' 3 (B.take 3 fullTag) `shouldBe` Nothing + it "tryAeadSimpleDecrypt verifies a short tag the caller asked for" $ + openWith' 8 (B.take 8 fullTag) `shouldBe` Just message + it "tryAeadSimpleDecrypt refuses a wrong tag" $ + openWith' 16 (B.map (+ 1) fullTag) `shouldBe` Nothing + where + key = B.replicate 16 0 + iv = B.replicate 12 0 + aad = "additional data" :: ByteString + message = "authenticated message" :: ByteString + ctx = throwCryptoError (cipherInit key) :: AES.AES128 + aead = throwCryptoError (aeadInit AEAD_GCM ctx iv) + (AuthTag tag, ciphertext) = aeadSimpleEncrypt aead aad message 16 + fullTag = BA.convert tag :: ByteString + openWith t = aeadSimpleDecrypt aead aad ciphertext (AuthTag (BA.convert t)) + openWith' n t = tryAeadSimpleDecrypt aead aad ciphertext n (AuthTag (BA.convert t)) + +-- The bulk loops -- eight blocks at a time under AES-NI, six at a time in +-- the assembly -- only start once the message is long enough to fill them, +-- and what they leave over goes down a different path. These lengths sit +-- either side of each of those boundaries, so a group that hashes the wrong +-- blocks or a tail that is picked up at the wrong offset shows up here. +gcmLongTests :: Spec +gcmLongTests = + describe "GCM long messages" $ mapM_ test KATGCMLong.vectors + where + test v@(klen, aadlen, ptlen, _, _) = + it + ( show klen + ++ "-byte key, " + ++ show aadlen + ++ "-byte AAD, " + ++ show ptlen + ++ "-byte message" + ) $ + case klen of + 16 -> run (undefined :: AES.AES128) v + 24 -> run (undefined :: AES.AES192) v + _ -> run (undefined :: AES.AES256) v + run + :: BlockCipher cipher + => cipher + -> KATGCMLong.KATGCMLong + -> Expectation + run cipherWitness (klen, aadlen, ptlen, tag, ctHash) = do + BA.convert authTag `shouldBe` tag + digest ciphertext `shouldBe` ctHash + aeadSimpleDecrypt aead aad ciphertext authTag `shouldBe` Just plaintext + where + cipher = throwCryptoError (cipherInit (KATGCMLong.gcmKey klen)) `asTypeOf` cipherWitness + aead = throwCryptoError (aeadInit AEAD_GCM cipher KATGCMLong.gcmIV) + aad = KATGCMLong.gcmAAD aadlen + plaintext = KATGCMLong.gcmPlaintext ptlen + (authTag, ciphertext) = aeadSimpleEncrypt aead aad plaintext 16 + digest bs = BA.convert (hash bs :: Digest SHA256) :: ByteString + +spec :: Spec +spec = do + testBlockCipher128 kats128 (undefined :: AES.AES128) + testBlockCipher128 kats192 (undefined :: AES.AES192) + testBlockCipher128 kats256 (undefined :: AES.AES256) + aeadIVLengthTests + aeadTagLengthTests + gcmLongTests diff --git a/tests/BlockCipher/BlowfishSpec.hs b/tests/BlockCipher/BlowfishSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/BlowfishSpec.hs @@ -0,0 +1,153 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.BlowfishSpec where + +import BlockCipher +import Crypto.Cipher.Blowfish +import Imports () +import Test.Hspec (Spec) + +vectors_ecb = + -- key plaintext cipher + [ KAT_ECB + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x4E\xF9\x97\x45\x61\x98\xDD\x78" + , KAT_ECB + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x51\x86\x6F\xD5\xB8\x5E\xCB\x8A" + , KAT_ECB + "\x30\x00\x00\x00\x00\x00\x00\x00" + "\x10\x00\x00\x00\x00\x00\x00\x01" + "\x7D\x85\x6F\x9A\x61\x30\x63\xF2" + , KAT_ECB + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x24\x66\xDD\x87\x8B\x96\x3C\x9D" + , KAT_ECB + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x61\xF9\xC3\x80\x22\x81\xB0\x96" + , KAT_ECB + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x7D\x0C\xC6\x30\xAF\xDA\x1E\xC7" + , KAT_ECB + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x4E\xF9\x97\x45\x61\x98\xDD\x78" + , KAT_ECB + "\xFE\xDC\xBA\x98\x76\x54\x32\x10" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x0A\xCE\xAB\x0F\xC6\xA0\xA2\x8D" + , KAT_ECB + "\x7C\xA1\x10\x45\x4A\x1A\x6E\x57" + "\x01\xA1\xD6\xD0\x39\x77\x67\x42" + "\x59\xC6\x82\x45\xEB\x05\x28\x2B" + , KAT_ECB + "\x01\x31\xD9\x61\x9D\xC1\x37\x6E" + "\x5C\xD5\x4C\xA8\x3D\xEF\x57\xDA" + "\xB1\xB8\xCC\x0B\x25\x0F\x09\xA0" + , KAT_ECB + "\x07\xA1\x13\x3E\x4A\x0B\x26\x86" + "\x02\x48\xD4\x38\x06\xF6\x71\x72" + "\x17\x30\xE5\x77\x8B\xEA\x1D\xA4" + , KAT_ECB + "\x38\x49\x67\x4C\x26\x02\x31\x9E" + "\x51\x45\x4B\x58\x2D\xDF\x44\x0A" + "\xA2\x5E\x78\x56\xCF\x26\x51\xEB" + , KAT_ECB + "\x04\xB9\x15\xBA\x43\xFE\xB5\xB6" + "\x42\xFD\x44\x30\x59\x57\x7F\xA2" + "\x35\x38\x82\xB1\x09\xCE\x8F\x1A" + , KAT_ECB + "\x01\x13\xB9\x70\xFD\x34\xF2\xCE" + "\x05\x9B\x5E\x08\x51\xCF\x14\x3A" + "\x48\xF4\xD0\x88\x4C\x37\x99\x18" + , KAT_ECB + "\x01\x70\xF1\x75\x46\x8F\xB5\xE6" + "\x07\x56\xD8\xE0\x77\x47\x61\xD2" + "\x43\x21\x93\xB7\x89\x51\xFC\x98" + , KAT_ECB + "\x43\x29\x7F\xAD\x38\xE3\x73\xFE" + "\x76\x25\x14\xB8\x29\xBF\x48\x6A" + "\x13\xF0\x41\x54\xD6\x9D\x1A\xE5" + , KAT_ECB + "\x07\xA7\x13\x70\x45\xDA\x2A\x16" + "\x3B\xDD\x11\x90\x49\x37\x28\x02" + "\x2E\xED\xDA\x93\xFF\xD3\x9C\x79" + , KAT_ECB + "\x04\x68\x91\x04\xC2\xFD\x3B\x2F" + "\x26\x95\x5F\x68\x35\xAF\x60\x9A" + "\xD8\x87\xE0\x39\x3C\x2D\xA6\xE3" + , KAT_ECB + "\x37\xD0\x6B\xB5\x16\xCB\x75\x46" + "\x16\x4D\x5E\x40\x4F\x27\x52\x32" + "\x5F\x99\xD0\x4F\x5B\x16\x39\x69" + , KAT_ECB + "\x1F\x08\x26\x0D\x1A\xC2\x46\x5E" + "\x6B\x05\x6E\x18\x75\x9F\x5C\xCA" + "\x4A\x05\x7A\x3B\x24\xD3\x97\x7B" + , KAT_ECB + "\x58\x40\x23\x64\x1A\xBA\x61\x76" + "\x00\x4B\xD6\xEF\x09\x17\x60\x62" + "\x45\x20\x31\xC1\xE4\xFA\xDA\x8E" + , KAT_ECB + "\x02\x58\x16\x16\x46\x29\xB0\x07" + "\x48\x0D\x39\x00\x6E\xE7\x62\xF2" + "\x75\x55\xAE\x39\xF5\x9B\x87\xBD" + , KAT_ECB + "\x49\x79\x3E\xBC\x79\xB3\x25\x8F" + "\x43\x75\x40\xC8\x69\x8F\x3C\xFA" + "\x53\xC5\x5F\x9C\xB4\x9F\xC0\x19" + , KAT_ECB + "\x4F\xB0\x5E\x15\x15\xAB\x73\xA7" + "\x07\x2D\x43\xA0\x77\x07\x52\x92" + "\x7A\x8E\x7B\xFA\x93\x7E\x89\xA3" + , KAT_ECB + "\x49\xE9\x5D\x6D\x4C\xA2\x29\xBF" + "\x02\xFE\x55\x77\x81\x17\xF1\x2A" + "\xCF\x9C\x5D\x7A\x49\x86\xAD\xB5" + , KAT_ECB + "\x01\x83\x10\xDC\x40\x9B\x26\xD6" + "\x1D\x9D\x5C\x50\x18\xF7\x28\xC2" + "\xD1\xAB\xB2\x90\x65\x8B\xC7\x78" + , KAT_ECB + "\x1C\x58\x7F\x1C\x13\x92\x4F\xEF" + "\x30\x55\x32\x28\x6D\x6F\x29\x5A" + "\x55\xCB\x37\x74\xD1\x3E\xF2\x01" + , KAT_ECB + "\x01\x01\x01\x01\x01\x01\x01\x01" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xFA\x34\xEC\x48\x47\xB2\x68\xB2" + , KAT_ECB + "\x1F\x1F\x1F\x1F\x0E\x0E\x0E\x0E" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xA7\x90\x79\x51\x08\xEA\x3C\xAE" + , KAT_ECB + "\xE0\xFE\xE0\xFE\xF1\xFE\xF1\xFE" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xC3\x9E\x07\x2D\x9F\xAC\x63\x1D" + , KAT_ECB + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x01\x49\x33\xE0\xCD\xAF\xF6\xE4" + , KAT_ECB + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xF2\x1E\x9A\x77\xB7\x1C\x49\xBC" + , KAT_ECB + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x24\x59\x46\x88\x57\x54\x36\x9A" + , KAT_ECB + "\xFE\xDC\xBA\x98\x76\x54\x32\x10" + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x6B\x5C\x5A\x9C\x5D\x9E\x0A\x5A" + ] + +kats = defaultKATs{kat_ECB = vectors_ecb} + +spec :: Spec +spec = testBlockCipher kats (undefined :: Blowfish64) diff --git a/tests/BlockCipher/CAST5Spec.hs b/tests/BlockCipher/CAST5Spec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/CAST5Spec.hs @@ -0,0 +1,28 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.CAST5Spec (spec) where + +import BlockCipher +import qualified Crypto.Cipher.CAST5 as CAST5 +import Test.Hspec (Spec) + +vectors_ecb = + -- key plaintext ciphertext + [ KAT_ECB + "\x01\x23\x45\x67\x12\x34\x56\x78\x23\x45\x67\x89\x34\x56\x78\x9A" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x23\x8B\x4F\xE5\x84\x7E\x44\xB2" + , KAT_ECB + "\x01\x23\x45\x67\x12\x34\x56\x78\x23\x45" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xEB\x6A\x71\x1A\x2C\x02\x27\x1B" + , KAT_ECB + "\x01\x23\x45\x67\x12" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x7A\xC8\x16\xD1\x6E\x9B\x30\x2E" + ] + +kats = defaultKATs{kat_ECB = vectors_ecb} + +spec :: Spec +spec = testBlockCipher kats (undefined :: CAST5.CAST5) diff --git a/tests/BlockCipher/CamelliaSpec.hs b/tests/BlockCipher/CamelliaSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/CamelliaSpec.hs @@ -0,0 +1,280 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE ViewPatterns #-} +{-# OPTIONS_GHC -fno-warn-unused-binds #-} +{-# OPTIONS_GHC -fno-warn-unused-matches #-} + +module BlockCipher.CamelliaSpec (spec) where + +import BlockCipher +import Imports () +import Test.Hspec + +import Control.Exception (evaluate) +import Crypto.Cipher.Camellia +import Crypto.Cipher.Types +import Crypto.Error (throwCryptoError) +import qualified Data.ByteString as B + +vectors_camellia128 = + [ KAT_ECB + (B.replicate 16 0) + (B.replicate 16 0) + ( B.pack + [ 0x3d + , 0x02 + , 0x80 + , 0x25 + , 0xb1 + , 0x56 + , 0x32 + , 0x7c + , 0x17 + , 0xf7 + , 0x62 + , 0xc1 + , 0xf2 + , 0xcb + , 0xca + , 0x71 + ] + ) + , KAT_ECB + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xab + , 0xcd + , 0xef + , 0xfe + , 0xdc + , 0xba + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + ] + ) + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xab + , 0xcd + , 0xef + , 0xfe + , 0xdc + , 0xba + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + ] + ) + ( B.pack + [ 0x67 + , 0x67 + , 0x31 + , 0x38 + , 0x54 + , 0x96 + , 0x69 + , 0x73 + , 0x08 + , 0x57 + , 0x06 + , 0x56 + , 0x48 + , 0xea + , 0xbe + , 0x43 + ] + ) + ] + +vectors_camellia192 = + [ KAT_ECB + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xab + , 0xcd + , 0xef + , 0xfe + , 0xdc + , 0xba + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + , 0x00 + , 0x11 + , 0x22 + , 0x33 + , 0x44 + , 0x55 + , 0x66 + , 0x77 + ] + ) + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xab + , 0xcd + , 0xef + , 0xfe + , 0xdc + , 0xba + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + ] + ) + ( B.pack + [ 0xb4 + , 0x99 + , 0x34 + , 0x01 + , 0xb3 + , 0xe9 + , 0x96 + , 0xf8 + , 0x4e + , 0xe5 + , 0xce + , 0xe7 + , 0xd7 + , 0x9b + , 0x09 + , 0xb9 + ] + ) + ] + +vectors_camellia256 = + [ KAT_ECB + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xab + , 0xcd + , 0xef + , 0xfe + , 0xdc + , 0xba + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + , 0x00 + , 0x11 + , 0x22 + , 0x33 + , 0x44 + , 0x55 + , 0x66 + , 0x77 + , 0x88 + , 0x99 + , 0xaa + , 0xbb + , 0xcc + , 0xdd + , 0xee + , 0xff + ] + ) + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xab + , 0xcd + , 0xef + , 0xfe + , 0xdc + , 0xba + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + ] + ) + ( B.pack + [ 0x9a + , 0xcc + , 0x23 + , 0x7d + , 0xff + , 0x16 + , 0xd7 + , 0x6c + , 0x20 + , 0xef + , 0x7c + , 0x91 + , 0x9e + , 0x3a + , 0x75 + , 0x09 + ] + ) + ] + +kats128 = defaultKATs{kat_ECB = vectors_camellia128} +kats192 = defaultKATs{kat_ECB = vectors_camellia192} +kats256 = defaultKATs{kat_ECB = vectors_camellia256} + +-- | Every vector here is one block long. ECB is the block operation applied +-- to each block and nothing else, so say that too, and say what happens to a +-- message that is not whole blocks. +manyBlockTests :: Spec +manyBlockTests = + describe "several blocks" $ do + it "ECB of a message is ECB of its blocks" $ + ecbEncrypt ctx message `shouldBe` B.concat (map (ecbEncrypt ctx) blocks) + it "and the same going back" $ + ecbDecrypt ctx cipherText + `shouldBe` B.concat (map (ecbDecrypt ctx) cipherBlocks) + it "a message of 64 KiB still decrypts to itself" $ + ecbDecrypt ctx (ecbEncrypt ctx big) `shouldBe` big + it "a message that is not whole blocks is refused" $ + -- the tail of the answer used to be whatever was in the buffer it + -- was allocated in + evaluate (B.length (ecbEncrypt ctx (B.take 20 message))) + `shouldThrow` anyErrorCall + where + ctx = throwCryptoError (cipherInit (B.replicate 16 0x2b)) :: Camellia128 + message = B.pack (map fromIntegral [1 .. 80 :: Int]) + blocks = [B.take 16 (B.drop i message) | i <- [0, 16 .. 64]] + cipherText = ecbEncrypt ctx message + cipherBlocks = [B.take 16 (B.drop i cipherText) | i <- [0, 16 .. 64]] + big = B.concat (replicate 819 message) + +spec :: Spec +spec = do + testBlockCipher kats128 (undefined :: Camellia128) + manyBlockTests diff --git a/tests/BlockCipher/DESSpec.hs b/tests/BlockCipher/DESSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/DESSpec.hs @@ -0,0 +1,155 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE ViewPatterns #-} + +module BlockCipher.DESSpec (spec) where + +import BlockCipher +import qualified Crypto.Cipher.DES as DES +import Imports + +vectors_ecb = + -- key plaintext ciphertext + [ KAT_ECB + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x8C\xA6\x4D\xE9\xC1\xB1\x23\xA7" + , KAT_ECB + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x73\x59\xB2\x16\x3E\x4E\xDC\x58" + , KAT_ECB + "\x30\x00\x00\x00\x00\x00\x00\x00" + "\x10\x00\x00\x00\x00\x00\x00\x01" + "\x95\x8E\x6E\x62\x7A\x05\x55\x7B" + , KAT_ECB + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\xF4\x03\x79\xAB\x9E\x0E\xC5\x33" + , KAT_ECB + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x17\x66\x8D\xFC\x72\x92\x53\x2D" + , KAT_ECB + "\x11\x11\x11\x11\x11\x11\x11\x11" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x8A\x5A\xE1\xF8\x1A\xB8\xF2\xDD" + , KAT_ECB + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\x8C\xA6\x4D\xE9\xC1\xB1\x23\xA7" + , KAT_ECB + "\xFE\xDC\xBA\x98\x76\x54\x32\x10" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xED\x39\xD9\x50\xFA\x74\xBC\xC4" + , KAT_ECB + "\x7C\xA1\x10\x45\x4A\x1A\x6E\x57" + "\x01\xA1\xD6\xD0\x39\x77\x67\x42" + "\x69\x0F\x5B\x0D\x9A\x26\x93\x9B" + , KAT_ECB + "\x01\x31\xD9\x61\x9D\xC1\x37\x6E" + "\x5C\xD5\x4C\xA8\x3D\xEF\x57\xDA" + "\x7A\x38\x9D\x10\x35\x4B\xD2\x71" + , KAT_ECB + "\x07\xA1\x13\x3E\x4A\x0B\x26\x86" + "\x02\x48\xD4\x38\x06\xF6\x71\x72" + "\x86\x8E\xBB\x51\xCA\xB4\x59\x9A" + , KAT_ECB + "\x38\x49\x67\x4C\x26\x02\x31\x9E" + "\x51\x45\x4B\x58\x2D\xDF\x44\x0A" + "\x71\x78\x87\x6E\x01\xF1\x9B\x2A" + , KAT_ECB + "\x04\xB9\x15\xBA\x43\xFE\xB5\xB6" + "\x42\xFD\x44\x30\x59\x57\x7F\xA2" + "\xAF\x37\xFB\x42\x1F\x8C\x40\x95" + , KAT_ECB + "\x01\x13\xB9\x70\xFD\x34\xF2\xCE" + "\x05\x9B\x5E\x08\x51\xCF\x14\x3A" + "\x86\xA5\x60\xF1\x0E\xC6\xD8\x5B" + , KAT_ECB + "\x01\x70\xF1\x75\x46\x8F\xB5\xE6" + "\x07\x56\xD8\xE0\x77\x47\x61\xD2" + "\x0C\xD3\xDA\x02\x00\x21\xDC\x09" + , KAT_ECB + "\x43\x29\x7F\xAD\x38\xE3\x73\xFE" + "\x76\x25\x14\xB8\x29\xBF\x48\x6A" + "\xEA\x67\x6B\x2C\xB7\xDB\x2B\x7A" + , KAT_ECB + "\x07\xA7\x13\x70\x45\xDA\x2A\x16" + "\x3B\xDD\x11\x90\x49\x37\x28\x02" + "\xDF\xD6\x4A\x81\x5C\xAF\x1A\x0F" + , KAT_ECB + "\x04\x68\x91\x04\xC2\xFD\x3B\x2F" + "\x26\x95\x5F\x68\x35\xAF\x60\x9A" + "\x5C\x51\x3C\x9C\x48\x86\xC0\x88" + , KAT_ECB + "\x37\xD0\x6B\xB5\x16\xCB\x75\x46" + "\x16\x4D\x5E\x40\x4F\x27\x52\x32" + "\x0A\x2A\xEE\xAE\x3F\xF4\xAB\x77" + , KAT_ECB + "\x1F\x08\x26\x0D\x1A\xC2\x46\x5E" + "\x6B\x05\x6E\x18\x75\x9F\x5C\xCA" + "\xEF\x1B\xF0\x3E\x5D\xFA\x57\x5A" + , KAT_ECB + "\x58\x40\x23\x64\x1A\xBA\x61\x76" + "\x00\x4B\xD6\xEF\x09\x17\x60\x62" + "\x88\xBF\x0D\xB6\xD7\x0D\xEE\x56" + , KAT_ECB + "\x02\x58\x16\x16\x46\x29\xB0\x07" + "\x48\x0D\x39\x00\x6E\xE7\x62\xF2" + "\xA1\xF9\x91\x55\x41\x02\x0B\x56" + , KAT_ECB + "\x49\x79\x3E\xBC\x79\xB3\x25\x8F" + "\x43\x75\x40\xC8\x69\x8F\x3C\xFA" + "\x6F\xBF\x1C\xAF\xCF\xFD\x05\x56" + , KAT_ECB + "\x4F\xB0\x5E\x15\x15\xAB\x73\xA7" + "\x07\x2D\x43\xA0\x77\x07\x52\x92" + "\x2F\x22\xE4\x9B\xAB\x7C\xA1\xAC" + , KAT_ECB + "\x49\xE9\x5D\x6D\x4C\xA2\x29\xBF" + "\x02\xFE\x55\x77\x81\x17\xF1\x2A" + "\x5A\x6B\x61\x2C\xC2\x6C\xCE\x4A" + , KAT_ECB + "\x01\x83\x10\xDC\x40\x9B\x26\xD6" + "\x1D\x9D\x5C\x50\x18\xF7\x28\xC2" + "\x5F\x4C\x03\x8E\xD1\x2B\x2E\x41" + , KAT_ECB + "\x1C\x58\x7F\x1C\x13\x92\x4F\xEF" + "\x30\x55\x32\x28\x6D\x6F\x29\x5A" + "\x63\xFA\xC0\xD0\x34\xD9\xF7\x93" + , KAT_ECB + "\x01\x01\x01\x01\x01\x01\x01\x01" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x61\x7B\x3A\x0C\xE8\xF0\x71\x00" + , KAT_ECB + "\x1F\x1F\x1F\x1F\x0E\x0E\x0E\x0E" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xDB\x95\x86\x05\xF8\xC8\xC6\x06" + , KAT_ECB + "\xE0\xFE\xE0\xFE\xF1\xFE\xF1\xFE" + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\xED\xBF\xD1\xC6\x6C\x29\xCC\xC7" + , KAT_ECB + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x35\x55\x50\xB2\x15\x0E\x24\x51" + , KAT_ECB + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xCA\xAA\xAF\x4D\xEA\xF1\xDB\xAE" + , KAT_ECB + "\x01\x23\x45\x67\x89\xAB\xCD\xEF" + "\x00\x00\x00\x00\x00\x00\x00\x00" + "\xD5\xD4\x4F\xF7\x20\x68\x3D\x0D" + , KAT_ECB + "\xFE\xDC\xBA\x98\x76\x54\x32\x10" + "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" + "\x2A\x2B\xB0\x08\xDF\x97\xC2\xF2" + ] + +kats = defaultKATs{kat_ECB = vectors_ecb} + +spec :: Spec +spec = + modifyMaxSuccess (const 5) $ + testBlockCipher kats (undefined :: DES.DES) diff --git a/tests/BlockCipher/ModesSpec.hs b/tests/BlockCipher/ModesSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/ModesSpec.hs @@ -0,0 +1,103 @@ +{-# LANGUAGE OverloadedStrings #-} + +module BlockCipher.ModesSpec (spec) where + +import Crypto.Cipher.Camellia (Camellia128) +import Crypto.Cipher.DES (DES) +import Crypto.Cipher.Types +import Crypto.Error (throwCryptoError) +import Data.Bits (xor) +import qualified Data.ByteString as B +import Imports + +-- | AES answers for its own modes in C; every other cipher reaches the generic +-- implementations in "Crypto.Cipher.Types.Block". The suite checks that those +-- round trip, which a mode that chains the wrong way does too, and it checks +-- them only at the lengths QuickCheck happens to draw. +-- +-- So write each mode out as its definition states it, and compare. +blocksOf :: Int -> ByteString -> [ByteString] +blocksOf n bs + | B.null bs = [] + | otherwise = let (a, b) = B.splitAt n bs in a : blocksOf n b + +bxor :: ByteString -> ByteString -> ByteString +bxor a b = B.pack (B.zipWith xor a b) + +-- big-endian increment, which is what CTR counts with +incr :: ByteString -> ByteString +incr bs = B.pack (reverse (go (reverse (B.unpack bs)))) + where + go [] = [] + go (w : ws) + | w == 0xff = 0 : go ws + | otherwise = (w + 1) : ws + +refCBCEncrypt + , refCBCDecrypt + , refCFBEncrypt + , refCFBDecrypt + , refCTR + :: BlockCipher c => c -> ByteString -> ByteString -> ByteString +refCBCEncrypt c iv msg = B.concat (go iv (blocksOf (blockSize c) msg)) + where + go _ [] = [] + go v (m : ms) = let o = ecbEncrypt c (bxor v m) in o : go o ms +refCBCDecrypt c iv msg = B.concat (go iv (blocksOf (blockSize c) msg)) + where + go _ [] = [] + go v (m : ms) = bxor v (ecbDecrypt c m) : go m ms +refCFBEncrypt c iv msg = B.concat (go iv (blocksOf (blockSize c) msg)) + where + go _ [] = [] + go v (m : ms) = let o = bxor m (ecbEncrypt c v) in o : go o ms +refCFBDecrypt c iv msg = B.concat (go iv (blocksOf (blockSize c) msg)) + where + go _ [] = [] + go v (m : ms) = bxor m (ecbEncrypt c v) : go m ms +refCTR c iv msg = + B.concat + (zipWith bxor (blocksOf (blockSize c) msg) (map (ecbEncrypt c) (iterate incr iv))) + +modeTests :: BlockCipher c => String -> c -> ByteString -> Spec +modeTests name c iv0 = + describe name $ do + it "CBC encryption is what the definition says" $ + disagree (cbcEncrypt c iv) (refCBCEncrypt c iv0) wholeBlocks `shouldBe` [] + it "CBC decryption is what the definition says" $ + disagree (cbcDecrypt c iv) (refCBCDecrypt c iv0) wholeBlocks `shouldBe` [] + it "CFB encryption is what the definition says" $ + disagree (cfbEncrypt c iv) (refCFBEncrypt c iv0) wholeBlocks `shouldBe` [] + it "CFB decryption is what the definition says" $ + disagree (cfbDecrypt c iv) (refCFBDecrypt c iv0) wholeBlocks `shouldBe` [] + it "CTR is what the definition says, whole blocks or not" $ + disagree (ctrCombine c iv) (refCTR c iv0) everyLength `shouldBe` [] + it "and on a message of 64 KiB" $ do + cbcEncrypt c iv big `shouldBe` refCBCEncrypt c iv0 big + cbcDecrypt c iv big `shouldBe` refCBCDecrypt c iv0 big + ctrCombine c iv big `shouldBe` refCTR c iv0 big + where + bsz = blockSize c + iv = maybe (error "bad IV") id (makeIV iv0) + -- the message, and the lengths to take of it + message = B.concat (replicate 4 (B.pack (map fromIntegral [1 .. 255 :: Int]))) + wholeBlocks = [bsz * i | i <- [0 .. 20]] + everyLength = [0 .. 40] + big = B.concat (replicate 256 message) + disagree lib ref lens = + [n | n <- lens, let m = B.take n message, lib m /= ref m] + +spec :: Spec +spec = do + modeTests + "DES" + (throwCryptoError (cipherInit desKey) :: DES) + (B.replicate 8 0x42) + modeTests + "Camellia128" + (throwCryptoError (cipherInit camKey) :: Camellia128) + (B.replicate 16 0x42) + where + desKey = "\x01\x23\x45\x67\x89\xab\xcd\xef" :: ByteString + camKey = + "\x01\x23\x45\x67\x89\xab\xcd\xef\xfe\xdc\xba\x98\x76\x54\x32\x10" :: ByteString diff --git a/tests/BlockCipher/TripleDESSpec.hs b/tests/BlockCipher/TripleDESSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/TripleDESSpec.hs @@ -0,0 +1,70 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE ViewPatterns #-} + +module BlockCipher.TripleDESSpec (spec) where + +import BlockCipher +import qualified Crypto.Cipher.DES as DES +import qualified Crypto.Cipher.TripleDES as TripleDES +import Crypto.Cipher.Types +import Crypto.Error (throwCryptoError) +import qualified Data.ByteString as B +import Imports + +kats = defaultKATs + +key1, key2, key3, message :: ByteString +key1 = "\x01\x23\x45\x67\x89\xab\xcd\xef" +key2 = "\xfe\xdc\xba\x98\x76\x54\x32\x10" +key3 = "\x13\x34\x57\x79\x9b\xbc\xdf\xf1" +message = "\x4e\x6f\x77\x20\x69\x73\x20\x74\x68\x65\x20\x74\x69\x6d\x65\x20" + +des :: ByteString -> DES.DES +des k = throwCryptoError (cipherInit k) + +cipher :: BlockCipher c => ByteString -> c +cipher k = throwCryptoError (cipherInit k) + +-- | What the three stage constructions are, said in terms of DES itself: the +-- keys are used in the order and the directions their names describe, and +-- three stages under one key are the one stage the middle one undoes. +-- +-- The suite had only round trips for these, which are equally happy with the +-- stages in the wrong order. +compositionTests :: Spec +compositionTests = + describe "composition" $ do + it "EEE3 is E,E,E under the three keys" $ + ecbEncrypt (cipher k123 :: TripleDES.DES_EEE3) message + `shouldBe` e key3 (e key2 (e key1 message)) + it "EDE3 is E,D,E under the three keys" $ + ecbEncrypt (cipher k123 :: TripleDES.DES_EDE3) message + `shouldBe` e key3 (d key2 (e key1 message)) + it "EEE2 is E,E,E with the first key again" $ + ecbEncrypt (cipher k12 :: TripleDES.DES_EEE2) message + `shouldBe` e key1 (e key2 (e key1 message)) + it "EDE2 is E,D,E with the first key again" $ + ecbEncrypt (cipher k12 :: TripleDES.DES_EDE2) message + `shouldBe` e key1 (d key2 (e key1 message)) + it "decryption undoes each of them" $ do + back (cipher k123 :: TripleDES.DES_EEE3) `shouldBe` message + back (cipher k123 :: TripleDES.DES_EDE3) `shouldBe` message + back (cipher k12 :: TripleDES.DES_EEE2) `shouldBe` message + back (cipher k12 :: TripleDES.DES_EDE2) `shouldBe` message + it "EDE under one key repeated is DES" $ do + ecbEncrypt (cipher (B.concat [key1, key1, key1]) :: TripleDES.DES_EDE3) message + `shouldBe` e key1 message + ecbEncrypt (cipher (B.concat [key1, key1]) :: TripleDES.DES_EDE2) message + `shouldBe` e key1 message + where + k123 = B.concat [key1, key2, key3] + k12 = B.concat [key1, key2] + e k m = ecbEncrypt (des k) m + d k m = ecbDecrypt (des k) m + back c = ecbDecrypt c (ecbEncrypt c message) + +spec :: Spec +spec = do + modifyMaxSuccess (const 5) $ + testBlockCipher kats (undefined :: TripleDES.DES_EEE3) + compositionTests diff --git a/tests/BlockCipher/TwofishSpec.hs b/tests/BlockCipher/TwofishSpec.hs new file mode 100644 --- /dev/null +++ b/tests/BlockCipher/TwofishSpec.hs @@ -0,0 +1,417 @@ +module BlockCipher.TwofishSpec (spec) where + +import BlockCipher +import Imports + +import Control.Exception (evaluate) +import Crypto.Cipher.Twofish +import Crypto.Cipher.Types +import Crypto.Error (throwCryptoError) +import qualified Data.ByteString as B + +vectors_twofish128 = + [ KAT_ECB + (B.replicate 16 0x00) + (B.replicate 16 0x00) + ( B.pack + [ 0x9F + , 0x58 + , 0x9F + , 0x5C + , 0xF6 + , 0x12 + , 0x2C + , 0x32 + , 0xB6 + , 0xBF + , 0xEC + , 0x2F + , 0x2A + , 0xE8 + , 0xC3 + , 0x5A + ] + ) + , KAT_ECB + ( B.pack + [ 0x9F + , 0x58 + , 0x9F + , 0x5C + , 0xF6 + , 0x12 + , 0x2C + , 0x32 + , 0xB6 + , 0xBF + , 0xEC + , 0x2F + , 0x2A + , 0xE8 + , 0xC3 + , 0x5A + ] + ) + ( B.pack + [ 0xD4 + , 0x91 + , 0xDB + , 0x16 + , 0xE7 + , 0xB1 + , 0xC3 + , 0x9E + , 0x86 + , 0xCB + , 0x08 + , 0x6B + , 0x78 + , 0x9F + , 0x54 + , 0x19 + ] + ) + ( B.pack + [ 0x01 + , 0x9F + , 0x98 + , 0x09 + , 0xDE + , 0x17 + , 0x11 + , 0x85 + , 0x8F + , 0xAA + , 0xC3 + , 0xA3 + , 0xBA + , 0x20 + , 0xFB + , 0xC3 + ] + ) + ] + +vectors_twofish192 = + [ KAT_ECB + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xAB + , 0xCD + , 0xEF + , 0xFE + , 0xDC + , 0xBA + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + , 0x00 + , 0x11 + , 0x22 + , 0x33 + , 0x44 + , 0x55 + , 0x66 + , 0x77 + ] + ) + ( B.pack + [ 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + ] + ) + ( B.pack + [ 0xCF + , 0xD1 + , 0xD2 + , 0xE5 + , 0xA9 + , 0xBE + , 0x9C + , 0xDF + , 0x50 + , 0x1F + , 0x13 + , 0xB8 + , 0x92 + , 0xBD + , 0x22 + , 0x48 + ] + ) + , KAT_ECB + ( B.pack + [ 0x88 + , 0xB2 + , 0xB2 + , 0x70 + , 0x6B + , 0x10 + , 0x5E + , 0x36 + , 0xB4 + , 0x46 + , 0xBB + , 0x6D + , 0x73 + , 0x1A + , 0x1E + , 0x88 + , 0xEF + , 0xA7 + , 0x1F + , 0x78 + , 0x89 + , 0x65 + , 0xBD + , 0x44 + ] + ) + ( B.pack + [ 0x39 + , 0xDA + , 0x69 + , 0xD6 + , 0xBA + , 0x49 + , 0x97 + , 0xD5 + , 0x85 + , 0xB6 + , 0xDC + , 0x07 + , 0x3C + , 0xA3 + , 0x41 + , 0xB2 + ] + ) + ( B.pack + [ 0x18 + , 0x2B + , 0x02 + , 0xD8 + , 0x14 + , 0x97 + , 0xEA + , 0x45 + , 0xF9 + , 0xDA + , 0xAC + , 0xDC + , 0x29 + , 0x19 + , 0x3A + , 0x65 + ] + ) + ] + +vectors_twofish256 = + [ KAT_ECB + ( B.pack + [ 0x01 + , 0x23 + , 0x45 + , 0x67 + , 0x89 + , 0xAB + , 0xCD + , 0xEF + , 0xFE + , 0xDC + , 0xBA + , 0x98 + , 0x76 + , 0x54 + , 0x32 + , 0x10 + , 0x00 + , 0x11 + , 0x22 + , 0x33 + , 0x44 + , 0x55 + , 0x66 + , 0x77 + , 0x88 + , 0x99 + , 0xAA + , 0xBB + , 0xCC + , 0xDD + , 0xEE + , 0xFF + ] + ) + ( B.pack + [ 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + , 0x00 + ] + ) + ( B.pack + [ 0x37 + , 0x52 + , 0x7B + , 0xE0 + , 0x05 + , 0x23 + , 0x34 + , 0xB8 + , 0x9F + , 0x0C + , 0xFC + , 0xCA + , 0xE8 + , 0x7C + , 0xFA + , 0x20 + ] + ) + , KAT_ECB + ( B.pack + [ 0xD4 + , 0x3B + , 0xB7 + , 0x55 + , 0x6E + , 0xA3 + , 0x2E + , 0x46 + , 0xF2 + , 0xA2 + , 0x82 + , 0xB7 + , 0xD4 + , 0x5B + , 0x4E + , 0x0D + , 0x57 + , 0xFF + , 0x73 + , 0x9D + , 0x4D + , 0xC9 + , 0x2C + , 0x1B + , 0xD7 + , 0xFC + , 0x01 + , 0x70 + , 0x0C + , 0xC8 + , 0x21 + , 0x6F + ] + ) + ( B.pack + [ 0x90 + , 0xAF + , 0xE9 + , 0x1B + , 0xB2 + , 0x88 + , 0x54 + , 0x4F + , 0x2C + , 0x32 + , 0xDC + , 0x23 + , 0x9B + , 0x26 + , 0x35 + , 0xE6 + ] + ) + ( B.pack + [ 0x6C + , 0xB4 + , 0x56 + , 0x1C + , 0x40 + , 0xBF + , 0x0A + , 0x97 + , 0x05 + , 0x93 + , 0x1C + , 0xB6 + , 0xD4 + , 0x08 + , 0xE7 + , 0xFA + ] + ) + ] + +kats128 = defaultKATs{kat_ECB = vectors_twofish128} +kats192 = defaultKATs{kat_ECB = vectors_twofish192} +kats256 = defaultKATs{kat_ECB = vectors_twofish256} + +-- | ECB is the block operation applied to each block and nothing else, so a +-- message of several blocks is the blocks encrypted one at a time and put back +-- together. The vectors above are all one block long, and the loop that walks +-- the blocks is about to be rewritten. +manyBlockTests :: Spec +manyBlockTests = + describe "several blocks" $ do + it "ECB of a message is ECB of its blocks" $ + ecbEncrypt ctx message `shouldBe` B.concat (map (ecbEncrypt ctx) blocks) + it "and the same going back" $ + ecbDecrypt ctx cipherText + `shouldBe` B.concat (map (ecbDecrypt ctx) cipherBlocks) + it "a message of 64 KiB still decrypts to itself" $ + ecbDecrypt ctx (ecbEncrypt ctx big) `shouldBe` big + it "a message that is not whole blocks is refused" $ + -- it used to come back longer than it went in: the short block was + -- read as though the bytes it does not have were zero + evaluate (B.length (ecbEncrypt ctx (B.take 20 message))) + `shouldThrow` anyErrorCall + where + ctx = throwCryptoError (cipherInit (B.replicate 16 0x2b)) :: Twofish128 + message = B.pack (map fromIntegral [1 .. 80 :: Int]) + blocks = [B.take 16 (B.drop i message) | i <- [0, 16 .. 64]] + cipherText = ecbEncrypt ctx message + cipherBlocks = [B.take 16 (B.drop i cipherText) | i <- [0, 16 .. 64]] + big = B.concat (replicate 819 message) + +spec :: Spec +spec = do + manyBlockTests + testBlockCipher kats128 (undefined :: Twofish128) + testBlockCipher kats192 (undefined :: Twofish192) + testBlockCipher kats256 (undefined :: Twofish256) diff --git a/tests/ChaCha.hs b/tests/ChaCha.hs deleted file mode 100644 --- a/tests/ChaCha.hs +++ /dev/null @@ -1,164 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module ChaCha (tests) where - -import qualified Crypto.Cipher.ChaCha as ChaCha -import Imports - -import qualified Data.ByteString as B - -b8_128_k0_i0 = - "\xe2\x8a\x5f\xa4\xa6\x7f\x8c\x5d\xef\xed\x3e\x6f\xb7\x30\x34\x86\xaa\x84\x27\xd3\x14\x19\xa7\x29\x57\x2d\x77\x79\x53\x49\x11\x20\xb6\x4a\xb8\xe7\x2b\x8d\xeb\x85\xcd\x6a\xea\x7c\xb6\x08\x9a\x10\x18\x24\xbe\xeb\x08\x81\x4a\x42\x8a\xab\x1f\xa2\xc8\x16\x08\x1b\x8a\x26\xaf\x44\x8a\x1b\xa9\x06\x36\x8f\xd8\xc8\x38\x31\xc1\x8c\xec\x8c\xed\x81\x1a\x02\x8e\x67\x5b\x8d\x2b\xe8\xfc\xe0\x81\x16\x5c\xea\xe9\xf1\xd1\xb7\xa9\x75\x49\x77\x49\x48\x05\x69\xce\xb8\x3d\xe6\xa0\xa5\x87\xd4\x98\x4f\x19\x92\x5f\x5d\x33\x8e\x43\x0d" - -b12_128_k0_i0 = - "\xe1\x04\x7b\xa9\x47\x6b\xf8\xff\x31\x2c\x01\xb4\x34\x5a\x7d\x8c\xa5\x79\x2b\x0a\xd4\x67\x31\x3f\x1d\xc4\x12\xb5\xfd\xce\x32\x41\x0d\xea\x8b\x68\xbd\x77\x4c\x36\xa9\x20\xf0\x92\xa0\x4d\x3f\x95\x27\x4f\xbe\xff\x97\xbc\x84\x91\xfc\xef\x37\xf8\x59\x70\xb4\x50\x1d\x43\xb6\x1a\x8f\x7e\x19\xfc\xed\xde\xf3\x68\xae\x6b\xfb\x11\x10\x1b\xd9\xfd\x3e\x4d\x12\x7d\xe3\x0d\xb2\xdb\x1b\x47\x2e\x76\x42\x68\x03\xa4\x5e\x15\xb9\x62\x75\x19\x86\xef\x1d\x9d\x50\xf5\x98\xa5\xdc\xdc\x9f\xa5\x29\xa2\x83\x57\x99\x1e\x78\x4e\xa2\x0f" - -b20_128_k0_i0 = - "\x89\x67\x09\x52\x60\x83\x64\xfd\x00\xb2\xf9\x09\x36\xf0\x31\xc8\xe7\x56\xe1\x5d\xba\x04\xb8\x49\x3d\x00\x42\x92\x59\xb2\x0f\x46\xcc\x04\xf1\x11\x24\x6b\x6c\x2c\xe0\x66\xbe\x3b\xfb\x32\xd9\xaa\x0f\xdd\xfb\xc1\x21\x23\xd4\xb9\xe4\x4f\x34\xdc\xa0\x5a\x10\x3f\x6c\xd1\x35\xc2\x87\x8c\x83\x2b\x58\x96\xb1\x34\xf6\x14\x2a\x9d\x4d\x8d\x0d\x8f\x10\x26\xd2\x0a\x0a\x81\x51\x2c\xbc\xe6\xe9\x75\x8a\x71\x43\xd0\x21\x97\x80\x22\xa3\x84\x14\x1a\x80\xce\xa3\x06\x2f\x41\xf6\x7a\x75\x2e\x66\xad\x34\x11\x98\x4c\x78\x7e\x30\xad" - -b8_256_k0_i0 = - "\x3e\x00\xef\x2f\x89\x5f\x40\xd6\x7f\x5b\xb8\xe8\x1f\x09\xa5\xa1\x2c\x84\x0e\xc3\xce\x9a\x7f\x3b\x18\x1b\xe1\x88\xef\x71\x1a\x1e\x98\x4c\xe1\x72\xb9\x21\x6f\x41\x9f\x44\x53\x67\x45\x6d\x56\x19\x31\x4a\x42\xa3\xda\x86\xb0\x01\x38\x7b\xfd\xb8\x0e\x0c\xfe\x42\xd2\xae\xfa\x0d\xea\xa5\xc1\x51\xbf\x0a\xdb\x6c\x01\xf2\xa5\xad\xc0\xfd\x58\x12\x59\xf9\xa2\xaa\xdc\xf2\x0f\x8f\xd5\x66\xa2\x6b\x50\x32\xec\x38\xbb\xc5\xda\x98\xee\x0c\x6f\x56\x8b\x87\x2a\x65\xa0\x8a\xbf\x25\x1d\xeb\x21\xbb\x4b\x56\xe5\xd8\x82\x1e\x68\xaa" - -b12_256_k0_i0 = - "\x9b\xf4\x9a\x6a\x07\x55\xf9\x53\x81\x1f\xce\x12\x5f\x26\x83\xd5\x04\x29\xc3\xbb\x49\xe0\x74\x14\x7e\x00\x89\xa5\x2e\xae\x15\x5f\x05\x64\xf8\x79\xd2\x7a\xe3\xc0\x2c\xe8\x28\x34\xac\xfa\x8c\x79\x3a\x62\x9f\x2c\xa0\xde\x69\x19\x61\x0b\xe8\x2f\x41\x13\x26\xbe\x0b\xd5\x88\x41\x20\x3e\x74\xfe\x86\xfc\x71\x33\x8c\xe0\x17\x3d\xc6\x28\xeb\xb7\x19\xbd\xcb\xcc\x15\x15\x85\x21\x4c\xc0\x89\xb4\x42\x25\x8d\xcd\xa1\x4c\xf1\x11\xc6\x02\xb8\x97\x1b\x8c\xc8\x43\xe9\x1e\x46\xca\x90\x51\x51\xc0\x27\x44\xa6\xb0\x17\xe6\x93\x16" - -b20_256_k0_i0 = - "\x76\xb8\xe0\xad\xa0\xf1\x3d\x90\x40\x5d\x6a\xe5\x53\x86\xbd\x28\xbd\xd2\x19\xb8\xa0\x8d\xed\x1a\xa8\x36\xef\xcc\x8b\x77\x0d\xc7\xda\x41\x59\x7c\x51\x57\x48\x8d\x77\x24\xe0\x3f\xb8\xd8\x4a\x37\x6a\x43\xb8\xf4\x15\x18\xa1\x1c\xc3\x87\xb6\x69\xb2\xee\x65\x86\x9f\x07\xe7\xbe\x55\x51\x38\x7a\x98\xba\x97\x7c\x73\x2d\x08\x0d\xcb\x0f\x29\xa0\x48\xe3\x65\x69\x12\xc6\x53\x3e\x32\xee\x7a\xed\x29\xb7\x21\x76\x9c\xe6\x4e\x43\xd5\x71\x33\xb0\x74\xd8\x39\xd5\x31\xed\x1f\x28\x51\x0a\xfb\x45\xac\xe1\x0a\x1f\x4b\x79\x4d\x6f" - --- XChaCha20 test vector from RFC draft: https://datatracker.ietf.org/doc/html/draft-arciszewski-xchacha - -xChaCha20_ExampleKAT = expected @=? fst (ChaCha.combine initState plaintext) - where - iv = B.pack $ [0x40 .. 0x56] ++ [0x58] - key = B.pack [0x80 .. 0x9f] - initState = ChaCha.initializeX 20 key iv - plaintext :: B.ByteString - plaintext = - "The dhole (pronounced \"dole\") is also known as the Asiatic wild dog, red dog, and whistling dog. It is about the size of a German shepherd but looks more like a long-legged fox. This highly elusive and skilled jumper is classified with wolves, coyotes, jackals, and foxes in the taxonomic family Canidae." - expected :: B.ByteString - expected = - "\x45\x59\xab\xba\x4e\x48\xc1\x61\x02\xe8\xbb\x2c\x05\xe6\x94\x7f\x50\xa7\x86\xde\x16\x2f\x9b\x0b\x7e\x59\x2a\x9b\x53\xd0\xd4\xe9\x8d\x8d\x64\x10\xd5\x40\xa1\xa6\x37\x5b\x26\xd8\x0d\xac\xe4\xfa\xb5\x23\x84\xc7\x31\xac\xbf\x16\xa5\x92\x3c\x0c\x48\xd3\x57\x5d\x4d\x0d\x2c\x67\x3b\x66\x6f\xaa\x73\x10\x61\x27\x77\x01\x09\x3a\x6b\xf7\xa1\x58\xa8\x86\x42\x92\xa4\x1c\x48\xe3\xa9\xb4\xc0\xda\xec\xe0\xf8\xd9\x8d\x0d\x7e\x05\xb3\x7a\x30\x7b\xbb\x66\x33\x31\x64\xec\x9e\x1b\x24\xea\x0d\x6c\x3f\xfd\xdc\xec\x4f\x68\xe7\x44\x30\x56\x19\x3a\x03\xc8\x10\xe1\x13\x44\xca\x06\xd8\xed\x8a\x2b\xfb\x1e\x8d\x48\xcf\xa6\xbc\x0e\xb4\xe2\x46\x4b\x74\x81\x42\x40\x7c\x9f\x43\x1a\xee\x76\x99\x60\xe1\x5b\xa8\xb9\x68\x90\x46\x6e\xf2\x45\x75\x99\x85\x23\x85\xc6\x61\xf7\x52\xce\x20\xf9\xda\x0c\x09\xab\x6b\x19\xdf\x74\xe7\x6a\x95\x96\x74\x46\xf8\xd0\xfd\x41\x5e\x7b\xee\x2a\x12\xa1\x14\xc2\x0e\xb5\x29\x2a\xe7\xa3\x49\xae\x57\x78\x20\xd5\x52\x0a\x1f\x3f\xb6\x2a\x17\xce\x6a\x7e\x68\xfa\x7c\x79\x11\x1d\x88\x60\x92\x0b\xc0\x48\xef\x43\xfe\x84\x48\x6c\xcb\x87\xc2\x5f\x0a\xe0\x45\xf0\xcc\xe1\xe7\x98\x9a\x9a\xa2\x20\xa2\x8b\xdd\x48\x27\xe7\x51\xa2\x4a\x6d\x5c\x62\xd7\x90\xa6\x63\x93\xb9\x31\x11\xc1\xa5\x5d\xd7\x42\x1a\x10\x18\x49\x74\xc7\xc5" - -rfc8439A2_1 = cipher @=? cipher' - where - key :: ByteString - key = - "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - nonce :: ByteString - nonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - plain :: ByteString - plain = - "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - cipher :: ByteString - cipher = - "\x76\xb8\xe0\xad\xa0\xf1\x3d\x90\x40\x5d\x6a\xe5\x53\x86\xbd\x28\xbd\xd2\x19\xb8\xa0\x8d\xed\x1a\xa8\x36\xef\xcc\x8b\x77\x0d\xc7\xda\x41\x59\x7c\x51\x57\x48\x8d\x77\x24\xe0\x3f\xb8\xd8\x4a\x37\x6a\x43\xb8\xf4\x15\x18\xa1\x1c\xc3\x87\xb6\x69\xb2\xee\x65\x86" - cipher' = fst $ ChaCha.combine (ChaCha.initialize 20 key nonce) plain - -rfc8439A2_2 = cipher @=? cipher' - where - key :: ByteString - key = - "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01" - nonce :: ByteString - nonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02" - plain :: ByteString - plain = - "\x41\x6e\x79\x20\x73\x75\x62\x6d\x69\x73\x73\x69\x6f\x6e\x20\x74\x6f\x20\x74\x68\x65\x20\x49\x45\x54\x46\x20\x69\x6e\x74\x65\x6e\x64\x65\x64\x20\x62\x79\x20\x74\x68\x65\x20\x43\x6f\x6e\x74\x72\x69\x62\x75\x74\x6f\x72\x20\x66\x6f\x72\x20\x70\x75\x62\x6c\x69\x63\x61\x74\x69\x6f\x6e\x20\x61\x73\x20\x61\x6c\x6c\x20\x6f\x72\x20\x70\x61\x72\x74\x20\x6f\x66\x20\x61\x6e\x20\x49\x45\x54\x46\x20\x49\x6e\x74\x65\x72\x6e\x65\x74\x2d\x44\x72\x61\x66\x74\x20\x6f\x72\x20\x52\x46\x43\x20\x61\x6e\x64\x20\x61\x6e\x79\x20\x73\x74\x61\x74\x65\x6d\x65\x6e\x74\x20\x6d\x61\x64\x65\x20\x77\x69\x74\x68\x69\x6e\x20\x74\x68\x65\x20\x63\x6f\x6e\x74\x65\x78\x74\x20\x6f\x66\x20\x61\x6e\x20\x49\x45\x54\x46\x20\x61\x63\x74\x69\x76\x69\x74\x79\x20\x69\x73\x20\x63\x6f\x6e\x73\x69\x64\x65\x72\x65\x64\x20\x61\x6e\x20\x22\x49\x45\x54\x46\x20\x43\x6f\x6e\x74\x72\x69\x62\x75\x74\x69\x6f\x6e\x22\x2e\x20\x53\x75\x63\x68\x20\x73\x74\x61\x74\x65\x6d\x65\x6e\x74\x73\x20\x69\x6e\x63\x6c\x75\x64\x65\x20\x6f\x72\x61\x6c\x20\x73\x74\x61\x74\x65\x6d\x65\x6e\x74\x73\x20\x69\x6e\x20\x49\x45\x54\x46\x20\x73\x65\x73\x73\x69\x6f\x6e\x73\x2c\x20\x61\x73\x20\x77\x65\x6c\x6c\x20\x61\x73\x20\x77\x72\x69\x74\x74\x65\x6e\x20\x61\x6e\x64\x20\x65\x6c\x65\x63\x74\x72\x6f\x6e\x69\x63\x20\x63\x6f\x6d\x6d\x75\x6e\x69\x63\x61\x74\x69\x6f\x6e\x73\x20\x6d\x61\x64\x65\x20\x61\x74\x20\x61\x6e\x79\x20\x74\x69\x6d\x65\x20\x6f\x72\x20\x70\x6c\x61\x63\x65\x2c\x20\x77\x68\x69\x63\x68\x20\x61\x72\x65\x20\x61\x64\x64\x72\x65\x73\x73\x65\x64\x20\x74\x6f" - cipher :: ByteString - cipher = - "\xa3\xfb\xf0\x7d\xf3\xfa\x2f\xde\x4f\x37\x6c\xa2\x3e\x82\x73\x70\x41\x60\x5d\x9f\x4f\x4f\x57\xbd\x8c\xff\x2c\x1d\x4b\x79\x55\xec\x2a\x97\x94\x8b\xd3\x72\x29\x15\xc8\xf3\xd3\x37\xf7\xd3\x70\x05\x0e\x9e\x96\xd6\x47\xb7\xc3\x9f\x56\xe0\x31\xca\x5e\xb6\x25\x0d\x40\x42\xe0\x27\x85\xec\xec\xfa\x4b\x4b\xb5\xe8\xea\xd0\x44\x0e\x20\xb6\xe8\xdb\x09\xd8\x81\xa7\xc6\x13\x2f\x42\x0e\x52\x79\x50\x42\xbd\xfa\x77\x73\xd8\xa9\x05\x14\x47\xb3\x29\x1c\xe1\x41\x1c\x68\x04\x65\x55\x2a\xa6\xc4\x05\xb7\x76\x4d\x5e\x87\xbe\xa8\x5a\xd0\x0f\x84\x49\xed\x8f\x72\xd0\xd6\x62\xab\x05\x26\x91\xca\x66\x42\x4b\xc8\x6d\x2d\xf8\x0e\xa4\x1f\x43\xab\xf9\x37\xd3\x25\x9d\xc4\xb2\xd0\xdf\xb4\x8a\x6c\x91\x39\xdd\xd7\xf7\x69\x66\xe9\x28\xe6\x35\x55\x3b\xa7\x6c\x5c\x87\x9d\x7b\x35\xd4\x9e\xb2\xe6\x2b\x08\x71\xcd\xac\x63\x89\x39\xe2\x5e\x8a\x1e\x0e\xf9\xd5\x28\x0f\xa8\xca\x32\x8b\x35\x1c\x3c\x76\x59\x89\xcb\xcf\x3d\xaa\x8b\x6c\xcc\x3a\xaf\x9f\x39\x79\xc9\x2b\x37\x20\xfc\x88\xdc\x95\xed\x84\xa1\xbe\x05\x9c\x64\x99\xb9\xfd\xa2\x36\xe7\xe8\x18\xb0\x4b\x0b\xc3\x9c\x1e\x87\x6b\x19\x3b\xfe\x55\x69\x75\x3f\x88\x12\x8c\xc0\x8a\xaa\x9b\x63\xd1\xa1\x6f\x80\xef\x25\x54\xd7\x18\x9c\x41\x1f\x58\x69\xca\x52\xc5\xb8\x3f\xa3\x6f\xf2\x16\xb9\xc1\xd3\x00\x62\xbe\xbc\xfd\x2d\xc5\xbc\xe0\x91\x19\x34\xfd\xa7\x9a\x86\xf6\xe6\x98\xce\xd7\x59\xc3\xff\x9b\x64\x77\x33\x8f\x3d\xa4\xf9\xcd\x85\x14\xea\x99\x82\xcc\xaf\xb3\x41\xb2\x38\x4d\xd9\x02\xf3\xd1\xab\x7a\xc6\x1d\xd2\x9c\x6f\x21\xba\x5b\x86\x2f\x37\x30\xe3\x7c\xfd\xc4\xfd\x80\x6c\x22\xf2\x21" - cipher' = - fst $ - ChaCha.combine (ChaCha.setCounter32 1 (ChaCha.initialize 20 key nonce)) plain - -rfc8439A2_3 = cipher @=? cipher' - where - key :: ByteString - key = - "\x1c\x92\x40\xa5\xeb\x55\xd3\x8a\xf3\x33\x88\x86\x04\xf6\xb5\xf0\x47\x39\x17\xc1\x40\x2b\x80\x09\x9d\xca\x5c\xbc\x20\x70\x75\xc0" - nonce :: ByteString - nonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02" - plain :: ByteString - plain = - "\x27\x54\x77\x61\x73\x20\x62\x72\x69\x6c\x6c\x69\x67\x2c\x20\x61\x6e\x64\x20\x74\x68\x65\x20\x73\x6c\x69\x74\x68\x79\x20\x74\x6f\x76\x65\x73\x0a\x44\x69\x64\x20\x67\x79\x72\x65\x20\x61\x6e\x64\x20\x67\x69\x6d\x62\x6c\x65\x20\x69\x6e\x20\x74\x68\x65\x20\x77\x61\x62\x65\x3a\x0a\x41\x6c\x6c\x20\x6d\x69\x6d\x73\x79\x20\x77\x65\x72\x65\x20\x74\x68\x65\x20\x62\x6f\x72\x6f\x67\x6f\x76\x65\x73\x2c\x0a\x41\x6e\x64\x20\x74\x68\x65\x20\x6d\x6f\x6d\x65\x20\x72\x61\x74\x68\x73\x20\x6f\x75\x74\x67\x72\x61\x62\x65\x2e" - cipher :: ByteString - cipher = - "\x62\xe6\x34\x7f\x95\xed\x87\xa4\x5f\xfa\xe7\x42\x6f\x27\xa1\xdf\x5f\xb6\x91\x10\x04\x4c\x0d\x73\x11\x8e\xff\xa9\x5b\x01\xe5\xcf\x16\x6d\x3d\xf2\xd7\x21\xca\xf9\xb2\x1e\x5f\xb1\x4c\x61\x68\x71\xfd\x84\xc5\x4f\x9d\x65\xb2\x83\x19\x6c\x7f\xe4\xf6\x05\x53\xeb\xf3\x9c\x64\x02\xc4\x22\x34\xe3\x2a\x35\x6b\x3e\x76\x43\x12\xa6\x1a\x55\x32\x05\x57\x16\xea\xd6\x96\x25\x68\xf8\x7d\x3f\x3f\x77\x04\xc6\xa8\xd1\xbc\xd1\xbf\x4d\x50\xd6\x15\x4b\x6d\xa7\x31\xb1\x87\xb5\x8d\xfd\x72\x8a\xfa\x36\x75\x7a\x79\x7a\xc1\x88\xd1" - cipher' = - fst $ - ChaCha.combine (ChaCha.setCounter32 42 (ChaCha.initialize 20 key nonce)) plain - -data Vector - = Vector - Int -- rounds - ByteString -- key - ByteString -- nonce - deriving (Show, Eq) - -instance Arbitrary Vector where - arbitrary = Vector 20 <$> arbitraryBS 16 <*> arbitraryBS 12 - -tests = - testGroup - "ChaCha" - [ testCase "8-128-K0-I0" (chachaRunSimple b8_128_k0_i0 8 16 8) - , testCase "12-128-K0-I0" (chachaRunSimple b12_128_k0_i0 12 16 8) - , testCase "20-128-K0-I0" (chachaRunSimple b20_128_k0_i0 20 16 8) - , testCase "8-256-K0-I0" (chachaRunSimple b8_256_k0_i0 8 32 8) - , testCase "12-256-K0-I0" (chachaRunSimple b12_256_k0_i0 12 32 8) - , testCase "20-256-K0-I0" (chachaRunSimple b20_256_k0_i0 20 32 8) - , testCase "XChaCha20 example KAT" xChaCha20_ExampleKAT - , testCase "RFC 8439 A2 #1 ChaCha20" rfc8439A2_1 - , testCase "RFC 8439 A2 #2 ChaCha20" rfc8439A2_2 - , testCase "RFC 8439 A2 #3 ChaCha20" rfc8439A2_3 - , testProperty "generate-combine" chachaGenerateCombine - , testProperty "chunking-generate" chachaGenerateChunks - , testProperty "chunking-combine" chachaCombineChunks - ] - where - chachaRunSimple expected rounds klen nonceLen = - let chacha = ChaCha.initialize rounds (B.replicate klen 0) (B.replicate nonceLen 0) - in expected @=? fst (ChaCha.generate chacha (B.length expected)) - - chachaGenerateChunks :: ChunkingLen -> Vector -> Bool - chachaGenerateChunks (ChunkingLen ckLen) (Vector rounds key iv) = - let initChaCha = ChaCha.initialize rounds key iv - nbBytes = 1048 - (expected, _) = ChaCha.generate initChaCha nbBytes - chunks = loop nbBytes ckLen initChaCha - in expected `propertyEq` B.concat chunks - where - loop n [] chacha = loop n ckLen chacha - loop 0 _ _ = [] - loop n (x : xs) chacha = - let len = min x n - (c, next) = ChaCha.generate chacha len - in c : loop (n - len) xs next - - chachaGenerateCombine :: ChunkingLen0_127 -> Vector -> Int0_2901 -> Bool - chachaGenerateCombine (ChunkingLen0_127 ckLen) (Vector rounds key iv) (Int0_2901 nbBytes) = - let initChaCha = ChaCha.initialize rounds key iv - in loop nbBytes ckLen initChaCha - where - loop n [] chacha = loop n ckLen chacha - loop 0 _ _ = True - loop n (x : xs) chacha = - let len = min x n - (c1, next) = ChaCha.generate chacha len - (c2, _) = ChaCha.combine chacha (B.replicate len 0) - in if c1 == c2 then loop (n - len) xs next else False - - chachaCombineChunks :: ChunkingLen0_127 -> Vector -> ArbitraryBS0_2901 -> Bool - chachaCombineChunks (ChunkingLen0_127 ckLen) (Vector rounds key iv) (ArbitraryBS0_2901 wholebs) = - let initChaCha = ChaCha.initialize rounds key iv - (expected, _) = ChaCha.combine initChaCha wholebs - chunks = loop wholebs ckLen initChaCha - in expected `propertyEq` B.concat chunks - where - loop bs [] chacha = loop bs ckLen chacha - loop bs (x : xs) chacha - | B.null bs = [] - | otherwise = - let (bs1, bs2) = B.splitAt (min x (B.length bs)) bs - (c, next) = ChaCha.combine chacha bs1 - in c : loop bs2 xs next diff --git a/tests/ChaChaPoly1305.hs b/tests/ChaChaPoly1305.hs deleted file mode 100644 --- a/tests/ChaChaPoly1305.hs +++ /dev/null @@ -1,169 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module ChaChaPoly1305 where - -import qualified Crypto.Cipher.ChaChaPoly1305 as CP -import Crypto.Cipher.Types -import Crypto.Error -import Imports -import Poly1305 () - -import qualified Data.ByteArray as B (convert) -import qualified Data.ByteString as B - -plaintext - , aad - , key - , iv - , ivX - , ciphertext - , ciphertextX - , tag - , tagX - , nonce1 - , nonce2 - , nonce3 - , nonce4 - , nonce5 - , nonce6 - , nonce7 - , nonce8 - , nonce9 - , nonce10 - :: B.ByteString -plaintext = - "Ladies and Gentlemen of the class of '99: If I could offer you only one tip for the future, sunscreen would be it." -aad = "\x50\x51\x52\x53\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7" -key = - "\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f" -iv = "\x40\x41\x42\x43\x44\x45\x46\x47" -ivX = B.pack [0x40 .. 0x57] -constant = "\x07\x00\x00\x00" -ciphertext = - "\xd3\x1a\x8d\x34\x64\x8e\x60\xdb\x7b\x86\xaf\xbc\x53\xef\x7e\xc2\xa4\xad\xed\x51\x29\x6e\x08\xfe\xa9\xe2\xb5\xa7\x36\xee\x62\xd6\x3d\xbe\xa4\x5e\x8c\xa9\x67\x12\x82\xfa\xfb\x69\xda\x92\x72\x8b\x1a\x71\xde\x0a\x9e\x06\x0b\x29\x05\xd6\xa5\xb6\x7e\xcd\x3b\x36\x92\xdd\xbd\x7f\x2d\x77\x8b\x8c\x98\x03\xae\xe3\x28\x09\x1b\x58\xfa\xb3\x24\xe4\xfa\xd6\x75\x94\x55\x85\x80\x8b\x48\x31\xd7\xbc\x3f\xf4\xde\xf0\x8e\x4b\x7a\x9d\xe5\x76\xd2\x65\x86\xce\xc6\x4b\x61\x16" -ciphertextX = - "\xbd\x6d\x17\x9d\x3e\x83\xd4\x3b\x95\x76\x57\x94\x93\xc0\xe9\x39\x57\x2a\x17\x00\x25\x2b\xfa\xcc\xbe\xd2\x90\x2c\x21\x39\x6c\xbb\x73\x1c\x7f\x1b\x0b\x4a\xa6\x44\x0b\xf3\xa8\x2f\x4e\xda\x7e\x39\xae\x64\xc6\x70\x8c\x54\xc2\x16\xcb\x96\xb7\x2e\x12\x13\xb4\x52\x2f\x8c\x9b\xa4\x0d\xb5\xd9\x45\xb1\x1b\x69\xb9\x82\xc1\xbb\x9e\x3f\x3f\xac\x2b\xc3\x69\x48\x8f\x76\xb2\x38\x35\x65\xd3\xff\xf9\x21\xf9\x66\x4c\x97\x63\x7d\xa9\x76\x88\x12\xf6\x15\xc6\x8b\x13\xb5\x2e" -tag = "\x1a\xe1\x0b\x59\x4f\x09\xe2\x6a\x7e\x90\x2e\xcb\xd0\x60\x06\x91" -tagX = "\xc0\x87\x59\x24\xc1\xc7\x98\x79\x47\xde\xaf\xd8\x78\x0a\xcf\x49" -nonce1 = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" -nonce2 = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" -nonce3 = "\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" -nonce4 = "\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" -nonce5 = "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" -nonce6 = "\x00\x00\x00\x00\x00\x00\x00\x00" -nonce7 = "\x01\x00\x00\x00\x00\x00\x00\x00" -nonce8 = "\xff\x00\x00\x00\x00\x00\x00\x00" -nonce9 = "\x00\x01\x00\x00\x00\x00\x00\x00" -nonce10 = "\xff\xff\xff\xff\xff\xff\xff\xff" - -a5key :: ByteString -a5key = - "\x1c\x92\x40\xa5\xeb\x55\xd3\x8a\xf3\x33\x88\x86\x04\xf6\xb5\xf0\x47\x39\x17\xc1\x40\x2b\x80\x09\x9d\xca\x5c\xbc\x20\x70\x75\xc0" - -a5nonce :: ByteString -a5nonce = "\x00\x00\x00\x00\x01\x02\x03\x04\x05\x06\x07\x08" - -a5aad :: ByteString -a5aad = "\xf3\x33\x88\x86\x00\x00\x00\x00\x00\x00\x4e\x91" - -a5cipher :: ByteString -a5cipher = - "\x64\xa0\x86\x15\x75\x86\x1a\xf4\x60\xf0\x62\xc7\x9b\xe6\x43\xbd\x5e\x80\x5c\xfd\x34\x5c\xf3\x89\xf1\x08\x67\x0a\xc7\x6c\x8c\xb2\x4c\x6c\xfc\x18\x75\x5d\x43\xee\xa0\x9e\xe9\x4e\x38\x2d\x26\xb0\xbd\xb7\xb7\x3c\x32\x1b\x01\x00\xd4\xf0\x3b\x7f\x35\x58\x94\xcf\x33\x2f\x83\x0e\x71\x0b\x97\xce\x98\xc8\xa8\x4a\xbd\x0b\x94\x81\x14\xad\x17\x6e\x00\x8d\x33\xbd\x60\xf9\x82\xb1\xff\x37\xc8\x55\x97\x97\xa0\x6e\xf4\xf0\xef\x61\xc1\x86\x32\x4e\x2b\x35\x06\x38\x36\x06\x90\x7b\x6a\x7c\x02\xb0\xf9\xf6\x15\x7b\x53\xc8\x67\xe4\xb9\x16\x6c\x76\x7b\x80\x4d\x46\xa5\x9b\x52\x16\xcd\xe7\xa4\xe9\x90\x40\xc5\xa4\x04\x33\x22\x5e\xe2\x82\xa1\xb0\xa0\x6c\x52\x3e\xaf\x45\x34\xd7\xf8\x3f\xa1\x15\x5b\x00\x47\x71\x8c\xbc\x54\x6a\x0d\x07\x2b\x04\xb3\x56\x4e\xea\x1b\x42\x22\x73\xf5\x48\x27\x1a\x0b\xb2\x31\x60\x53\xfa\x76\x99\x19\x55\xeb\xd6\x31\x59\x43\x4e\xce\xbb\x4e\x46\x6d\xae\x5a\x10\x73\xa6\x72\x76\x27\x09\x7a\x10\x49\xe6\x17\xd9\x1d\x36\x10\x94\xfa\x68\xf0\xff\x77\x98\x71\x30\x30\x5b\xea\xba\x2e\xda\x04\xdf\x99\x7b\x71\x4d\x6c\x6f\x2c\x29\xa6\xad\x5c\xb4\x02\x2b\x02\x70\x9b" - -a5plain :: ByteString -a5plain = - "\x49\x6e\x74\x65\x72\x6e\x65\x74\x2d\x44\x72\x61\x66\x74\x73\x20\x61\x72\x65\x20\x64\x72\x61\x66\x74\x20\x64\x6f\x63\x75\x6d\x65\x6e\x74\x73\x20\x76\x61\x6c\x69\x64\x20\x66\x6f\x72\x20\x61\x20\x6d\x61\x78\x69\x6d\x75\x6d\x20\x6f\x66\x20\x73\x69\x78\x20\x6d\x6f\x6e\x74\x68\x73\x20\x61\x6e\x64\x20\x6d\x61\x79\x20\x62\x65\x20\x75\x70\x64\x61\x74\x65\x64\x2c\x20\x72\x65\x70\x6c\x61\x63\x65\x64\x2c\x20\x6f\x72\x20\x6f\x62\x73\x6f\x6c\x65\x74\x65\x64\x20\x62\x79\x20\x6f\x74\x68\x65\x72\x20\x64\x6f\x63\x75\x6d\x65\x6e\x74\x73\x20\x61\x74\x20\x61\x6e\x79\x20\x74\x69\x6d\x65\x2e\x20\x49\x74\x20\x69\x73\x20\x69\x6e\x61\x70\x70\x72\x6f\x70\x72\x69\x61\x74\x65\x20\x74\x6f\x20\x75\x73\x65\x20\x49\x6e\x74\x65\x72\x6e\x65\x74\x2d\x44\x72\x61\x66\x74\x73\x20\x61\x73\x20\x72\x65\x66\x65\x72\x65\x6e\x63\x65\x20\x6d\x61\x74\x65\x72\x69\x61\x6c\x20\x6f\x72\x20\x74\x6f\x20\x63\x69\x74\x65\x20\x74\x68\x65\x6d\x20\x6f\x74\x68\x65\x72\x20\x74\x68\x61\x6e\x20\x61\x73\x20\x2f\xe2\x80\x9c\x77\x6f\x72\x6b\x20\x69\x6e\x20\x70\x72\x6f\x67\x72\x65\x73\x73\x2e\x2f\xe2\x80\x9d" - -a5tag :: ByteString -a5tag = "\xee\xad\x9d\x67\x89\x0c\xbb\x22\x39\x23\x36\xfe\xa1\x85\x1f\x38" - -rfc8439encrypt = a5cipher @=? ct - where - ct = case CP.aeadChacha20poly1305Init a5key a5nonce of - CryptoPassed st -> snd $ aeadSimpleEncrypt st a5aad a5plain 16 - _ -> "dummy" - -rfc8439decrypt = Just a5plain @=? mpt - where - mpt = case CP.aeadChacha20poly1305Init a5key a5nonce of - CryptoPassed st -> aeadSimpleDecrypt st a5aad a5cipher (AuthTag $ B.convert a5tag) - _ -> Nothing - -tests = - testGroup - "ChaChaPoly1305" - [ testCase "V1" runEncrypt - , testCase "V1-decrypt" runDecrypt - , testCase "V1-extended" runEncryptX - , testCase "V1-extended-decrypt" runDecryptX - , testCase "nonce increment" runNonceInc - , testCase "RFC8439 A5 enc" rfc8439encrypt - , testCase "RFC8439 A5 dec" rfc8439decrypt - ] - where - runEncrypt = - let ini = - throwCryptoError $ - CP.initialize key (throwCryptoError $ CP.nonce8 constant iv) - afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) - (out, afterEncrypt) = CP.encrypt plaintext afterAAD - outtag = CP.finalize afterEncrypt - in propertyHoldCase - [ eqTest "ciphertext" ciphertext out - , eqTest "tag" tag (B.convert outtag) - ] - runEncryptX = - let ini = - throwCryptoError $ CP.initializeX key (throwCryptoError $ CP.nonce24 ivX) - afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) - (out, afterEncrypt) = CP.encrypt plaintext afterAAD - outtag = CP.finalize afterEncrypt - in propertyHoldCase - [ eqTest "ciphertext" ciphertextX out - , eqTest "tag" tagX (B.convert outtag) - ] - - runDecrypt = - let ini = - throwCryptoError $ - CP.initialize key (throwCryptoError $ CP.nonce8 constant iv) - afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) - (out, afterDecrypt) = CP.decrypt ciphertext afterAAD - outtag = CP.finalize afterDecrypt - in propertyHoldCase - [ eqTest "plaintext" plaintext out - , eqTest "tag" tag (B.convert outtag) - ] - - runDecryptX = - let ini = - throwCryptoError $ CP.initializeX key (throwCryptoError $ CP.nonce24 ivX) - afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) - (out, afterDecrypt) = CP.decrypt ciphertextX afterAAD - outtag = CP.finalize afterDecrypt - in propertyHoldCase - [ eqTest "plaintext" plaintext out - , eqTest "tag" tagX (B.convert outtag) - ] - - runNonceInc = - let n1 = throwCryptoError . CP.nonce12 $ nonce1 - n3 = throwCryptoError . CP.nonce12 $ nonce3 - n5 = throwCryptoError . CP.nonce12 $ nonce5 - n6 = throwCryptoError . CP.nonce8 constant $ nonce6 - n8 = throwCryptoError . CP.nonce8 constant $ nonce8 - n10 = throwCryptoError . CP.nonce8 constant $ nonce10 - in propertyHoldCase - [ eqTest "nonce12a" nonce2 $ B.convert . CP.incrementNonce $ n1 - , eqTest "nonce12b" nonce4 $ B.convert . CP.incrementNonce $ n3 - , eqTest "nonce12c" nonce1 $ B.convert . CP.incrementNonce $ n5 - , eqTest "nonce8a" (B.concat [constant, nonce7]) $ - B.convert . CP.incrementNonce $ - n6 - , eqTest "nonce8b" (B.concat [constant, nonce9]) $ - B.convert . CP.incrementNonce $ - n8 - , eqTest "nonce8c" (B.concat [constant, nonce6]) $ - B.convert . CP.incrementNonce $ - n10 - ] diff --git a/tests/ConstructHash/MiyaguchiPreneelSpec.hs b/tests/ConstructHash/MiyaguchiPreneelSpec.hs new file mode 100644 --- /dev/null +++ b/tests/ConstructHash/MiyaguchiPreneelSpec.hs @@ -0,0 +1,52 @@ +module ConstructHash.MiyaguchiPreneelSpec (spec) where + +import Crypto.Cipher.AES (AES128) +import Crypto.ConstructHash.MiyaguchiPreneel as MiyaguchiPreneel + +import Imports + +import qualified Data.ByteArray as B +import Data.ByteArray.Encoding (Base (Base16), convertFromBase) +import qualified Data.ByteString.Char8 as B8 + +runMP128 :: ByteString -> ByteString +runMP128 s = B.convert (MiyaguchiPreneel.compute s :: MiyaguchiPreneel AES128) + +hxs :: String -> ByteString +hxs = + either (error . ("hxs:" ++)) id + . convertFromBase Base16 + . B8.pack + . filter (/= ' ') + +gAES128 :: Spec +gAES128 = + igroup + "aes128" + [ runMP128 B8.empty + `shouldBe` hxs "66e94bd4 ef8a2c3b 884cfa59 ca342b2e" + , runMP128 (hxs "01000000 00000000 00000000 00000000") + `shouldBe` hxs "46711816 e91d6ff0 59bbbf2b f58e0fd3" + , runMP128 (hxs "00000000 00000000 00000000 00000001") + `shouldBe` hxs "58e2fcce fa7e3061 367f1d57 a4e7455b" + , runMP128 + ( hxs $ + "00000000 00000000 00000000 00000000" + ++ "01" + ) + `shouldBe` hxs "a5ff35ae 097adf5d 646abf5e bf4c16f4" + ] + +igroup :: String -> [Expectation] -> Spec +igroup nm = describe nm . sequence_ . zipWith (flip ($)) [1 ..] . map icase + where + icase c i = it (show (i :: Int)) c + +vectors :: Spec +vectors = + describe "KATs" $ do + gAES128 + +spec :: Spec +spec = do + vectors diff --git a/tests/Curve25519Spec.hs b/tests/Curve25519Spec.hs new file mode 100644 --- /dev/null +++ b/tests/Curve25519Spec.hs @@ -0,0 +1,52 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Curve25519Spec (spec) where + +import Crypto.Error +import qualified Crypto.PubKey.Curve25519 as Curve25519 +import Data.ByteArray as B +import Imports + +alicePrivate = + throwCryptoError $ + Curve25519.secretKey + ( "\x77\x07\x6d\x0a\x73\x18\xa5\x7d\x3c\x16\xc1\x72\x51\xb2\x66\x45\xdf\x4c\x2f\x87\xeb\xc0\x99\x2a\xb1\x77\xfb\xa5\x1d\xb9\x2c\x2a" + :: ByteString + ) +alicePublic = + throwCryptoError $ + Curve25519.publicKey + ( "\x85\x20\xf0\x09\x89\x30\xa7\x54\x74\x8b\x7d\xdc\xb4\x3e\xf7\x5a\x0d\xbf\x3a\x0d\x26\x38\x1a\xf4\xeb\xa4\xa9\x8e\xaa\x9b\x4e\x6a" + :: ByteString + ) +bobPrivate = + throwCryptoError $ + Curve25519.secretKey + ( "\x5d\xab\x08\x7e\x62\x4a\x8a\x4b\x79\xe1\x7f\x8b\x83\x80\x0e\xe6\x6f\x3b\xb1\x29\x26\x18\xb6\xfd\x1c\x2f\x8b\x27\xff\x88\xe0\xeb" + :: ByteString + ) +bobPublic = + throwCryptoError $ + Curve25519.publicKey + ( "\xde\x9e\xdb\x7d\x7b\x7d\xc1\xb4\xd3\x5b\x61\xc2\xec\xe4\x35\x37\x3f\x83\x43\xc8\x5b\x78\x67\x4d\xad\xfc\x7e\x14\x6f\x88\x2b\x4f" + :: ByteString + ) +aliceMultBob = + "\x4a\x5d\x9d\x5b\xa4\xce\x2d\xe1\x72\x8e\x3b\xf4\x80\x35\x0f\x25\xe0\x7e\x21\xc9\x47\xd1\x9e\x33\x76\xf0\x9b\x3c\x1e\x16\x17\x42" + :: ByteString + +katTests :: [Spec] +katTests = + [ it + "0" + (B.convert (Curve25519.dh alicePublic bobPrivate) `shouldBe` aliceMultBob) + , it + "1" + (B.convert (Curve25519.dh bobPublic alicePrivate) `shouldBe` aliceMultBob) + , it "2" (Curve25519.toPublic alicePrivate `shouldBe` alicePublic) + , it "3" (Curve25519.toPublic bobPrivate `shouldBe` bobPublic) + ] + +spec :: Spec +spec = do + describe "KATs" $ sequence_ katTests diff --git a/tests/Curve448Spec.hs b/tests/Curve448Spec.hs new file mode 100644 --- /dev/null +++ b/tests/Curve448Spec.hs @@ -0,0 +1,48 @@ +{-# LANGUAGE OverloadedStrings #-} + +module Curve448Spec (spec) where + +import Crypto.Error +import qualified Crypto.PubKey.Curve448 as Curve448 +import Data.ByteArray as B +import Imports + +alicePrivate = + throwCryptoError $ + Curve448.secretKey + ( "\x9a\x8f\x49\x25\xd1\x51\x9f\x57\x75\xcf\x46\xb0\x4b\x58\x00\xd4\xee\x9e\xe8\xba\xe8\xbc\x55\x65\xd4\x98\xc2\x8d\xd9\xc9\xba\xf5\x74\xa9\x41\x97\x44\x89\x73\x91\x00\x63\x82\xa6\xf1\x27\xab\x1d\x9a\xc2\xd8\xc0\xa5\x98\x72\x6b" + :: ByteString + ) +alicePublic = + throwCryptoError $ + Curve448.publicKey + ( "\x9b\x08\xf7\xcc\x31\xb7\xe3\xe6\x7d\x22\xd5\xae\xa1\x21\x07\x4a\x27\x3b\xd2\xb8\x3d\xe0\x9c\x63\xfa\xa7\x3d\x2c\x22\xc5\xd9\xbb\xc8\x36\x64\x72\x41\xd9\x53\xd4\x0c\x5b\x12\xda\x88\x12\x0d\x53\x17\x7f\x80\xe5\x32\xc4\x1f\xa0" + :: ByteString + ) +bobPrivate = + throwCryptoError $ + Curve448.secretKey + ( "\x1c\x30\x6a\x7a\xc2\xa0\xe2\xe0\x99\x0b\x29\x44\x70\xcb\xa3\x39\xe6\x45\x37\x72\xb0\x75\x81\x1d\x8f\xad\x0d\x1d\x69\x27\xc1\x20\xbb\x5e\xe8\x97\x2b\x0d\x3e\x21\x37\x4c\x9c\x92\x1b\x09\xd1\xb0\x36\x6f\x10\xb6\x51\x73\x99\x2d" + :: ByteString + ) +bobPublic = + throwCryptoError $ + Curve448.publicKey + ( "\x3e\xb7\xa8\x29\xb0\xcd\x20\xf5\xbc\xfc\x0b\x59\x9b\x6f\xec\xcf\x6d\xa4\x62\x71\x07\xbd\xb0\xd4\xf3\x45\xb4\x30\x27\xd8\xb9\x72\xfc\x3e\x34\xfb\x42\x32\xa1\x3c\xa7\x06\xdc\xb5\x7a\xec\x3d\xae\x07\xbd\xc1\xc6\x7b\xf3\x36\x09" + :: ByteString + ) +aliceMultBob = + "\x07\xff\xf4\x18\x1a\xc6\xcc\x95\xec\x1c\x16\xa9\x4a\x0f\x74\xd1\x2d\xa2\x32\xce\x40\xa7\x75\x52\x28\x1d\x28\x2b\xb6\x0c\x0b\x56\xfd\x24\x64\xc3\x35\x54\x39\x36\x52\x1c\x24\x40\x30\x85\xd5\x9a\x44\x9a\x50\x37\x51\x4a\x87\x9d" + :: ByteString + +katTests :: [Spec] +katTests = + [ it "0" (B.convert (Curve448.dh alicePublic bobPrivate) `shouldBe` aliceMultBob) + , it "1" (B.convert (Curve448.dh bobPublic alicePrivate) `shouldBe` aliceMultBob) + , it "2" (Curve448.toPublic alicePrivate `shouldBe` alicePublic) + , it "3" (Curve448.toPublic bobPrivate `shouldBe` bobPublic) + ] + +spec :: Spec +spec = do + describe "KATs" $ sequence_ katTests diff --git a/tests/ECC.hs b/tests/ECC.hs deleted file mode 100644 --- a/tests/ECC.hs +++ /dev/null @@ -1,399 +0,0 @@ -{-# LANGUAGE ExistentialQuantification #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE OverloadedStrings #-} - -module ECC (tests) where - -import Data.Either - -import qualified Crypto.ECC as ECC -import Crypto.Error - -import Data.ByteArray.Encoding - -import Imports - -data Curve - = forall curve. - (ECC.EllipticCurveDH curve, Show curve, Eq (ECC.Point curve)) => - Curve curve - -instance Show Curve where - showsPrec d (Curve curve) = showsPrec d curve - -instance Arbitrary Curve where - arbitrary = - elements - [ Curve ECC.Curve_P256R1 - , Curve ECC.Curve_P384R1 - , Curve ECC.Curve_P521R1 - , Curve ECC.Curve_X25519 - , Curve ECC.Curve_X448 - ] - -data CurveArith - = forall curve. (ECC.EllipticCurveBasepointArith curve, Show curve) => CurveArith curve - -instance Show CurveArith where - showsPrec d (CurveArith curve) = showsPrec d curve - -instance Arbitrary CurveArith where - arbitrary = - elements - [ CurveArith ECC.Curve_P256R1 - , CurveArith ECC.Curve_P384R1 - , CurveArith ECC.Curve_P521R1 - , CurveArith ECC.Curve_Edwards25519 - ] - -data VectorPoint = VectorPoint - { vpCurve :: Curve - , vpHex :: ByteString - , vpError :: Maybe CryptoError - } - -vectorsPoint :: [VectorPoint] -vectorsPoint = - [ VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = "" - , vpError = Just CryptoError_PointSizeInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = "00" - , vpError = Just CryptoError_PointFormatInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "0408edd7b50085a952172228aca391beebe9ba942a0ae9eb15bcc8d50795d1a5505221c7b9b3bb4310f165fc3ac3114339db8170ceae6697e0f9736698b33551b8" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "04216f25b00717d46deef3402628f6abf265bfa12aea515ae8f100ce415e251e72cd5cd8f47f613a0f4e0f4f9410dd9c85c149cffcb320c2d52bf550a397ec92e5" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "0421eba6080610926609bb8d52afd3331ed1b07e0ba4c1441a118b62497d3e85f39a50c865027cdd84298cdf094b7818f2a65ae59f46c971a32ab4ea3c2c93c959" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "0400d7fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a0001a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "040000fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a0001a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" - , vpError = Just CryptoError_PointCoordinatesInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "04d7fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a01a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" - , vpError = Just CryptoError_PublicKeySizeInvalid -- tests leading zeros - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P256R1 - , vpHex = - "040000d7fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a000001a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" - , vpError = Just CryptoError_PublicKeySizeInvalid -- tests leading zeros - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = "" - , vpError = Just CryptoError_PointSizeInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = "00" - , vpError = Just CryptoError_PointFormatInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "0409281a103fb1773445e16eec86adb095e32928ccc9c806bd210c649712813bdb6cab40163a8cb163b578ea8dda5eb32cfb5208ebf0d31a6c590fa92f5a61f32dbc0d518b166ea5a9adf9dd21c1bd09932ca21c6a5725ca89542ac57b6a9eca6f" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "040c7b3fb575c1db7bc61fe7a456cc34a8289f41e167938a56e5ba2787723f3de2c645112705e13ed24f477730173935ca4e0ff468e7e0acf78a9f59dadff8193a0e23789eb3737730c089b27a0f94de7d95b8db4466d017fb21a5710d6ca85775" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "0438e7705220b60460194be63d21c8945be2a211957168fa60f26b2ad4e8f5cd96a7779e7edff4deda9ded63243c2127e273d4444edaaba03b79b6caafc5033432af13776f851c0c7e1080c60d7ee3b61740720ab98461813dab5fb8c31bfa9ed9" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "04000836bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884c00b1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "04000036bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884c00b1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" - , vpError = Just CryptoError_PointCoordinatesInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "040836bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884cb1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" - , vpError = Nothing -- ignores leading zeros - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P384R1 - , vpHex = - "0400000836bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884c0000b1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" - , vpError = Nothing -- ignores leading zeros - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = "" - , vpError = Just CryptoError_PointSizeInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = "00" - , vpError = Just CryptoError_PointFormatInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "04000ce5c207335134567026063743df82c1b551a009cf616471f0e23fa9767a50cc7f8771ef13a65c49ce7e1cd1ac3ad721dcc3ddd35f98ae5d380a0832f87a9f0ca4012914911d6bea7f3c481d694fb1645be27c7b66b09b28e261f8030b3fb8206f6a95f6ad73db755765b64f592a799234f8f451cb787abe95b1a54991a799ad0d69da" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "04003a5e6c1ce3a6a323757005da17b357db991bd1ad835e6201411f458b5c2edb3c66786b727b7e15fbad7dd74a4b0eb542183b5242e5952061cb85e7229353eb0dc300aac2dbd5232d582481ba7a59a993eb04c4466a1b17ba0015b65c616ce8703e70880969d8d58e633acb29c3ca017eb1b88649387b867466090ce1a57c2b4f8376bb" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "04003e0659fe9498695a3d8c88b8e25fa8133c30ab10eccbe9094344c99924f89fb69d9b3acf03bf438328f9cba55fa28a05be9a7e18780706b3728abfee2592aeb86d0001ea5ff64f2ca7a6453c79f80550e971843e073f4f8fec75bad2e52a4483ebf1f16f43d0de27e1967ea22f9722527652fa74439fdc03a569fba29e2d6f7c012db6" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "040043f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def306000a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "040000f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def306000a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" - , vpError = Just CryptoError_PointCoordinatesInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "0443f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def3060a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" - , vpError = Nothing -- ignores leading zeros - } - , VectorPoint - { vpCurve = Curve ECC.Curve_P521R1 - , vpHex = - "04000043f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def30600000a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" - , vpError = Nothing -- ignores leading zeros - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "" - , vpError = Just CryptoError_PublicKeySizeInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "22cd98c65fb50db3be0d6d359456c0cd3516952a6e7229ff672893944f703f10" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "23cd98c65fb50db3be0d6d359456c0cd3516952a6e7229ff672893944f703f10" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "0023cd98c65fb50db3be0d6d359456c0cd3516952a6e7229ff672893944f703f10" - , vpError = Just CryptoError_PublicKeySizeInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X448 - , vpHex = "" - , vpError = Just CryptoError_PublicKeySizeInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X448 - , vpHex = - "2b162c2fef165ecbb203e40975ae4424f0f8db25ab582cb96b2e5ffe90a31798b35480b594c99dc32b437e61a74f792d8ecf5fc3e8cfeb75" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X448 - , vpHex = - "2c162c2fef165ecbb203e40975ae4424f0f8db25ab582cb96b2e5ffe90a31798b35480b594c99dc32b437e61a74f792d8ecf5fc3e8cfeb75" - , vpError = Nothing - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X448 - , vpHex = - "002c162c2fef165ecbb203e40975ae4424f0f8db25ab582cb96b2e5ffe90a31798b35480b594c99dc32b437e61a74f792d8ecf5fc3e8cfeb75" - , vpError = Just CryptoError_PublicKeySizeInvalid - } - ] - -vectorsWeakPoint :: [VectorPoint] -vectorsWeakPoint = - [ VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "0000000000000000000000000000000000000000000000000000000000000000" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "0100000000000000000000000000000000000000000000000000000000000000" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "e0eb7a7c3b41b8ae1656e3faf19fc46ada098deb9c32b1fd866205165f49b800" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "5f9c95bca3508c24b1d0b1559c83ef5b04445cc4581c8e86d8224eddd09f1157" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "ecffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "edffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X25519 - , vpHex = "eeffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X448 - , vpHex = - "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - , VectorPoint - { vpCurve = Curve ECC.Curve_X448 - , vpHex = - "0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" - , vpError = Just CryptoError_ScalarMultiplicationInvalid - } - ] - -vpEncodedPoint :: VectorPoint -> ByteString -vpEncodedPoint vector = fromRight (error "vpEncodedPoint") $ convertFromBase Base16 (vpHex vector) - -cryptoError :: CryptoFailable a -> Maybe CryptoError -cryptoError = onCryptoFailure Just (const Nothing) - -doPointDecodeTest :: Show p => p -> VectorPoint -> TestTree -doPointDecodeTest i vector = - case vpCurve vector of - Curve curve -> - let prx = Just curve -- using Maybe as Proxy - in testCase - (show i) - (vpError vector @=? cryptoError (ECC.decodePoint prx $ vpEncodedPoint vector)) - -doWeakPointECDHTest :: Show p => p -> VectorPoint -> TestTree -doWeakPointECDHTest i vector = - case vpCurve vector of - Curve curve -> testCase (show i) $ do - let prx = Just curve -- using Maybe as Proxy - public = throwCryptoError $ ECC.decodePoint prx $ vpEncodedPoint vector - keyPair <- ECC.curveGenerateKeyPair prx - vpError vector - @=? cryptoError (ECC.ecdh prx (ECC.keypairGetPrivate keyPair) public) - -tests :: TestTree -tests = - testGroup - "ECC" - [ testGroup "decodePoint" $ zipWith doPointDecodeTest [katZero ..] vectorsPoint - , testGroup "ECDH weak points" $ - zipWith doWeakPointECDHTest [katZero ..] vectorsWeakPoint - , testGroup - "property" - [ testProperty "decodePoint.encodePoint==id" $ \testDRG (Curve curve) -> - let prx = Just curve -- using Maybe as Proxy - keyPair = withTestDRG testDRG $ ECC.curveGenerateKeyPair prx - p1 = ECC.keypairGetPublic keyPair - bs = ECC.encodePoint prx p1 :: ByteString - p2 = ECC.decodePoint prx bs - in CryptoPassed p1 == p2 - , localOption (QuickCheckTests 20) $ testProperty "ECDH commutes" $ \testDRG (Curve curve) -> - let prx = Just curve -- using Maybe as Proxy - (alice, bob) = - withTestDRG testDRG $ - (,) - <$> ECC.curveGenerateKeyPair prx - <*> ECC.curveGenerateKeyPair prx - aliceShared = ECC.ecdh prx (ECC.keypairGetPrivate alice) (ECC.keypairGetPublic bob) - bobShared = ECC.ecdh prx (ECC.keypairGetPrivate bob) (ECC.keypairGetPublic alice) - aliceShared' = ECC.ecdhRaw prx (ECC.keypairGetPrivate alice) (ECC.keypairGetPublic bob) - bobShared' = ECC.ecdhRaw prx (ECC.keypairGetPrivate bob) (ECC.keypairGetPublic alice) - in aliceShared == bobShared - && aliceShared == CryptoPassed aliceShared' - && bobShared == CryptoPassed bobShared' - , testProperty "decodeScalar.encodeScalar==id" $ \testDRG (CurveArith curve) -> - let prx = Just curve -- using Maybe as Proxy - s1 = withTestDRG testDRG $ ECC.curveGenerateScalar prx - bs = ECC.encodeScalar prx s1 :: ByteString - s2 = ECC.decodeScalar prx bs - in CryptoPassed s1 == s2 - , testProperty "scalarFromInteger.scalarToInteger==id" $ \testDRG (CurveArith curve) -> - let prx = Just curve -- using Maybe as Proxy - s1 = withTestDRG testDRG $ ECC.curveGenerateScalar prx - bs = ECC.scalarToInteger prx s1 - s2 = ECC.scalarFromInteger prx bs - in CryptoPassed s1 == s2 - , localOption (QuickCheckTests 20) $ testProperty "(a + b).P = a.P + b.P" $ \testDRG (CurveArith curve) -> - let prx = Just curve -- using Maybe as Proxy - (s, a, b) = - withTestDRG testDRG $ - (,,) - <$> ECC.curveGenerateScalar prx - <*> ECC.curveGenerateScalar prx - <*> ECC.curveGenerateScalar prx - p = ECC.pointBaseSmul prx s - in ECC.pointSmul prx (ECC.scalarAdd prx a b) p - == ECC.pointAdd prx (ECC.pointSmul prx a p) (ECC.pointSmul prx b p) - , localOption (QuickCheckTests 20) $ testProperty "(a * b).P = a.(b.P)" $ \testDRG (CurveArith curve) -> - let prx = Just curve -- using Maybe as Proxy - (s, a, b) = - withTestDRG testDRG $ - (,,) - <$> ECC.curveGenerateScalar prx - <*> ECC.curveGenerateScalar prx - <*> ECC.curveGenerateScalar prx - p = ECC.pointBaseSmul prx s - in ECC.pointSmul prx (ECC.scalarMul prx a b) p - == ECC.pointSmul prx a (ECC.pointSmul prx b p) - ] - ] diff --git a/tests/ECC/Edwards25519.hs b/tests/ECC/Edwards25519.hs deleted file mode 100644 --- a/tests/ECC/Edwards25519.hs +++ /dev/null @@ -1,195 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module ECC.Edwards25519 (tests) where - -import Crypto.ECC.Edwards25519 -import Crypto.Error -import qualified Data.ByteString as B -import Data.Word (Word8) -import Imports - -instance Arbitrary Scalar where - arbitrary = - fmap - (throwCryptoError . scalarDecodeLong) - (arbitraryBS 64) - -smallScalar :: Word8 -> Scalar -smallScalar = throwCryptoError . scalarDecodeLong . B.singleton - -newtype PrimeOrder = PrimeOrder Point - deriving (Show) - --- points in the prime-order subgroup -instance Arbitrary PrimeOrder where - arbitrary = (PrimeOrder . toPoint) `fmap` arbitrary - --- arbitrary curve point, including points with a torsion component -instance Arbitrary Point where - arbitrary = do - a <- arbitrary - b <- elements $ map smallScalar [0 .. 7] - return (pointsMulVarTime a b torsion8) - --- an 8-torsion point -torsion8 :: Point -torsion8 = - throwCryptoError $ - pointDecode - ( "\199\ETBjp=M\216O\186<\vv\r\DLEg\SI* S\250,9\204\198N\199\253w\146\172\ETXz" - :: ByteString - ) - -tests = - testGroup - "ECC.Edwards25519" - [ testGroup - "vectors" - [ testCase "11*G" $ p011 @=? toPoint s011 - , testCase "123*G" $ p123 @=? toPoint s123 - , testCase "134*G" $ p134 @=? toPoint s134 - , testCase "123*G + 11*G" $ p134 @=? pointAdd p123 p011 - ] - , testGroup - "scalar arithmetic" - [ testProperty "scalarDecodeLong.scalarEncode==id" $ \s -> - let bs = scalarEncode s :: ByteString - ss = scalarDecodeLong bs - in CryptoPassed s `propertyEq` ss - , testCase "curve order" $ s0 @=? sN - , testProperty "addition with zero" $ \s -> - propertyHold - [ eqTest "zero left" s (scalarAdd s0 s) - , eqTest "zero right" s (scalarAdd s s0) - ] - , testProperty "addition associative" $ \sa sb sc -> - scalarAdd sa (scalarAdd sb sc) === scalarAdd (scalarAdd sa sb) sc - , testProperty "addition commutative" $ \sa sb -> - scalarAdd sa sb === scalarAdd sb sa - , testProperty "multiplication with zero" $ \s -> - propertyHold - [ eqTest "zero left" s0 (scalarMul s0 s) - , eqTest "zero right" s0 (scalarMul s s0) - ] - , testProperty "multiplication with one" $ \s -> - propertyHold - [ eqTest "one left" s (scalarMul s1 s) - , eqTest "one right" s (scalarMul s s1) - ] - , testProperty "multiplication associative" $ \sa sb sc -> - scalarMul sa (scalarMul sb sc) === scalarMul (scalarMul sa sb) sc - , testProperty "multiplication commutative" $ \sa sb -> - scalarMul sa sb === scalarMul sb sa - , testProperty "multiplication distributive" $ \sa sb sc -> - propertyHold - [ eqTest - "distributive left" - ((sa `scalarMul` sb) `scalarAdd` (sa `scalarMul` sc)) - (sa `scalarMul` (sb `scalarAdd` sc)) - , eqTest - "distributive right" - ((sb `scalarMul` sa) `scalarAdd` (sc `scalarMul` sa)) - ((sb `scalarAdd` sc) `scalarMul` sa) - ] - ] - , testGroup - "point arithmetic" - [ testProperty "pointDecode.pointEncode==id" $ \p -> - let bs = pointEncode p :: ByteString - p' = pointDecode bs - in CryptoPassed p `propertyEq` p' - , testProperty "pointEncode.pointDecode==id" $ \p -> - let b = pointEncode p :: ByteString - p' = pointDecode b - b' = pointEncode `fmap` p' - in CryptoPassed b `propertyEq` b' - , testProperty "addition with identity" $ \p -> - propertyHold - [ eqTest "identity left" p (pointAdd p0 p) - , eqTest "identity right" p (pointAdd p p0) - ] - , testProperty "addition associative" $ \pa pb pc -> - pointAdd pa (pointAdd pb pc) === pointAdd (pointAdd pa pb) pc - , testProperty "addition commutative" $ \pa pb -> - pointAdd pa pb === pointAdd pb pa - , testProperty "negation" $ \p -> - p0 `propertyEq` pointAdd p (pointNegate p) - , testProperty "doubling" $ \p -> - pointAdd p p `propertyEq` pointDouble p - , testProperty "multiplication by cofactor" $ \p -> - pointMul s8 p `propertyEq` pointMulByCofactor p - , testProperty "prime order" $ \(PrimeOrder p) -> - True `propertyEq` pointHasPrimeOrder p - , testCase "8-torsion point" $ do - assertBool "mul by 4" $ p0 /= pointMul s4 torsion8 - assertBool "mul by 8" $ p0 == pointMul s8 torsion8 - , testProperty "scalarmult with zero" $ \p -> - p0 `propertyEq` pointMul s0 p - , testProperty "scalarmult with one" $ \p -> - p `propertyEq` pointMul s1 p - , testProperty "scalarmult with two" $ \p -> - pointDouble p `propertyEq` pointMul s2 p - , testProperty "scalarmult with curve order - 1" $ \p -> - pointHasPrimeOrder p === (pointNegate p == pointMul sI p) - , testProperty "scalarmult commutative" $ \a b -> - pointMul a (toPoint b) === pointMul b (toPoint a) - , testProperty "scalarmult distributive" $ \x y (PrimeOrder p) -> - let pR = pointMul x p `pointAdd` pointMul y p - in pR `propertyEq` pointMul (x `scalarAdd` y) p - , testProperty "double scalarmult" $ \n1 n2 p -> - let pR = pointAdd (toPoint n1) (pointMul n2 p) - in pR `propertyEq` pointsMulVarTime n1 n2 p - ] - ] - where - p0 = toPoint s0 - s0 = smallScalar 0 - s1 = smallScalar 1 - s2 = smallScalar 2 - s4 = smallScalar 4 - s8 = smallScalar 8 - sI = - throwCryptoError $ - scalarDecodeLong - ( "\236\211\245\\\SUBc\DC2X\214\156\247\162\222\249\222\DC4\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\DLE" - :: ByteString - ) - sN = - throwCryptoError $ - scalarDecodeLong - ( "\237\211\245\\\SUBc\DC2X\214\156\247\162\222\249\222\DC4\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\DLE" - :: ByteString - ) - - s011 = throwCryptoError $ scalarDecodeLong ("\011" :: ByteString) - s123 = throwCryptoError $ scalarDecodeLong ("\123" :: ByteString) - s134 = throwCryptoError $ scalarDecodeLong ("\134" :: ByteString) - - p011 = - throwCryptoError $ - pointDecode - ( "\x13\x37\x03\x6a\xc3\x2d\x8f\x30\xd4\x58\x9c\x3c\x1c\x59\x58\x12\xce\x0f\xff\x40\xe3\x7c\x6f\x5a\x97\xab\x21\x3f\x31\x82\x90\xad" - :: ByteString - ) - p123 = - throwCryptoError $ - pointDecode - ( "\xc4\xb8\x00\xc8\x70\x10\xf9\x46\x83\x03\xde\xea\x87\x65\x03\xe8\x86\xbf\xde\x19\x00\xe9\xe8\x46\xfd\x4c\x3c\xd0\x9c\x1c\xbc\x9f" - :: ByteString - ) - p134 = - throwCryptoError $ - pointDecode - ( "\x51\x20\xab\xe0\x3c\xa2\xaf\x66\xc7\x7c\xa3\x20\xf0\xb2\x1f\xb5\x56\xf6\xb6\x5f\xdd\x7e\x32\x64\xc1\x4a\x30\xd9\x7b\xf7\xa7\x6f" - :: ByteString - ) - --- Using : --- --- >>> import ed25519 --- >>> encodepoint(scalarmult(B, 11)).encode('hex') --- '1337036ac32d8f30d4589c3c1c595812ce0fff40e37c6f5a97ab213f318290ad' --- >>> encodepoint(scalarmult(B, 123)).encode('hex') --- 'c4b800c87010f9468303deea876503e886bfde1900e9e846fd4c3cd09c1cbc9f' --- >>> encodepoint(scalarmult(B, 134)).encode('hex') --- '5120abe03ca2af66c77ca320f0b21fb556f6b65fdd7e3264c14a30d97bf7a76f' diff --git a/tests/ECC/Edwards25519Spec.hs b/tests/ECC/Edwards25519Spec.hs new file mode 100644 --- /dev/null +++ b/tests/ECC/Edwards25519Spec.hs @@ -0,0 +1,187 @@ +{-# LANGUAGE OverloadedStrings #-} + +module ECC.Edwards25519Spec (spec) where + +import Crypto.ECC.Edwards25519 +import Crypto.Error +import qualified Data.ByteString as B +import Data.Word (Word8) +import Imports + +instance Arbitrary Scalar where + arbitrary = + fmap + (throwCryptoError . scalarDecodeLong) + (arbitraryBS 64) + +smallScalar :: Word8 -> Scalar +smallScalar = throwCryptoError . scalarDecodeLong . B.singleton + +newtype PrimeOrder = PrimeOrder Point + deriving (Show) + +-- points in the prime-order subgroup +instance Arbitrary PrimeOrder where + arbitrary = (PrimeOrder . toPoint) `fmap` arbitrary + +-- arbitrary curve point, including points with a torsion component +instance Arbitrary Point where + arbitrary = do + a <- arbitrary + b <- elements $ map smallScalar [0 .. 7] + return (pointsMulVarTime a b torsion8) + +-- an 8-torsion point +torsion8 :: Point +torsion8 = + throwCryptoError $ + pointDecode + ( "\199\ETBjp=M\216O\186<\vv\r\DLEg\SI* S\250,9\204\198N\199\253w\146\172\ETXz" + :: ByteString + ) + +spec :: Spec +spec = do + describe "vectors" $ do + it "11*G" $ toPoint s011 `shouldBe` p011 + it "123*G" $ toPoint s123 `shouldBe` p123 + it "134*G" $ toPoint s134 `shouldBe` p134 + it "123*G + 11*G" $ pointAdd p123 p011 `shouldBe` p134 + describe "scalar arithmetic" $ do + prop "scalarDecodeLong.scalarEncode==id" $ \s -> + let bs = scalarEncode s :: ByteString + ss = scalarDecodeLong bs + in CryptoPassed s `propertyEq` ss + it "curve order" $ sN `shouldBe` s0 + prop "addition with zero" $ \s -> + propertyHold + [ eqTest "zero left" s (scalarAdd s0 s) + , eqTest "zero right" s (scalarAdd s s0) + ] + prop "addition associative" $ \sa sb sc -> + scalarAdd sa (scalarAdd sb sc) === scalarAdd (scalarAdd sa sb) sc + prop "addition commutative" $ \sa sb -> + scalarAdd sa sb === scalarAdd sb sa + prop "multiplication with zero" $ \s -> + propertyHold + [ eqTest "zero left" s0 (scalarMul s0 s) + , eqTest "zero right" s0 (scalarMul s s0) + ] + prop "multiplication with one" $ \s -> + propertyHold + [ eqTest "one left" s (scalarMul s1 s) + , eqTest "one right" s (scalarMul s s1) + ] + prop "multiplication associative" $ \sa sb sc -> + scalarMul sa (scalarMul sb sc) === scalarMul (scalarMul sa sb) sc + prop "multiplication commutative" $ \sa sb -> + scalarMul sa sb === scalarMul sb sa + prop "multiplication distributive" $ \sa sb sc -> + propertyHold + [ eqTest + "distributive left" + ((sa `scalarMul` sb) `scalarAdd` (sa `scalarMul` sc)) + (sa `scalarMul` (sb `scalarAdd` sc)) + , eqTest + "distributive right" + ((sb `scalarMul` sa) `scalarAdd` (sc `scalarMul` sa)) + ((sb `scalarAdd` sc) `scalarMul` sa) + ] + describe "point arithmetic" $ do + prop "pointDecode.pointEncode==id" $ \p -> + let bs = pointEncode p :: ByteString + p' = pointDecode bs + in CryptoPassed p `propertyEq` p' + prop "pointEncode.pointDecode==id" $ \p -> + let b = pointEncode p :: ByteString + p' = pointDecode b + b' = pointEncode `fmap` p' + in CryptoPassed b `propertyEq` b' + prop "addition with identity" $ \p -> + propertyHold + [ eqTest "identity left" p (pointAdd p0 p) + , eqTest "identity right" p (pointAdd p p0) + ] + prop "addition associative" $ \pa pb pc -> + pointAdd pa (pointAdd pb pc) === pointAdd (pointAdd pa pb) pc + prop "addition commutative" $ \pa pb -> + pointAdd pa pb === pointAdd pb pa + prop "negation" $ \p -> + p0 `propertyEq` pointAdd p (pointNegate p) + prop "doubling" $ \p -> + pointAdd p p `propertyEq` pointDouble p + prop "multiplication by cofactor" $ \p -> + pointMul s8 p `propertyEq` pointMulByCofactor p + prop "prime order" $ \(PrimeOrder p) -> + True `propertyEq` pointHasPrimeOrder p + it "8-torsion point" $ do + assertBool "mul by 4" $ p0 /= pointMul s4 torsion8 + assertBool "mul by 8" $ p0 == pointMul s8 torsion8 + prop "scalarmult with zero" $ \p -> + p0 `propertyEq` pointMul s0 p + prop "scalarmult with one" $ \p -> + p `propertyEq` pointMul s1 p + prop "scalarmult with two" $ \p -> + pointDouble p `propertyEq` pointMul s2 p + prop "scalarmult with curve order - 1" $ \p -> + pointHasPrimeOrder p === (pointNegate p == pointMul sI p) + prop "scalarmult commutative" $ \a b -> + pointMul a (toPoint b) === pointMul b (toPoint a) + prop "scalarmult distributive" $ \x y (PrimeOrder p) -> + let pR = pointMul x p `pointAdd` pointMul y p + in pR `propertyEq` pointMul (x `scalarAdd` y) p + prop "double scalarmult" $ \n1 n2 p -> + let pR = pointAdd (toPoint n1) (pointMul n2 p) + in pR `propertyEq` pointsMulVarTime n1 n2 p + where + p0 = toPoint s0 + s0 = smallScalar 0 + s1 = smallScalar 1 + s2 = smallScalar 2 + s4 = smallScalar 4 + s8 = smallScalar 8 + sI = + throwCryptoError $ + scalarDecodeLong + ( "\236\211\245\\\SUBc\DC2X\214\156\247\162\222\249\222\DC4\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\DLE" + :: ByteString + ) + sN = + throwCryptoError $ + scalarDecodeLong + ( "\237\211\245\\\SUBc\DC2X\214\156\247\162\222\249\222\DC4\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\NUL\DLE" + :: ByteString + ) + + s011 = throwCryptoError $ scalarDecodeLong ("\011" :: ByteString) + s123 = throwCryptoError $ scalarDecodeLong ("\123" :: ByteString) + s134 = throwCryptoError $ scalarDecodeLong ("\134" :: ByteString) + + p011 = + throwCryptoError $ + pointDecode + ( "\x13\x37\x03\x6a\xc3\x2d\x8f\x30\xd4\x58\x9c\x3c\x1c\x59\x58\x12\xce\x0f\xff\x40\xe3\x7c\x6f\x5a\x97\xab\x21\x3f\x31\x82\x90\xad" + :: ByteString + ) + p123 = + throwCryptoError $ + pointDecode + ( "\xc4\xb8\x00\xc8\x70\x10\xf9\x46\x83\x03\xde\xea\x87\x65\x03\xe8\x86\xbf\xde\x19\x00\xe9\xe8\x46\xfd\x4c\x3c\xd0\x9c\x1c\xbc\x9f" + :: ByteString + ) + p134 = + throwCryptoError $ + pointDecode + ( "\x51\x20\xab\xe0\x3c\xa2\xaf\x66\xc7\x7c\xa3\x20\xf0\xb2\x1f\xb5\x56\xf6\xb6\x5f\xdd\x7e\x32\x64\xc1\x4a\x30\xd9\x7b\xf7\xa7\x6f" + :: ByteString + ) + +-- Using : +-- +-- >>> import ed25519 +-- >>> encodepoint(scalarmult(B, 11)).encode('hex') +-- '1337036ac32d8f30d4589c3c1c595812ce0fff40e37c6f5a97ab213f318290ad' +-- >>> encodepoint(scalarmult(B, 123)).encode('hex') +-- 'c4b800c87010f9468303deea876503e886bfde1900e9e846fd4c3cd09c1cbc9f' +-- >>> encodepoint(scalarmult(B, 134)).encode('hex') +-- '5120abe03ca2af66c77ca320f0b21fb556f6b65fdd7e3264c14a30d97bf7a76f' diff --git a/tests/ECCSpec.hs b/tests/ECCSpec.hs new file mode 100644 --- /dev/null +++ b/tests/ECCSpec.hs @@ -0,0 +1,426 @@ +{-# LANGUAGE ExistentialQuantification #-} +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE OverloadedStrings #-} + +module ECCSpec (spec) where + +import Data.Either + +import qualified Crypto.ECC as ECC +import Crypto.Error + +import Data.ByteArray (convert) +import Data.ByteArray.Encoding + +import Imports + +data Curve + = forall curve. + (ECC.EllipticCurveDH curve, Show curve, Eq (ECC.Point curve)) => + Curve curve + +instance Show Curve where + showsPrec d (Curve curve) = showsPrec d curve + +instance Arbitrary Curve where + arbitrary = + elements + [ Curve ECC.Curve_P256R1 + , Curve ECC.Curve_P384R1 + , Curve ECC.Curve_P521R1 + , Curve ECC.Curve_X25519 + , Curve ECC.Curve_X448 + ] + +data CurveArith + = forall curve. (ECC.EllipticCurveBasepointArith curve, Show curve) => CurveArith curve + +instance Show CurveArith where + showsPrec d (CurveArith curve) = showsPrec d curve + +instance Arbitrary CurveArith where + arbitrary = + elements + [ CurveArith ECC.Curve_P256R1 + , CurveArith ECC.Curve_P384R1 + , CurveArith ECC.Curve_P521R1 + , CurveArith ECC.Curve_Edwards25519 + ] + +data VectorPoint = VectorPoint + { vpCurve :: Curve + , vpHex :: ByteString + , vpError :: Maybe CryptoError + } + +vectorsPoint :: [VectorPoint] +vectorsPoint = + [ VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = "" + , vpError = Just CryptoError_PointSizeInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = "00" + , vpError = Just CryptoError_PointFormatInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "0408edd7b50085a952172228aca391beebe9ba942a0ae9eb15bcc8d50795d1a5505221c7b9b3bb4310f165fc3ac3114339db8170ceae6697e0f9736698b33551b8" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "04216f25b00717d46deef3402628f6abf265bfa12aea515ae8f100ce415e251e72cd5cd8f47f613a0f4e0f4f9410dd9c85c149cffcb320c2d52bf550a397ec92e5" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "0421eba6080610926609bb8d52afd3331ed1b07e0ba4c1441a118b62497d3e85f39a50c865027cdd84298cdf094b7818f2a65ae59f46c971a32ab4ea3c2c93c959" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "0400d7fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a0001a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "040000fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a0001a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" + , vpError = Just CryptoError_PointCoordinatesInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "04d7fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a01a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" + , vpError = Just CryptoError_PublicKeySizeInvalid -- tests leading zeros + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P256R1 + , vpHex = + "040000d7fc4050dfe73475502d5d1fadc105d7725508f48da2cd4729bf191fd6490a000001a16f417a27530e756efeb4a228f02db878072b9f833e99a2821d85fa78fc" + , vpError = Just CryptoError_PublicKeySizeInvalid -- tests leading zeros + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = "" + , vpError = Just CryptoError_PointSizeInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = "00" + , vpError = Just CryptoError_PointFormatInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "0409281a103fb1773445e16eec86adb095e32928ccc9c806bd210c649712813bdb6cab40163a8cb163b578ea8dda5eb32cfb5208ebf0d31a6c590fa92f5a61f32dbc0d518b166ea5a9adf9dd21c1bd09932ca21c6a5725ca89542ac57b6a9eca6f" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "040c7b3fb575c1db7bc61fe7a456cc34a8289f41e167938a56e5ba2787723f3de2c645112705e13ed24f477730173935ca4e0ff468e7e0acf78a9f59dadff8193a0e23789eb3737730c089b27a0f94de7d95b8db4466d017fb21a5710d6ca85775" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "0438e7705220b60460194be63d21c8945be2a211957168fa60f26b2ad4e8f5cd96a7779e7edff4deda9ded63243c2127e273d4444edaaba03b79b6caafc5033432af13776f851c0c7e1080c60d7ee3b61740720ab98461813dab5fb8c31bfa9ed9" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "04000836bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884c00b1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "04000036bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884c00b1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" + , vpError = Just CryptoError_PointCoordinatesInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "040836bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884cb1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" + , vpError = Nothing -- ignores leading zeros + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P384R1 + , vpHex = + "0400000836bf09614bf5b3c0ffe9b0822a2cc109a90b13d4d3510ce14f766e7d90875ec4bc8d6bee11fc1fdf97473a67884c0000b1e2685367bdb846c95181b0f35a35cfbee04451122cc55a1e363acaa6c002e71b0b6ff7d0f5dc830a32f0e5086189" + , vpError = Nothing -- ignores leading zeros + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = "" + , vpError = Just CryptoError_PointSizeInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = "00" + , vpError = Just CryptoError_PointFormatInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "04000ce5c207335134567026063743df82c1b551a009cf616471f0e23fa9767a50cc7f8771ef13a65c49ce7e1cd1ac3ad721dcc3ddd35f98ae5d380a0832f87a9f0ca4012914911d6bea7f3c481d694fb1645be27c7b66b09b28e261f8030b3fb8206f6a95f6ad73db755765b64f592a799234f8f451cb787abe95b1a54991a799ad0d69da" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "04003a5e6c1ce3a6a323757005da17b357db991bd1ad835e6201411f458b5c2edb3c66786b727b7e15fbad7dd74a4b0eb542183b5242e5952061cb85e7229353eb0dc300aac2dbd5232d582481ba7a59a993eb04c4466a1b17ba0015b65c616ce8703e70880969d8d58e633acb29c3ca017eb1b88649387b867466090ce1a57c2b4f8376bb" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "04003e0659fe9498695a3d8c88b8e25fa8133c30ab10eccbe9094344c99924f89fb69d9b3acf03bf438328f9cba55fa28a05be9a7e18780706b3728abfee2592aeb86d0001ea5ff64f2ca7a6453c79f80550e971843e073f4f8fec75bad2e52a4483ebf1f16f43d0de27e1967ea22f9722527652fa74439fdc03a569fba29e2d6f7c012db6" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "040043f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def306000a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "040000f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def306000a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" + , vpError = Just CryptoError_PointCoordinatesInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "0443f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def3060a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" + , vpError = Nothing -- ignores leading zeros + } + , VectorPoint + { vpCurve = Curve ECC.Curve_P521R1 + , vpHex = + "04000043f91fd92d9ccd6d5584b265a2a775d222f4a41ff98190677d985e0889737cbe631d525835fe04faffcdebeccb783538280f4600ae82347b0470583abd9def30600000a2e9bdc34f42b134517fc1e961befea0affd1f9666361a039192082a892dd722931d5865b62b69d7369e74895120e540cb10030cccb6049d809fbcf3f54537b378" + , vpError = Nothing -- ignores leading zeros + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "" + , vpError = Just CryptoError_PublicKeySizeInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "22cd98c65fb50db3be0d6d359456c0cd3516952a6e7229ff672893944f703f10" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "23cd98c65fb50db3be0d6d359456c0cd3516952a6e7229ff672893944f703f10" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "0023cd98c65fb50db3be0d6d359456c0cd3516952a6e7229ff672893944f703f10" + , vpError = Just CryptoError_PublicKeySizeInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X448 + , vpHex = "" + , vpError = Just CryptoError_PublicKeySizeInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X448 + , vpHex = + "2b162c2fef165ecbb203e40975ae4424f0f8db25ab582cb96b2e5ffe90a31798b35480b594c99dc32b437e61a74f792d8ecf5fc3e8cfeb75" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X448 + , vpHex = + "2c162c2fef165ecbb203e40975ae4424f0f8db25ab582cb96b2e5ffe90a31798b35480b594c99dc32b437e61a74f792d8ecf5fc3e8cfeb75" + , vpError = Nothing + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X448 + , vpHex = + "002c162c2fef165ecbb203e40975ae4424f0f8db25ab582cb96b2e5ffe90a31798b35480b594c99dc32b437e61a74f792d8ecf5fc3e8cfeb75" + , vpError = Just CryptoError_PublicKeySizeInvalid + } + ] + +vectorsWeakPoint :: [VectorPoint] +vectorsWeakPoint = + [ VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "0000000000000000000000000000000000000000000000000000000000000000" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "0100000000000000000000000000000000000000000000000000000000000000" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "e0eb7a7c3b41b8ae1656e3faf19fc46ada098deb9c32b1fd866205165f49b800" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "5f9c95bca3508c24b1d0b1559c83ef5b04445cc4581c8e86d8224eddd09f1157" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "ecffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "edffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X25519 + , vpHex = "eeffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff7f" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X448 + , vpHex = + "0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + , VectorPoint + { vpCurve = Curve ECC.Curve_X448 + , vpHex = + "0100000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000" + , vpError = Just CryptoError_ScalarMultiplicationInvalid + } + ] + +vpEncodedPoint :: VectorPoint -> ByteString +vpEncodedPoint vector = fromRight (error "vpEncodedPoint") $ convertFromBase Base16 (vpHex vector) + +-- Wycheproof ecdh_secp256r1_ecpoint_test.json, tcIds 1 and 3. +vectorsECDH :: [(ByteString, ByteString, ByteString)] +vectorsECDH = + [ + ( "0612465c89a023ab17855b0a6bcebfd3febb53aef84138647b5352e02c10c346" + , "0462d5bd3372af75fe85a040715d0f502428e07046868b0bfdfa61d731afe44f26ac333a93a9e70a81cd5a95b5bf8d13990eb741c8c38872b4a07d275a014e30cf" + , "53020d908b0219328b658b525f26780e3ae12bcd952bb25a93bc0895e1714285" + ) + , + ( "0a0d622a47e48f6bc1038ace438c6f528aa00ad2bd1da5f13ee46bf5f633d71a" + , "0458fd4168a87795603e2b04390285bdca6e57de6027fe211dd9d25e2212d29e62080d36bd224d7405509295eed02a17150e03b314f96da37445b0d1d29377d12c" + , "0000000000000000000000000000000000000000000000000000000000000000" + ) + ] + +unhex :: ByteString -> ByteString +unhex = fromRight (error "unhex") . convertFromBase Base16 + +doECDHTest :: Show p => p -> (ByteString, ByteString, ByteString) -> Spec +doECDHTest i (priv, pub, shared) = + it (show i) $ + (convert <$> ECC.ecdh prx sk pk) `shouldBe` CryptoPassed (unhex shared) + where + prx = Just ECC.Curve_P256R1 + sk = throwCryptoError $ ECC.decodeScalar prx (unhex priv) + pk = throwCryptoError $ ECC.decodePoint prx (unhex pub) + +cryptoError :: CryptoFailable a -> Maybe CryptoError +cryptoError = onCryptoFailure Just (const Nothing) + +doPointDecodeTest :: Show p => p -> VectorPoint -> Spec +doPointDecodeTest i vector = + case vpCurve vector of + Curve curve -> + let prx = Just curve -- using Maybe as Proxy + in it + (show i) + ( cryptoError (ECC.decodePoint prx $ vpEncodedPoint vector) + `shouldBe` vpError vector + ) + +doWeakPointECDHTest :: Show p => p -> VectorPoint -> Spec +doWeakPointECDHTest i vector = + case vpCurve vector of + Curve curve -> it (show i) $ do + let prx = Just curve -- using Maybe as Proxy + public = throwCryptoError $ ECC.decodePoint prx $ vpEncodedPoint vector + keyPair <- ECC.curveGenerateKeyPair prx + cryptoError (ECC.ecdh prx (ECC.keypairGetPrivate keyPair) public) + `shouldBe` vpError vector + +spec :: Spec +spec = do + describe "decodePoint" $ zipWithM_ doPointDecodeTest [katZero ..] vectorsPoint + describe "ECDH KATs" $ zipWithM_ doECDHTest [katZero ..] vectorsECDH + describe "ECDH weak points" $ + sequence_ $ + zipWith doWeakPointECDHTest [katZero ..] vectorsWeakPoint + describe "property" $ do + prop "decodePoint.encodePoint==id" $ \testDRG (Curve curve) -> + let prx = Just curve -- using Maybe as Proxy + keyPair = withTestDRG testDRG $ ECC.curveGenerateKeyPair prx + p1 = ECC.keypairGetPublic keyPair + bs = ECC.encodePoint prx p1 :: ByteString + p2 = ECC.decodePoint prx bs + in CryptoPassed p1 == p2 + modifyMaxSuccess (const 20) $ prop "ECDH commutes" $ \testDRG (Curve curve) -> + let prx = Just curve -- using Maybe as Proxy + (alice, bob) = + withTestDRG testDRG $ + (,) + <$> ECC.curveGenerateKeyPair prx + <*> ECC.curveGenerateKeyPair prx + aliceShared = ECC.ecdh prx (ECC.keypairGetPrivate alice) (ECC.keypairGetPublic bob) + bobShared = ECC.ecdh prx (ECC.keypairGetPrivate bob) (ECC.keypairGetPublic alice) + aliceShared' = ECC.ecdhRaw prx (ECC.keypairGetPrivate alice) (ECC.keypairGetPublic bob) + bobShared' = ECC.ecdhRaw prx (ECC.keypairGetPrivate bob) (ECC.keypairGetPublic alice) + in aliceShared == bobShared + && aliceShared == CryptoPassed aliceShared' + && bobShared == CryptoPassed bobShared' + prop "decodeScalar.encodeScalar==id" $ \testDRG (CurveArith curve) -> + let prx = Just curve -- using Maybe as Proxy + s1 = withTestDRG testDRG $ ECC.curveGenerateScalar prx + bs = ECC.encodeScalar prx s1 :: ByteString + s2 = ECC.decodeScalar prx bs + in CryptoPassed s1 == s2 + prop "scalarFromInteger.scalarToInteger==id" $ \testDRG (CurveArith curve) -> + let prx = Just curve -- using Maybe as Proxy + s1 = withTestDRG testDRG $ ECC.curveGenerateScalar prx + bs = ECC.scalarToInteger prx s1 + s2 = ECC.scalarFromInteger prx bs + in CryptoPassed s1 == s2 + modifyMaxSuccess (const 20) $ prop "(a + b).P = a.P + b.P" $ \testDRG (CurveArith curve) -> + let prx = Just curve -- using Maybe as Proxy + (s, a, b) = + withTestDRG testDRG $ + (,,) + <$> ECC.curveGenerateScalar prx + <*> ECC.curveGenerateScalar prx + <*> ECC.curveGenerateScalar prx + p = ECC.pointBaseSmul prx s + in ECC.pointSmul prx (ECC.scalarAdd prx a b) p + == ECC.pointAdd prx (ECC.pointSmul prx a p) (ECC.pointSmul prx b p) + modifyMaxSuccess (const 20) $ prop "(a * b).P = a.(b.P)" $ \testDRG (CurveArith curve) -> + let prx = Just curve -- using Maybe as Proxy + (s, a, b) = + withTestDRG testDRG $ + (,,) + <$> ECC.curveGenerateScalar prx + <*> ECC.curveGenerateScalar prx + <*> ECC.curveGenerateScalar prx + p = ECC.pointBaseSmul prx s + in ECC.pointSmul prx (ECC.scalarMul prx a b) p + == ECC.pointSmul prx a (ECC.pointSmul prx b p) diff --git a/tests/ECDSA.hs b/tests/ECDSA.hs deleted file mode 100644 --- a/tests/ECDSA.hs +++ /dev/null @@ -1,132 +0,0 @@ -{-# LANGUAGE ExistentialQuantification #-} -{-# LANGUAGE FlexibleContexts #-} - -module ECDSA (tests) where - -import qualified Crypto.ECC as ECDSA -import Crypto.Error -import Crypto.Hash -import qualified Crypto.PubKey.ECC.ECDSA as ECC -import qualified Crypto.PubKey.ECC.Generate as ECC -import qualified Crypto.PubKey.ECC.Types as ECC -import qualified Crypto.PubKey.ECDSA as ECDSA -import qualified Data.ByteString as B -import Data.Maybe - -import Imports - -data Curve - = forall curve. - (ECDSA.EllipticCurveECDSA curve, Show (ECDSA.Scalar curve)) => - Curve curve ECC.Curve ECC.CurveName - -instance Show Curve where - showsPrec d (Curve _ _ name) = showsPrec d name - -instance Arbitrary Curve where - arbitrary = - elements - [ makeCurve ECDSA.Curve_P256R1 ECC.SEC_p256r1 - , makeCurve ECDSA.Curve_P384R1 ECC.SEC_p384r1 - , makeCurve ECDSA.Curve_P521R1 ECC.SEC_p521r1 - ] - where - makeCurve c name = Curve c (ECC.getCurveByName name) name - -arbitraryScalar :: ECC.Curve -> Gen Integer -arbitraryScalar curve = choose (1, n - 1) - where - n = ECC.ecc_n (ECC.common_curve curve) - -sigECDSAtoECC - :: ECDSA.EllipticCurveECDSA curve - => proxy curve -> ECDSA.Signature curve -> ECC.Signature -sigECDSAtoECC prx (ECDSA.Signature r s) = ECC.Signature (ECDSA.scalarToInteger prx r) (ECDSA.scalarToInteger prx s) - -normalizeECC :: ECC.Curve -> ECC.Signature -> ECC.Signature -normalizeECC curve (ECC.Signature r s) - | s <= n `div` 2 = ECC.Signature r s - | otherwise = ECC.Signature r (n - s) - where - n = ECC.ecc_n $ ECC.common_curve curve - -testRecover :: ECC.CurveName -> TestTree -testRecover name = testProperty (show name) $ \(ArbitraryBS0_2901 msg) -> do - let curve = ECC.getCurveByName name - let n = ECC.ecc_n $ ECC.common_curve curve - k <- choose (1, n - 1) - d <- choose (1, n - 1) - let key = ECC.PrivateKey curve d - let digest = hashWith SHA256 msg - let pub = - ECC.signExtendedDigestWith k key digest >>= \signature -> ECC.recoverDigest curve signature digest - pure $ - propertyHold - [eqTest "recovery" (Just $ ECC.generateQ curve d) (ECC.public_q <$> pub)] - -testNormalize :: ECC.CurveName -> TestTree -testNormalize name = testProperty (show name) $ \(ArbitraryBS0_2901 msg) -> do - let curve = ECC.getCurveByName name - let n = ECC.ecc_n $ ECC.common_curve curve - k <- choose (1, n - 1) - d <- choose (1, n - 1) - let key = ECC.PrivateKey curve d - let digest = hashWith SHA256 msg - let check = - ECC.signExtendedDigestWith k key digest >>= \s -> pure $ ECC.sign_s (ECC.signature s) <= n `div` 2 - pure $ propertyHold [eqTest "normalized" (Just True) check] - -tests :: TestTree -tests = - testGroup - "ECDSA" - [ localOption (QuickCheckTests 5) $ - testGroup - "verification" - [ testProperty "SHA1" $ propertyECDSA SHA1 - , testProperty "SHA224" $ propertyECDSA SHA224 - , testProperty "SHA256" $ propertyECDSA SHA256 - , testProperty "SHA384" $ propertyECDSA SHA384 - , testProperty "SHA512" $ propertyECDSA SHA512 - ] - , testGroup - "recovery" - [ localOption (QuickCheckTests 100) $ testRecover ECC.SEC_p128r1 - , localOption (QuickCheckTests 100) $ testRecover ECC.SEC_p128r2 - , localOption (QuickCheckTests 100) $ testRecover ECC.SEC_p256k1 - , localOption (QuickCheckTests 100) $ testRecover ECC.SEC_p256r1 - , localOption (QuickCheckTests 50) $ testRecover ECC.SEC_t131r1 - , localOption (QuickCheckTests 50) $ testRecover ECC.SEC_t131r2 - , localOption (QuickCheckTests 20) $ testRecover ECC.SEC_t233k1 - , localOption (QuickCheckTests 20) $ testRecover ECC.SEC_t233r1 - ] - , testGroup - "normalize" - [ localOption (QuickCheckTests 100) $ testNormalize ECC.SEC_p128r1 - , localOption (QuickCheckTests 100) $ testNormalize ECC.SEC_p128r2 - , localOption (QuickCheckTests 100) $ testNormalize ECC.SEC_p256k1 - , localOption (QuickCheckTests 100) $ testNormalize ECC.SEC_p256r1 - , localOption (QuickCheckTests 50) $ testNormalize ECC.SEC_t131r1 - , localOption (QuickCheckTests 50) $ testNormalize ECC.SEC_t131r2 - , localOption (QuickCheckTests 20) $ testNormalize ECC.SEC_t233k1 - , localOption (QuickCheckTests 20) $ testNormalize ECC.SEC_t233r1 - ] - ] - where - propertyECDSA hashAlg (Curve c curve _) (ArbitraryBS0_2901 msg) = do - d <- arbitraryScalar curve - kECC <- arbitraryScalar curve - let privECC = ECC.PrivateKey curve d - prx = Just c -- using Maybe as Proxy - kECDSA = throwCryptoError $ ECDSA.scalarFromInteger prx kECC - privECDSA = throwCryptoError $ ECDSA.scalarFromInteger prx d - pubECDSA = ECDSA.toPublic prx privECDSA - sigECC = fromJust $ ECC.signWith kECC privECC hashAlg msg - sigECDSA = fromJust $ ECDSA.signWith prx kECDSA privECDSA hashAlg msg - msg' = msg `B.append` B.singleton 42 - return $ - propertyHold - [ eqTest "signature" sigECC $ normalizeECC curve $ sigECDSAtoECC prx sigECDSA - , eqTest "verification" True (ECDSA.verify prx hashAlg pubECDSA sigECDSA msg) - , eqTest "alteration" False (ECDSA.verify prx hashAlg pubECDSA sigECDSA msg') - ] diff --git a/tests/ECDSASpec.hs b/tests/ECDSASpec.hs new file mode 100644 --- /dev/null +++ b/tests/ECDSASpec.hs @@ -0,0 +1,126 @@ +-- The binary curves are deprecated and still supported, so the tests +-- that hold them to their behaviour name them on purpose. +{-# OPTIONS_GHC -Wno-deprecations #-} +{-# LANGUAGE ExistentialQuantification #-} +{-# LANGUAGE FlexibleContexts #-} + +module ECDSASpec (spec) where + +import qualified Crypto.ECC as ECDSA +import Crypto.Error +import Crypto.Hash +import qualified Crypto.PubKey.ECC.ECDSA as ECC +import qualified Crypto.PubKey.ECC.Generate as ECC +import qualified Crypto.PubKey.ECC.Types as ECC +import qualified Crypto.PubKey.ECDSA as ECDSA +import qualified Data.ByteString as B +import Data.Maybe + +import Imports + +data Curve + = forall curve. + (ECDSA.EllipticCurveECDSA curve, Show (ECDSA.Scalar curve)) => + Curve curve ECC.Curve ECC.CurveName + +instance Show Curve where + showsPrec d (Curve _ _ name) = showsPrec d name + +instance Arbitrary Curve where + arbitrary = + elements + [ makeCurve ECDSA.Curve_P256R1 ECC.SEC_p256r1 + , makeCurve ECDSA.Curve_P384R1 ECC.SEC_p384r1 + , makeCurve ECDSA.Curve_P521R1 ECC.SEC_p521r1 + ] + where + makeCurve c name = Curve c (ECC.getCurveByName name) name + +arbitraryScalar :: ECC.Curve -> Gen Integer +arbitraryScalar curve = choose (1, n - 1) + where + n = ECC.ecc_n (ECC.common_curve curve) + +sigECDSAtoECC + :: ECDSA.EllipticCurveECDSA curve + => proxy curve -> ECDSA.Signature curve -> ECC.Signature +sigECDSAtoECC prx (ECDSA.Signature r s) = ECC.Signature (ECDSA.scalarToInteger prx r) (ECDSA.scalarToInteger prx s) + +normalizeECC :: ECC.Curve -> ECC.Signature -> ECC.Signature +normalizeECC curve (ECC.Signature r s) + | s <= n `div` 2 = ECC.Signature r s + | otherwise = ECC.Signature r (n - s) + where + n = ECC.ecc_n $ ECC.common_curve curve + +testRecover :: ECC.CurveName -> Spec +testRecover name = prop (show name) $ \(ArbitraryBS0_2901 msg) -> do + let curve = ECC.getCurveByName name + let n = ECC.ecc_n $ ECC.common_curve curve + k <- choose (1, n - 1) + d <- choose (1, n - 1) + let key = ECC.PrivateKey curve d + let digest = hashWith SHA256 msg + let pub = + ECC.signExtendedDigestWith k key digest >>= \signature -> ECC.recoverDigest curve signature digest + pure $ + propertyHold + [eqTest "recovery" (Just $ ECC.generateQ curve d) (ECC.public_q <$> pub)] + +testNormalize :: ECC.CurveName -> Spec +testNormalize name = prop (show name) $ \(ArbitraryBS0_2901 msg) -> do + let curve = ECC.getCurveByName name + let n = ECC.ecc_n $ ECC.common_curve curve + k <- choose (1, n - 1) + d <- choose (1, n - 1) + let key = ECC.PrivateKey curve d + let digest = hashWith SHA256 msg + let check = + ECC.signExtendedDigestWith k key digest >>= \s -> pure $ ECC.sign_s (ECC.signature s) <= n `div` 2 + pure $ propertyHold [eqTest "normalized" (Just True) check] + +spec :: Spec +spec = do + modifyMaxSuccess (const 5) $ + describe "verification" $ do + prop "SHA1" $ propertyECDSA SHA1 + prop "SHA224" $ propertyECDSA SHA224 + prop "SHA256" $ propertyECDSA SHA256 + prop "SHA384" $ propertyECDSA SHA384 + prop "SHA512" $ propertyECDSA SHA512 + describe "recovery" $ do + modifyMaxSuccess (const 100) $ testRecover ECC.SEC_p128r1 + modifyMaxSuccess (const 100) $ testRecover ECC.SEC_p128r2 + modifyMaxSuccess (const 100) $ testRecover ECC.SEC_p256k1 + modifyMaxSuccess (const 100) $ testRecover ECC.SEC_p256r1 + modifyMaxSuccess (const 50) $ testRecover ECC.SEC_t131r1 + modifyMaxSuccess (const 50) $ testRecover ECC.SEC_t131r2 + modifyMaxSuccess (const 20) $ testRecover ECC.SEC_t233k1 + modifyMaxSuccess (const 20) $ testRecover ECC.SEC_t233r1 + describe "normalize" $ do + modifyMaxSuccess (const 100) $ testNormalize ECC.SEC_p128r1 + modifyMaxSuccess (const 100) $ testNormalize ECC.SEC_p128r2 + modifyMaxSuccess (const 100) $ testNormalize ECC.SEC_p256k1 + modifyMaxSuccess (const 100) $ testNormalize ECC.SEC_p256r1 + modifyMaxSuccess (const 50) $ testNormalize ECC.SEC_t131r1 + modifyMaxSuccess (const 50) $ testNormalize ECC.SEC_t131r2 + modifyMaxSuccess (const 20) $ testNormalize ECC.SEC_t233k1 + modifyMaxSuccess (const 20) $ testNormalize ECC.SEC_t233r1 + where + propertyECDSA hashAlg (Curve c curve _) (ArbitraryBS0_2901 msg) = do + d <- arbitraryScalar curve + kECC <- arbitraryScalar curve + let privECC = ECC.PrivateKey curve d + prx = Just c -- using Maybe as Proxy + kECDSA = throwCryptoError $ ECDSA.scalarFromInteger prx kECC + privECDSA = throwCryptoError $ ECDSA.scalarFromInteger prx d + pubECDSA = ECDSA.toPublic prx privECDSA + sigECC = fromJust $ ECC.signWith kECC privECC hashAlg msg + sigECDSA = fromJust $ ECDSA.signWith prx kECDSA privECDSA hashAlg msg + msg' = msg `B.append` B.singleton 42 + return $ + propertyHold + [ eqTest "signature" sigECC $ normalizeECC curve $ sigECDSAtoECC prx sigECDSA + , eqTest "verification" True (ECDSA.verify prx hashAlg pubECDSA sigECDSA msg) + , eqTest "alteration" False (ECDSA.verify prx hashAlg pubECDSA sigECDSA msg') + ] diff --git a/tests/Ed25519Spec.hs b/tests/Ed25519Spec.hs new file mode 100644 --- /dev/null +++ b/tests/Ed25519Spec.hs @@ -0,0 +1,137 @@ +{-# LANGUAGE BangPatterns #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE RecordWildCards #-} + +module Ed25519Spec (spec) where + +import Crypto.Error +import qualified Crypto.PubKey.Ed25519 as Ed25519 +import Data.ByteArray.Encoding (Base (Base16), convertFromBase) +import Imports + +data Vec = Vec + { vecSec :: ByteString + , vecPub :: ByteString + , vecMsg :: ByteString + , vecSig :: ByteString + } + deriving (Show, Eq) + +vectors = + [ Vec + { vecSec = + "\x9d\x61\xb1\x9d\xef\xfd\x5a\x60\xba\x84\x4a\xf4\x92\xec\x2c\xc4\x44\x49\xc5\x69\x7b\x32\x69\x19\x70\x3b\xac\x03\x1c\xae\x7f\x60" + , vecPub = + "\xd7\x5a\x98\x01\x82\xb1\x0a\xb7\xd5\x4b\xfe\xd3\xc9\x64\x07\x3a\x0e\xe1\x72\xf3\xda\xa6\x23\x25\xaf\x02\x1a\x68\xf7\x07\x51\x1a" + , vecMsg = "" + , vecSig = + "\xe5\x56\x43\x00\xc3\x60\xac\x72\x90\x86\xe2\xcc\x80\x6e\x82\x8a\x84\x87\x7f\x1e\xb8\xe5\xd9\x74\xd8\x73\xe0\x65\x22\x49\x01\x55\x5f\xb8\x82\x15\x90\xa3\x3b\xac\xc6\x1e\x39\x70\x1c\xf9\xb4\x6b\xd2\x5b\xf5\xf0\x59\x5b\xbe\x24\x65\x51\x41\x43\x8e\x7a\x10\x0b" + } + , Vec + { vecSec = + "\x4c\xcd\x08\x9b\x28\xff\x96\xda\x9d\xb6\xc3\x46\xec\x11\x4e\x0f\x5b\x8a\x31\x9f\x35\xab\xa6\x24\xda\x8c\xf6\xed\x4f\xb8\xa6\xfb" + , vecPub = + "\x3d\x40\x17\xc3\xe8\x43\x89\x5a\x92\xb7\x0a\xa7\x4d\x1b\x7e\xbc\x9c\x98\x2c\xcf\x2e\xc4\x96\x8c\xc0\xcd\x55\xf1\x2a\xf4\x66\x0c" + , vecMsg = "\x72" + , vecSig = + "\x92\xa0\x09\xa9\xf0\xd4\xca\xb8\x72\x0e\x82\x0b\x5f\x64\x25\x40\xa2\xb2\x7b\x54\x16\x50\x3f\x8f\xb3\x76\x22\x23\xeb\xdb\x69\xda\x08\x5a\xc1\xe4\x3e\x15\x99\x6e\x45\x8f\x36\x13\xd0\xf1\x1d\x8c\x38\x7b\x2e\xae\xb4\x30\x2a\xee\xb0\x0d\x29\x16\x12\xbb\x0c\x00" + } + , Vec + { vecSec = + "\xc5\xaa\x8d\xf4\x3f\x9f\x83\x7b\xed\xb7\x44\x2f\x31\xdc\xb7\xb1\x66\xd3\x85\x35\x07\x6f\x09\x4b\x85\xce\x3a\x2e\x0b\x44\x58\xf7" + , vecPub = + "\xfc\x51\xcd\x8e\x62\x18\xa1\xa3\x8d\xa4\x7e\xd0\x02\x30\xf0\x58\x08\x16\xed\x13\xba\x33\x03\xac\x5d\xeb\x91\x15\x48\x90\x80\x25" + , vecMsg = "\xaf\x82" + , vecSig = + "\x62\x91\xd6\x57\xde\xec\x24\x02\x48\x27\xe6\x9c\x3a\xbe\x01\xa3\x0c\xe5\x48\xa2\x84\x74\x3a\x44\x5e\x36\x80\xd7\xdb\x5a\xc3\xac\x18\xff\x9b\x53\x8d\x16\xf2\x90\xae\x67\xf7\x60\x98\x4d\xc6\x59\x4a\x7c\x15\xe9\x71\x6e\xd2\x8d\xc0\x27\xbe\xce\xea\x1e\xc4\x0a" + } + , Vec + { vecSec = + "\xf5\xe5\x76\x7c\xf1\x53\x31\x95\x17\x63\x0f\x22\x68\x76\xb8\x6c\x81\x60\xcc\x58\x3b\xc0\x13\x74\x4c\x6b\xf2\x55\xf5\xcc\x0e\xe5" + , vecPub = + "\x27\x81\x17\xfc\x14\x4c\x72\x34\x0f\x67\xd0\xf2\x31\x6e\x83\x86\xce\xff\xbf\x2b\x24\x28\xc9\xc5\x1f\xef\x7c\x59\x7f\x1d\x42\x6e" + , vecMsg = + "\x08\xb8\xb2\xb7\x33\x42\x42\x43\x76\x0f\xe4\x26\xa4\xb5\x49\x08\x63\x21\x10\xa6\x6c\x2f\x65\x91\xea\xbd\x33\x45\xe3\xe4\xeb\x98\xfa\x6e\x26\x4b\xf0\x9e\xfe\x12\xee\x50\xf8\xf5\x4e\x9f\x77\xb1\xe3\x55\xf6\xc5\x05\x44\xe2\x3f\xb1\x43\x3d\xdf\x73\xbe\x84\xd8\x79\xde\x7c\x00\x46\xdc\x49\x96\xd9\xe7\x73\xf4\xbc\x9e\xfe\x57\x38\x82\x9a\xdb\x26\xc8\x1b\x37\xc9\x3a\x1b\x27\x0b\x20\x32\x9d\x65\x86\x75\xfc\x6e\xa5\x34\xe0\x81\x0a\x44\x32\x82\x6b\xf5\x8c\x94\x1e\xfb\x65\xd5\x7a\x33\x8b\xbd\x2e\x26\x64\x0f\x89\xff\xbc\x1a\x85\x8e\xfc\xb8\x55\x0e\xe3\xa5\xe1\x99\x8b\xd1\x77\xe9\x3a\x73\x63\xc3\x44\xfe\x6b\x19\x9e\xe5\xd0\x2e\x82\xd5\x22\xc4\xfe\xba\x15\x45\x2f\x80\x28\x8a\x82\x1a\x57\x91\x16\xec\x6d\xad\x2b\x3b\x31\x0d\xa9\x03\x40\x1a\xa6\x21\x00\xab\x5d\x1a\x36\x55\x3e\x06\x20\x3b\x33\x89\x0c\xc9\xb8\x32\xf7\x9e\xf8\x05\x60\xcc\xb9\xa3\x9c\xe7\x67\x96\x7e\xd6\x28\xc6\xad\x57\x3c\xb1\x16\xdb\xef\xef\xd7\x54\x99\xda\x96\xbd\x68\xa8\xa9\x7b\x92\x8a\x8b\xbc\x10\x3b\x66\x21\xfc\xde\x2b\xec\xa1\x23\x1d\x20\x6b\xe6\xcd\x9e\xc7\xaf\xf6\xf6\xc9\x4f\xcd\x72\x04\xed\x34\x55\xc6\x8c\x83\xf4\xa4\x1d\xa4\xaf\x2b\x74\xef\x5c\x53\xf1\xd8\xac\x70\xbd\xcb\x7e\xd1\x85\xce\x81\xbd\x84\x35\x9d\x44\x25\x4d\x95\x62\x9e\x98\x55\xa9\x4a\x7c\x19\x58\xd1\xf8\xad\xa5\xd0\x53\x2e\xd8\xa5\xaa\x3f\xb2\xd1\x7b\xa7\x0e\xb6\x24\x8e\x59\x4e\x1a\x22\x97\xac\xbb\xb3\x9d\x50\x2f\x1a\x8c\x6e\xb6\xf1\xce\x22\xb3\xde\x1a\x1f\x40\xcc\x24\x55\x41\x19\xa8\x31\xa9\xaa\xd6\x07\x9c\xad\x88\x42\x5d\xe6\xbd\xe1\xa9\x18\x7e\xbb\x60\x92\xcf\x67\xbf\x2b\x13\xfd\x65\xf2\x70\x88\xd7\x8b\x7e\x88\x3c\x87\x59\xd2\xc4\xf5\xc6\x5a\xdb\x75\x53\x87\x8a\xd5\x75\xf9\xfa\xd8\x78\xe8\x0a\x0c\x9b\xa6\x3b\xcb\xcc\x27\x32\xe6\x94\x85\xbb\xc9\xc9\x0b\xfb\xd6\x24\x81\xd9\x08\x9b\xec\xcf\x80\xcf\xe2\xdf\x16\xa2\xcf\x65\xbd\x92\xdd\x59\x7b\x07\x07\xe0\x91\x7a\xf4\x8b\xbb\x75\xfe\xd4\x13\xd2\x38\xf5\x55\x5a\x7a\x56\x9d\x80\xc3\x41\x4a\x8d\x08\x59\xdc\x65\xa4\x61\x28\xba\xb2\x7a\xf8\x7a\x71\x31\x4f\x31\x8c\x78\x2b\x23\xeb\xfe\x80\x8b\x82\xb0\xce\x26\x40\x1d\x2e\x22\xf0\x4d\x83\xd1\x25\x5d\xc5\x1a\xdd\xd3\xb7\x5a\x2b\x1a\xe0\x78\x45\x04\xdf\x54\x3a\xf8\x96\x9b\xe3\xea\x70\x82\xff\x7f\xc9\x88\x8c\x14\x4d\xa2\xaf\x58\x42\x9e\xc9\x60\x31\xdb\xca\xd3\xda\xd9\xaf\x0d\xcb\xaa\xaf\x26\x8c\xb8\xfc\xff\xea\xd9\x4f\x3c\x7c\xa4\x95\xe0\x56\xa9\xb4\x7a\xcd\xb7\x51\xfb\x73\xe6\x66\xc6\xc6\x55\xad\xe8\x29\x72\x97\xd0\x7a\xd1\xba\x5e\x43\xf1\xbc\xa3\x23\x01\x65\x13\x39\xe2\x29\x04\xcc\x8c\x42\xf5\x8c\x30\xc0\x4a\xaf\xdb\x03\x8d\xda\x08\x47\xdd\x98\x8d\xcd\xa6\xf3\xbf\xd1\x5c\x4b\x4c\x45\x25\x00\x4a\xa0\x6e\xef\xf8\xca\x61\x78\x3a\xac\xec\x57\xfb\x3d\x1f\x92\xb0\xfe\x2f\xd1\xa8\x5f\x67\x24\x51\x7b\x65\xe6\x14\xad\x68\x08\xd6\xf6\xee\x34\xdf\xf7\x31\x0f\xdc\x82\xae\xbf\xd9\x04\xb0\x1e\x1d\xc5\x4b\x29\x27\x09\x4b\x2d\xb6\x8d\x6f\x90\x3b\x68\x40\x1a\xde\xbf\x5a\x7e\x08\xd7\x8f\xf4\xef\x5d\x63\x65\x3a\x65\x04\x0c\xf9\xbf\xd4\xac\xa7\x98\x4a\x74\xd3\x71\x45\x98\x67\x80\xfc\x0b\x16\xac\x45\x16\x49\xde\x61\x88\xa7\xdb\xdf\x19\x1f\x64\xb5\xfc\x5e\x2a\xb4\x7b\x57\xf7\xf7\x27\x6c\xd4\x19\xc1\x7a\x3c\xa8\xe1\xb9\x39\xae\x49\xe4\x88\xac\xba\x6b\x96\x56\x10\xb5\x48\x01\x09\xc8\xb1\x7b\x80\xe1\xb7\xb7\x50\xdf\xc7\x59\x8d\x5d\x50\x11\xfd\x2d\xcc\x56\x00\xa3\x2e\xf5\xb5\x2a\x1e\xcc\x82\x0e\x30\x8a\xa3\x42\x72\x1a\xac\x09\x43\xbf\x66\x86\xb6\x4b\x25\x79\x37\x65\x04\xcc\xc4\x93\xd9\x7e\x6a\xed\x3f\xb0\xf9\xcd\x71\xa4\x3d\xd4\x97\xf0\x1f\x17\xc0\xe2\xcb\x37\x97\xaa\x2a\x2f\x25\x66\x56\x16\x8e\x6c\x49\x6a\xfc\x5f\xb9\x32\x46\xf6\xb1\x11\x63\x98\xa3\x46\xf1\xa6\x41\xf3\xb0\x41\xe9\x89\xf7\x91\x4f\x90\xcc\x2c\x7f\xff\x35\x78\x76\xe5\x06\xb5\x0d\x33\x4b\xa7\x7c\x22\x5b\xc3\x07\xba\x53\x71\x52\xf3\xf1\x61\x0e\x4e\xaf\xe5\x95\xf6\xd9\xd9\x0d\x11\xfa\xa9\x33\xa1\x5e\xf1\x36\x95\x46\x86\x8a\x7f\x3a\x45\xa9\x67\x68\xd4\x0f\xd9\xd0\x34\x12\xc0\x91\xc6\x31\x5c\xf4\xfd\xe7\xcb\x68\x60\x69\x37\x38\x0d\xb2\xea\xaa\x70\x7b\x4c\x41\x85\xc3\x2e\xdd\xcd\xd3\x06\x70\x5e\x4d\xc1\xff\xc8\x72\xee\xee\x47\x5a\x64\xdf\xac\x86\xab\xa4\x1c\x06\x18\x98\x3f\x87\x41\xc5\xef\x68\xd3\xa1\x01\xe8\xa3\xb8\xca\xc6\x0c\x90\x5c\x15\xfc\x91\x08\x40\xb9\x4c\x00\xa0\xb9\xd0" + , vecSig = + "\x0a\xab\x4c\x90\x05\x01\xb3\xe2\x4d\x7c\xdf\x46\x63\x32\x6a\x3a\x87\xdf\x5e\x48\x43\xb2\xcb\xdb\x67\xcb\xf6\xe4\x60\xfe\xc3\x50\xaa\x53\x71\xb1\x50\x8f\x9f\x45\x28\xec\xea\x23\xc4\x36\xd9\x4b\x5e\x8f\xcd\x4f\x68\x1e\x30\xa6\xac\x00\xa9\x70\x4a\x18\x8a\x03" + } + , Vec + { vecSec = + "\x83\x3f\xe6\x24\x09\x23\x7b\x9d\x62\xec\x77\x58\x75\x20\x91\x1e\x9a\x75\x9c\xec\x1d\x19\x75\x5b\x7d\xa9\x01\xb9\x6d\xca\x3d\x42" + , vecPub = + "\xec\x17\x2b\x93\xad\x5e\x56\x3b\xf4\x93\x2c\x70\xe1\x24\x50\x34\xc3\x54\x67\xef\x2e\xfd\x4d\x64\xeb\xf8\x19\x68\x34\x67\xe2\xbf" + , vecMsg = + "\xdd\xaf\x35\xa1\x93\x61\x7a\xba\xcc\x41\x73\x49\xae\x20\x41\x31\x12\xe6\xfa\x4e\x89\xa9\x7e\xa2\x0a\x9e\xee\xe6\x4b\x55\xd3\x9a\x21\x92\x99\x2a\x27\x4f\xc1\xa8\x36\xba\x3c\x23\xa3\xfe\xeb\xbd\x45\x4d\x44\x23\x64\x3c\xe8\x0e\x2a\x9a\xc9\x4f\xa5\x4c\xa4\x9f" + , vecSig = + "\xdc\x2a\x44\x59\xe7\x36\x96\x33\xa5\x2b\x1b\xf2\x77\x83\x9a\x00\x20\x10\x09\xa3\xef\xbf\x3e\xcb\x69\xbe\xa2\x18\x6c\x26\xb5\x89\x09\x35\x1f\xc9\xac\x90\xb3\xec\xfd\xfb\xc7\xc6\x64\x31\xe0\x30\x3d\xca\x17\x9c\x13\x8a\xc1\x7a\xd9\xbe\xf1\x17\x73\x31\xa7\x04" + } + ] + +doPublicKeyTest i vec = it (show i) (Ed25519.toPublic sec `shouldBe` pub) + where + !pub = throwCryptoError $ Ed25519.publicKey (vecPub vec) + !sec = throwCryptoError $ Ed25519.secretKey (vecSec vec) + +doSignatureTest i vec = it (show i) (Ed25519.sign sec pub (vecMsg vec) `shouldBe` sig) + where + !sig = throwCryptoError $ Ed25519.signature (vecSig vec) + !pub = throwCryptoError $ Ed25519.publicKey (vecPub vec) + !sec = throwCryptoError $ Ed25519.secretKey (vecSec vec) + +doVerifyTest i vec = it (show i) (Ed25519.verify pub (vecMsg vec) sig `shouldBe` True) + where + !sig = throwCryptoError $ Ed25519.signature (vecSig vec) + !pub = throwCryptoError $ Ed25519.publicKey (vecPub vec) + +unhex :: ByteString -> ByteString +unhex = either error id . convertFromBase Base16 + +-- | Invalid signatures from Wycheproof's ed25519_test.json. +data NegVec = NegVec + { negTc :: Int + , negWhy :: String + , negPub :: ByteString + , negMsg :: ByteString + , negSig :: ByteString + } + +negVectors = + [ NegVec + { negTc = 63 + , negWhy = "s replaced by s + L" + , negPub = + unhex "7d4d0e7f6153a69b6242b522abbee685fda4420f8834b108c3bdae369ef549fa" + , negMsg = unhex "54657374" + , negSig = + unhex + "7c38e026f29e14aabd059a0f2db8b0cd783040609a8be684db12f82a27774ab067654bce3832c2d76f8f6f5dafc08d9339d4eef676573336a5c51eb6f946b31d" + } + , NegVec + { negTc = 85 + , negWhy = "s just above the bound" + , negPub = + unhex "100fdf47fb94f1536a4f7c3fda27383fa03375a8f527c537e6f1703c47f94f86" + , negMsg = + unhex "6a0bc2b0057cedfc0fa2e3f7f7d39279b30f454a69dfd1117c758d86b19d85e0" + , negSig = + unhex + "0971f86d2c9c78582524a103cb9cf949522ae528f8054dc20107d999be673ff4e25ebf2f2928766b1248bec6e91697775f8446639ede46ad4df4053000000010" + } + ] + +doNegVerifyTest :: NegVec -> Spec +doNegVerifyTest NegVec{..} = + it + (show negTc ++ ": " ++ negWhy) + (Ed25519.verify pub negMsg sig `shouldBe` False) + where + !sig = throwCryptoError $ Ed25519.signature negSig + !pub = throwCryptoError $ Ed25519.publicKey negPub + +spec :: Spec +spec = do + it "gen secretkey" (Ed25519.generateSecretKey *> pure () :: Expectation) + describe "gen publickey" $ zipWithM_ doPublicKeyTest [katZero ..] vectors + describe "gen signature" $ zipWithM_ doSignatureTest [katZero ..] vectors + describe "verify sig" $ zipWithM_ doVerifyTest [katZero ..] vectors + describe "reject non-canonical scalar" $ mapM_ doNegVerifyTest negVectors diff --git a/tests/Ed448Spec.hs b/tests/Ed448Spec.hs new file mode 100644 --- /dev/null +++ b/tests/Ed448Spec.hs @@ -0,0 +1,117 @@ +{-# LANGUAGE BangPatterns #-} +{-# LANGUAGE OverloadedStrings #-} + +module Ed448Spec (spec) where + +import Crypto.Error +import qualified Crypto.PubKey.Ed448 as Ed448 +import Imports + +data Vec = Vec + { vecSec :: ByteString + , vecPub :: ByteString + , vecMsg :: ByteString + , vecSig :: ByteString + } + deriving (Show, Eq) + +vectors = + [ Vec + { vecSec = + "\x6c\x82\xa5\x62\xcb\x80\x8d\x10\xd6\x32\xbe\x89\xc8\x51\x3e\xbf\x6c\x92\x9f\x34\xdd\xfa\x8c\x9f\x63\xc9\x96\x0e\xf6\xe3\x48\xa3\x52\x8c\x8a\x3f\xcc\x2f\x04\x4e\x39\xa3\xfc\x5b\x94\x49\x2f\x8f\x03\x2e\x75\x49\xa2\x00\x98\xf9\x5b" + , vecPub = + "\x5f\xd7\x44\x9b\x59\xb4\x61\xfd\x2c\xe7\x87\xec\x61\x6a\xd4\x6a\x1d\xa1\x34\x24\x85\xa7\x0e\x1f\x8a\x0e\xa7\x5d\x80\xe9\x67\x78\xed\xf1\x24\x76\x9b\x46\xc7\x06\x1b\xd6\x78\x3d\xf1\xe5\x0f\x6c\xd1\xfa\x1a\xbe\xaf\xe8\x25\x61\x80" + , vecMsg = "" + , vecSig = + "\x53\x3a\x37\xf6\xbb\xe4\x57\x25\x1f\x02\x3c\x0d\x88\xf9\x76\xae\x2d\xfb\x50\x4a\x84\x3e\x34\xd2\x07\x4f\xd8\x23\xd4\x1a\x59\x1f\x2b\x23\x3f\x03\x4f\x62\x82\x81\xf2\xfd\x7a\x22\xdd\xd4\x7d\x78\x28\xc5\x9b\xd0\xa2\x1b\xfd\x39\x80\xff\x0d\x20\x28\xd4\xb1\x8a\x9d\xf6\x3e\x00\x6c\x5d\x1c\x2d\x34\x5b\x92\x5d\x8d\xc0\x0b\x41\x04\x85\x2d\xb9\x9a\xc5\xc7\xcd\xda\x85\x30\xa1\x13\xa0\xf4\xdb\xb6\x11\x49\xf0\x5a\x73\x63\x26\x8c\x71\xd9\x58\x08\xff\x2e\x65\x26\x00" + } + , Vec + { vecSec = + "\xc4\xea\xb0\x5d\x35\x70\x07\xc6\x32\xf3\xdb\xb4\x84\x89\x92\x4d\x55\x2b\x08\xfe\x0c\x35\x3a\x0d\x4a\x1f\x00\xac\xda\x2c\x46\x3a\xfb\xea\x67\xc5\xe8\xd2\x87\x7c\x5e\x3b\xc3\x97\xa6\x59\x94\x9e\xf8\x02\x1e\x95\x4e\x0a\x12\x27\x4e" + , vecPub = + "\x43\xba\x28\xf4\x30\xcd\xff\x45\x6a\xe5\x31\x54\x5f\x7e\xcd\x0a\xc8\x34\xa5\x5d\x93\x58\xc0\x37\x2b\xfa\x0c\x6c\x67\x98\xc0\x86\x6a\xea\x01\xeb\x00\x74\x28\x02\xb8\x43\x8e\xa4\xcb\x82\x16\x9c\x23\x51\x60\x62\x7b\x4c\x3a\x94\x80" + , vecMsg = "\x03" + , vecSig = + "\x26\xb8\xf9\x17\x27\xbd\x62\x89\x7a\xf1\x5e\x41\xeb\x43\xc3\x77\xef\xb9\xc6\x10\xd4\x8f\x23\x35\xcb\x0b\xd0\x08\x78\x10\xf4\x35\x25\x41\xb1\x43\xc4\xb9\x81\xb7\xe1\x8f\x62\xde\x8c\xcd\xf6\x33\xfc\x1b\xf0\x37\xab\x7c\xd7\x79\x80\x5e\x0d\xbc\xc0\xaa\xe1\xcb\xce\xe1\xaf\xb2\xe0\x27\xdf\x36\xbc\x04\xdc\xec\xbf\x15\x43\x36\xc1\x9f\x0a\xf7\xe0\xa6\x47\x29\x05\xe7\x99\xf1\x95\x3d\x2a\x0f\xf3\x34\x8a\xb2\x1a\xa4\xad\xaf\xd1\xd2\x34\x44\x1c\xf8\x07\xc0\x3a\x00" + } + , Vec + { vecSec = + "\xcd\x23\xd2\x4f\x71\x42\x74\xe7\x44\x34\x32\x37\xb9\x32\x90\xf5\x11\xf6\x42\x5f\x98\xe6\x44\x59\xff\x20\x3e\x89\x85\x08\x3f\xfd\xf6\x05\x00\x55\x3a\xbc\x0e\x05\xcd\x02\x18\x4b\xdb\x89\xc4\xcc\xd6\x7e\x18\x79\x51\x26\x7e\xb3\x28" + , vecPub = + "\xdc\xea\x9e\x78\xf3\x5a\x1b\xf3\x49\x9a\x83\x1b\x10\xb8\x6c\x90\xaa\xc0\x1c\xd8\x4b\x67\xa0\x10\x9b\x55\xa3\x6e\x93\x28\xb1\xe3\x65\xfc\xe1\x61\xd7\x1c\xe7\x13\x1a\x54\x3e\xa4\xcb\x5f\x7e\x9f\x1d\x8b\x00\x69\x64\x47\x00\x14\x00" + , vecMsg = "\x0c\x3e\x54\x40\x74\xec\x63\xb0\x26\x5e\x0c" + , vecSig = + "\x1f\x0a\x88\x88\xce\x25\xe8\xd4\x58\xa2\x11\x30\x87\x9b\x84\x0a\x90\x89\xd9\x99\xaa\xba\x03\x9e\xaf\x3e\x3a\xfa\x09\x0a\x09\xd3\x89\xdb\xa8\x2c\x4f\xf2\xae\x8a\xc5\xcd\xfb\x7c\x55\xe9\x4d\x5d\x96\x1a\x29\xfe\x01\x09\x94\x1e\x00\xb8\xdb\xde\xea\x6d\x3b\x05\x10\x68\xdf\x72\x54\xc0\xcd\xc1\x29\xcb\xe6\x2d\xb2\xdc\x95\x7d\xbb\x47\xb5\x1f\xd3\xf2\x13\xfb\x86\x98\xf0\x64\x77\x42\x50\xa5\x02\x89\x61\xc9\xbf\x8f\xfd\x97\x3f\xe5\xd5\xc2\x06\x49\x2b\x14\x0e\x00" + } + , Vec + { vecSec = + "\x25\x8c\xdd\x4a\xda\x32\xed\x9c\x9f\xf5\x4e\x63\x75\x6a\xe5\x82\xfb\x8f\xab\x2a\xc7\x21\xf2\xc8\xe6\x76\xa7\x27\x68\x51\x3d\x93\x9f\x63\xdd\xdb\x55\x60\x91\x33\xf2\x9a\xdf\x86\xec\x99\x29\xdc\xcb\x52\xc1\xc5\xfd\x2f\xf7\xe2\x1b" + , vecPub = + "\x3b\xa1\x6d\xa0\xc6\xf2\xcc\x1f\x30\x18\x77\x40\x75\x6f\x5e\x79\x8d\x6b\xc5\xfc\x01\x5d\x7c\x63\xcc\x95\x10\xee\x3f\xd4\x4a\xdc\x24\xd8\xe9\x68\xb6\xe4\x6e\x6f\x94\xd1\x9b\x94\x53\x61\x72\x6b\xd7\x5e\x14\x9e\xf0\x98\x17\xf5\x80" + , vecMsg = "\x64\xa6\x5f\x3c\xde\xdc\xdd\x66\x81\x1e\x29\x15" + , vecSig = + "\x7e\xee\xab\x7c\x4e\x50\xfb\x79\x9b\x41\x8e\xe5\xe3\x19\x7f\xf6\xbf\x15\xd4\x3a\x14\xc3\x43\x89\xb5\x9d\xd1\xa7\xb1\xb8\x5b\x4a\xe9\x04\x38\xac\xa6\x34\xbe\xa4\x5e\x3a\x26\x95\xf1\x27\x0f\x07\xfd\xcd\xf7\xc6\x2b\x8e\xfe\xaf\x00\xb4\x5c\x2c\x96\xba\x45\x7e\xb1\xa8\xbf\x07\x5a\x3d\xb2\x8e\x5c\x24\xf6\xb9\x23\xed\x4a\xd7\x47\xc3\xc9\xe0\x3c\x70\x79\xef\xb8\x7c\xb1\x10\xd3\xa9\x98\x61\xe7\x20\x03\xcb\xae\x6d\x6b\x8b\x82\x7e\x4e\x6c\x14\x30\x64\xff\x3c\x00" + } + , Vec + { vecSec = + "\x7e\xf4\xe8\x45\x44\x23\x67\x52\xfb\xb5\x6b\x8f\x31\xa2\x3a\x10\xe4\x28\x14\xf5\xf5\x5c\xa0\x37\xcd\xcc\x11\xc6\x4c\x9a\x3b\x29\x49\xc1\xbb\x60\x70\x03\x14\x61\x17\x32\xa6\xc2\xfe\xa9\x8e\xeb\xc0\x26\x6a\x11\xa9\x39\x70\x10\x0e" + , vecPub = + "\xb3\xda\x07\x9b\x0a\xa4\x93\xa5\x77\x20\x29\xf0\x46\x7b\xae\xbe\xe5\xa8\x11\x2d\x9d\x3a\x22\x53\x23\x61\xda\x29\x4f\x7b\xb3\x81\x5c\x5d\xc5\x9e\x17\x6b\x4d\x9f\x38\x1c\xa0\x93\x8e\x13\xc6\xc0\x7b\x17\x4b\xe6\x5d\xfa\x57\x8e\x80" + , vecMsg = "\x64\xa6\x5f\x3c\xde\xdc\xdd\x66\x81\x1e\x29\x15\xe7" + , vecSig = + "\x6a\x12\x06\x6f\x55\x33\x1b\x6c\x22\xac\xd5\xd5\xbf\xc5\xd7\x12\x28\xfb\xda\x80\xae\x8d\xec\x26\xbd\xd3\x06\x74\x3c\x50\x27\xcb\x48\x90\x81\x0c\x16\x2c\x02\x74\x68\x67\x5e\xcf\x64\x5a\x83\x17\x6c\x0d\x73\x23\xa2\xcc\xde\x2d\x80\xef\xe5\xa1\x26\x8e\x8a\xca\x1d\x6f\xbc\x19\x4d\x3f\x77\xc4\x49\x86\xeb\x4a\xb4\x17\x79\x19\xad\x8b\xec\x33\xeb\x47\xbb\xb5\xfc\x6e\x28\x19\x6f\xd1\xca\xf5\x6b\x4e\x7e\x0b\xa5\x51\x92\x34\xd0\x47\x15\x5a\xc7\x27\xa1\x05\x31\x00" + } + , Vec + { vecSec = + "\xd6\x5d\xf3\x41\xad\x13\xe0\x08\x56\x76\x88\xba\xed\xda\x8e\x9d\xcd\xc1\x7d\xc0\x24\x97\x4e\xa5\xb4\x22\x7b\x65\x30\xe3\x39\xbf\xf2\x1f\x99\xe6\x8c\xa6\x96\x8f\x3c\xca\x6d\xfe\x0f\xb9\xf4\xfa\xb4\xfa\x13\x5d\x55\x42\xea\x3f\x01" + , vecPub = + "\xdf\x97\x05\xf5\x8e\xdb\xab\x80\x2c\x7f\x83\x63\xcf\xe5\x56\x0a\xb1\xc6\x13\x2c\x20\xa9\xf1\xdd\x16\x34\x83\xa2\x6f\x8a\xc5\x3a\x39\xd6\x80\x8b\xf4\xa1\xdf\xbd\x26\x1b\x09\x9b\xb0\x3b\x3f\xb5\x09\x06\xcb\x28\xbd\x8a\x08\x1f\x00" + , vecMsg = + "\xbd\x0f\x6a\x37\x47\xcd\x56\x1b\xdd\xdf\x46\x40\xa3\x32\x46\x1a\x4a\x30\xa1\x2a\x43\x4c\xd0\xbf\x40\xd7\x66\xd9\xc6\xd4\x58\xe5\x51\x22\x04\xa3\x0c\x17\xd1\xf5\x0b\x50\x79\x63\x1f\x64\xeb\x31\x12\x18\x2d\xa3\x00\x58\x35\x46\x11\x13\x71\x8d\x1a\x5e\xf9\x44" + , vecSig = + "\x55\x4b\xc2\x48\x08\x60\xb4\x9e\xab\x85\x32\xd2\xa5\x33\xb7\xd5\x78\xef\x47\x3e\xeb\x58\xc9\x8b\xb2\xd0\xe1\xce\x48\x8a\x98\xb1\x8d\xfd\xe9\xb9\xb9\x07\x75\xe6\x7f\x47\xd4\xa1\xc3\x48\x20\x58\xef\xc9\xf4\x0d\x2c\xa0\x33\xa0\x80\x1b\x63\xd4\x5b\x3b\x72\x2e\xf5\x52\xba\xd3\xb4\xcc\xb6\x67\xda\x35\x01\x92\xb6\x1c\x50\x8c\xf7\xb6\xb5\xad\xad\xc2\xc8\xd9\xa4\x46\xef\x00\x3f\xb0\x5c\xba\x5f\x30\xe8\x8e\x36\xec\x27\x03\xb3\x49\xca\x22\x9c\x26\x70\x83\x39\x00" + } + , Vec + { vecSec = + "\x2e\xc5\xfe\x3c\x17\x04\x5a\xbd\xb1\x36\xa5\xe6\xa9\x13\xe3\x2a\xb7\x5a\xe6\x8b\x53\xd2\xfc\x14\x9b\x77\xe5\x04\x13\x2d\x37\x56\x9b\x7e\x76\x6b\xa7\x4a\x19\xbd\x61\x62\x34\x3a\x21\xc8\x59\x0a\xa9\xce\xbc\xa9\x01\x4c\x63\x6d\xf5" + , vecPub = + "\x79\x75\x6f\x01\x4d\xcf\xe2\x07\x9f\x5d\xd9\xe7\x18\xbe\x41\x71\xe2\xef\x24\x86\xa0\x8f\x25\x18\x6f\x6b\xff\x43\xa9\x93\x6b\x9b\xfe\x12\x40\x2b\x08\xae\x65\x79\x8a\x3d\x81\xe2\x2e\x9e\xc8\x0e\x76\x90\x86\x2e\xf3\xd4\xed\x3a\x00" + , vecMsg = + "\x15\x77\x75\x32\xb0\xbd\xd0\xd1\x38\x9f\x63\x6c\x5f\x6b\x9b\xa7\x34\xc9\x0a\xf5\x72\x87\x7e\x2d\x27\x2d\xd0\x78\xaa\x1e\x56\x7c\xfa\x80\xe1\x29\x28\xbb\x54\x23\x30\xe8\x40\x9f\x31\x74\x50\x41\x07\xec\xd5\xef\xac\x61\xae\x75\x04\xda\xbe\x2a\x60\x2e\xde\x89\xe5\xcc\xa6\x25\x7a\x7c\x77\xe2\x7a\x70\x2b\x3a\xe3\x9f\xc7\x69\xfc\x54\xf2\x39\x5a\xe6\xa1\x17\x8c\xab\x47\x38\xe5\x43\x07\x2f\xc1\xc1\x77\xfe\x71\xe9\x2e\x25\xbf\x03\xe4\xec\xb7\x2f\x47\xb6\x4d\x04\x65\xaa\xea\x4c\x7f\xad\x37\x25\x36\xc8\xba\x51\x6a\x60\x39\xc3\xc2\xa3\x9f\x0e\x4d\x83\x2b\xe4\x32\xdf\xa9\xa7\x06\xa6\xe5\xc7\xe1\x9f\x39\x79\x64\xca\x42\x58\x00\x2f\x7c\x05\x41\xb5\x90\x31\x6d\xbc\x56\x22\xb6\xb2\xa6\xfe\x7a\x4a\xbf\xfd\x96\x10\x5e\xca\x76\xea\x7b\x98\x81\x6a\xf0\x74\x8c\x10\xdf\x04\x8c\xe0\x12\xd9\x01\x01\x5a\x51\xf1\x89\xf3\x88\x81\x45\xc0\x36\x50\xaa\x23\xce\x89\x4c\x3b\xd8\x89\xe0\x30\xd5\x65\x07\x1c\x59\xf4\x09\xa9\x98\x1b\x51\x87\x8f\xd6\xfc\x11\x06\x24\xdc\xbc\xde\x0b\xf7\xa6\x9c\xcc\xe3\x8f\xab\xdf\x86\xf3\xbe\xf6\x04\x48\x19\xde\x11" + , vecSig = + "\xc6\x50\xdd\xbb\x06\x01\xc1\x9c\xa1\x14\x39\xe1\x64\x0d\xd9\x31\xf4\x3c\x51\x8e\xa5\xbe\xa7\x0d\x3d\xcd\xe5\xf4\x19\x1f\xe5\x3f\x00\xcf\x96\x65\x46\xb7\x2b\xcc\x7d\x58\xbe\x2b\x9b\xad\xef\x28\x74\x39\x54\xe3\xa4\x4a\x23\xf8\x80\xe8\xd4\xf1\xcf\xce\x2d\x7a\x61\x45\x2d\x26\xda\x05\x89\x6f\x0a\x50\xda\x66\xa2\x39\xa8\xa1\x88\xb6\xd8\x25\xb3\x30\x5a\xd7\x7b\x73\xfb\xac\x08\x36\xec\xc6\x09\x87\xfd\x08\x52\x7c\x1a\x8e\x80\xd5\x82\x3e\x65\xca\xfe\x2a\x3d\x00" + } + , Vec + { vecSec = + "\x87\x2d\x09\x37\x80\xf5\xd3\x73\x0d\xf7\xc2\x12\x66\x4b\x37\xb8\xa0\xf2\x4f\x56\x81\x0d\xaa\x83\x82\xcd\x4f\xa3\xf7\x76\x34\xec\x44\xdc\x54\xf1\xc2\xed\x9b\xea\x86\xfa\xfb\x76\x32\xd8\xbe\x19\x9e\xa1\x65\xf5\xad\x55\xdd\x9c\xe8" + , vecPub = + "\xa8\x1b\x2e\x8a\x70\xa5\xac\x94\xff\xdb\xcc\x9b\xad\xfc\x3f\xeb\x08\x01\xf2\x58\x57\x8b\xb1\x14\xad\x44\xec\xe1\xec\x0e\x79\x9d\xa0\x8e\xff\xb8\x1c\x5d\x68\x5c\x0c\x56\xf6\x4e\xec\xae\xf8\xcd\xf1\x1c\xc3\x87\x37\x83\x8c\xf4\x00" + , vecMsg = + "\x6d\xdf\x80\x2e\x1a\xae\x49\x86\x93\x5f\x7f\x98\x1b\xa3\xf0\x35\x1d\x62\x73\xc0\xa0\xc2\x2c\x9c\x0e\x83\x39\x16\x8e\x67\x54\x12\xa3\xde\xbf\xaf\x43\x5e\xd6\x51\x55\x80\x07\xdb\x43\x84\xb6\x50\xfc\xc0\x7e\x3b\x58\x6a\x27\xa4\xf7\xa0\x0a\xc8\xa6\xfe\xc2\xcd\x86\xae\x4b\xf1\x57\x0c\x41\xe6\xa4\x0c\x93\x1d\xb2\x7b\x2f\xaa\x15\xa8\xce\xdd\x52\xcf\xf7\x36\x2c\x4e\x6e\x23\xda\xec\x0f\xbc\x3a\x79\xb6\x80\x6e\x31\x6e\xfc\xc7\xb6\x81\x19\xbf\x46\xbc\x76\xa2\x60\x67\xa5\x3f\x29\x6d\xaf\xdb\xdc\x11\xc7\x7f\x77\x77\xe9\x72\x66\x0c\xf4\xb6\xa9\xb3\x69\xa6\x66\x5f\x02\xe0\xcc\x9b\x6e\xdf\xad\x13\x6b\x4f\xab\xe7\x23\xd2\x81\x3d\xb3\x13\x6c\xfd\xe9\xb6\xd0\x44\x32\x2f\xee\x29\x47\x95\x2e\x03\x1b\x73\xab\x5c\x60\x33\x49\xb3\x07\xbd\xc2\x7b\xc6\xcb\x8b\x8b\xbd\x7b\xd3\x23\x21\x9b\x80\x33\xa5\x81\xb5\x9e\xad\xeb\xb0\x9b\x3c\x4f\x3d\x22\x77\xd4\xf0\x34\x36\x24\xac\xc8\x17\x80\x47\x28\xb2\x5a\xb7\x97\x17\x2b\x4c\x5c\x21\xa2\x2f\x9c\x78\x39\xd6\x43\x00\x23\x2e\xb6\x6e\x53\xf3\x1c\x72\x3f\xa3\x7f\xe3\x87\xc7\xd3\xe5\x0b\xdf\x98\x13\xa3\x0e\x5b\xb1\x2c\xf4\xcd\x93\x0c\x40\xcf\xb4\xe1\xfc\x62\x25\x92\xa4\x95\x88\x79\x44\x94\xd5\x6d\x24\xea\x4b\x40\xc8\x9f\xc0\x59\x6c\xc9\xeb\xb9\x61\xc8\xcb\x10\xad\xde\x97\x6a\x5d\x60\x2b\x1c\x3f\x85\xb9\xb9\xa0\x01\xed\x3c\x6a\x4d\x3b\x14\x37\xf5\x20\x96\xcd\x19\x56\xd0\x42\xa5\x97\xd5\x61\xa5\x96\xec\xd3\xd1\x73\x5a\x8d\x57\x0e\xa0\xec\x27\x22\x5a\x2c\x4a\xaf\xf2\x63\x06\xd1\x52\x6c\x1a\xf3\xca\x6d\x9c\xf5\xa2\xc9\x8f\x47\xe1\xc4\x6d\xb9\xa3\x32\x34\xcf\xd4\xd8\x1f\x2c\x98\x53\x8a\x09\xeb\xe7\x69\x98\xd0\xd8\xfd\x25\x99\x7c\x7d\x25\x5c\x6d\x66\xec\xe6\xfa\x56\xf1\x11\x44\x95\x0f\x02\x77\x95\xe6\x53\x00\x8f\x4b\xd7\xca\x2d\xee\x85\xd8\xe9\x0f\x3d\xc3\x15\x13\x0c\xe2\xa0\x03\x75\xa3\x18\xc7\xc3\xd9\x7b\xe2\xc8\xce\x5b\x6d\xb4\x1a\x62\x54\xff\x26\x4f\xa6\x15\x5b\xae\xe3\xb0\x77\x3c\x0f\x49\x7c\x57\x3f\x19\xbb\x4f\x42\x40\x28\x1f\x0b\x1f\x4f\x7b\xe8\x57\xa4\xe5\x9d\x41\x6c\x06\xb4\xc5\x0f\xa0\x9e\x18\x10\xdd\xc6\xb1\x46\x7b\xae\xac\x5a\x36\x68\xd1\x1b\x6e\xca\xa9\x01\x44\x00\x16\xf3\x89\xf8\x0a\xcc\x4d\xb9\x77\x02\x5e\x7f\x59\x24\x38\x8c\x7e\x34\x0a\x73\x2e\x55\x44\x40\xe7\x65\x70\xf8\xdd\x71\xb7\xd6\x40\xb3\x45\x0d\x1f\xd5\xf0\x41\x0a\x18\xf9\xa3\x49\x4f\x70\x7c\x71\x7b\x79\xb4\xbf\x75\xc9\x84\x00\xb0\x96\xb2\x16\x53\xb5\xd2\x17\xcf\x35\x65\xc9\x59\x74\x56\xf7\x07\x03\x49\x7a\x07\x87\x63\x82\x9b\xc0\x1b\xb1\xcb\xc8\xfa\x04\xea\xdc\x9a\x6e\x3f\x66\x99\x58\x7a\x9e\x75\xc9\x4e\x5b\xab\x00\x36\xe0\xb2\xe7\x11\x39\x2c\xff\x00\x47\xd0\xd6\xb0\x5b\xd2\xa5\x88\xbc\x10\x97\x18\x95\x42\x59\xf1\xd8\x66\x78\xa5\x79\xa3\x12\x0f\x19\xcf\xb2\x96\x3f\x17\x7a\xeb\x70\xf2\xd4\x84\x48\x26\x26\x2e\x51\xb8\x02\x71\x27\x20\x68\xef\x5b\x38\x56\xfa\x85\x35\xaa\x2a\x88\xb2\xd4\x1f\x2a\x0e\x2f\xda\x76\x24\xc2\x85\x02\x72\xac\x4a\x2f\x56\x1f\x8f\x2f\x7a\x31\x8b\xfd\x5c\xaf\x96\x96\x14\x9e\x4a\xc8\x24\xad\x34\x60\x53\x8f\xdc\x25\x42\x1b\xee\xc2\xcc\x68\x18\x16\x2d\x06\xbb\xed\x0c\x40\xa3\x87\x19\x23\x49\xdb\x67\xa1\x18\xba\xda\x6c\xd5\xab\x01\x40\xee\x27\x32\x04\xf6\x28\xaa\xd1\xc1\x35\xf7\x70\x27\x9a\x65\x1e\x24\xd8\xc1\x4d\x75\xa6\x05\x9d\x76\xb9\x6a\x6f\xd8\x57\xde\xf5\xe0\xb3\x54\xb2\x7a\xb9\x37\xa5\x81\x5d\x16\xb5\xfa\xe4\x07\xff\x18\x22\x2c\x6d\x1e\xd2\x63\xbe\x68\xc9\x5f\x32\xd9\x08\xbd\x89\x5c\xd7\x62\x07\xae\x72\x64\x87\x56\x7f\x9a\x67\xda\xd7\x9a\xbe\xc3\x16\xf6\x83\xb1\x7f\x2d\x02\xbf\x07\xe0\xac\x8b\x5b\xc6\x16\x2c\xf9\x46\x97\xb3\xc2\x7c\xd1\xfe\xa4\x9b\x27\xf2\x3b\xa2\x90\x18\x71\x96\x25\x06\x52\x0c\x39\x2d\xa8\xb6\xad\x0d\x99\xf7\x01\x3f\xbc\x06\xc2\xc1\x7a\x56\x95\x00\xc8\xa7\x69\x64\x81\xc1\xcd\x33\xe9\xb1\x4e\x40\xb8\x2e\x79\xa5\xf5\xdb\x82\x57\x1b\xa9\x7b\xae\x3a\xd3\xe0\x47\x95\x15\xbb\x0e\x2b\x0f\x3b\xfc\xd1\xfd\x33\x03\x4e\xfc\x62\x45\xed\xdd\x7e\xe2\x08\x6d\xda\xe2\x60\x0d\x8c\xa7\x3e\x21\x4e\x8c\x2b\x0b\xdb\x2b\x04\x7c\x6a\x46\x4a\x56\x2e\xd7\x7b\x73\xd2\xd8\x41\xc4\xb3\x49\x73\x55\x12\x57\x71\x3b\x75\x36\x32\xef\xba\x34\x81\x69\xab\xc9\x0a\x68\xf4\x26\x11\xa4\x01\x26\xd7\xcb\x21\xb5\x86\x95\x56\x81\x86\xf7\xe5\x69\xd2\xff\x0f\x9e\x74\x5d\x04\x87\xdd\x2e\xb9\x97\xca\xfc\x5a\xbf\x9d\xd1\x02\xe6\x2f\xf6\x6c\xba\x87" + , vecSig = + "\xe3\x01\x34\x5a\x41\xa3\x9a\x4d\x72\xff\xf8\xdf\x69\xc9\x80\x75\xa0\xcc\x08\x2b\x80\x2f\xc9\xb2\xb6\xbc\x50\x3f\x92\x6b\x65\xbd\xdf\x7f\x4c\x8f\x1c\xb4\x9f\x63\x96\xaf\xc8\xa7\x0a\xbe\x6d\x8a\xef\x0d\xb4\x78\xd4\xc6\xb2\x97\x00\x76\xc6\xa0\x48\x4f\xe7\x6d\x76\xb3\xa9\x76\x25\xd7\x9f\x1c\xe2\x40\xe7\xc5\x76\x75\x0d\x29\x55\x28\x28\x6f\x71\x9b\x41\x3d\xe9\xad\xa3\xe8\xeb\x78\xed\x57\x36\x03\xce\x30\xd8\xbb\x76\x17\x85\xdc\x30\xdb\xc3\x20\x86\x9e\x1a\x00" + } + ] + +doPublicKeyTest i vec = it (show i) (Ed448.toPublic sec `shouldBe` pub) + where + !pub = throwCryptoError $ Ed448.publicKey (vecPub vec) + !sec = throwCryptoError $ Ed448.secretKey (vecSec vec) + +doSignatureTest i vec = it (show i) (Ed448.sign sec pub (vecMsg vec) `shouldBe` sig) + where + !sig = throwCryptoError $ Ed448.signature (vecSig vec) + !pub = throwCryptoError $ Ed448.publicKey (vecPub vec) + !sec = throwCryptoError $ Ed448.secretKey (vecSec vec) + +doVerifyTest i vec = it (show i) (Ed448.verify pub (vecMsg vec) sig `shouldBe` True) + where + !sig = throwCryptoError $ Ed448.signature (vecSig vec) + !pub = throwCryptoError $ Ed448.publicKey (vecPub vec) + +spec :: Spec +spec = do + it "gen secretkey" (Ed448.generateSecretKey *> pure () :: Expectation) + describe "gen publickey" $ zipWithM_ doPublicKeyTest [katZero ..] vectors + describe "gen signature" $ zipWithM_ doSignatureTest [katZero ..] vectors + describe "verify sig" $ zipWithM_ doVerifyTest [katZero ..] vectors diff --git a/tests/EdDSASpec.hs b/tests/EdDSASpec.hs new file mode 100644 --- /dev/null +++ b/tests/EdDSASpec.hs @@ -0,0 +1,244 @@ +{-# LANGUAGE BangPatterns #-} +{-# LANGUAGE ExistentialQuantification #-} +{-# LANGUAGE GADTs #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE RecordWildCards #-} +{-# LANGUAGE TypeOperators #-} + +module EdDSASpec (spec) where + +import Crypto.ECC +import Crypto.Error +import Crypto.Hash.Algorithms +import Crypto.Hash.IO +import qualified Crypto.PubKey.EdDSA as EdDSA +import Data.ByteArray.Encoding (Base (Base16), convertFromBase) +import Imports + +data Vec + = forall curve hash. + ( EdDSA.EllipticCurveEdDSA curve + , HashAlgorithm hash + , HashDigestSize hash ~ EdDSA.CurveDigestSize curve + ) => + Vec + { vecPrx :: Maybe curve + , vecAlg :: hash + , vecSec :: ByteString + , vecPub :: ByteString + , vecMsg :: ByteString + , vecSig :: ByteString + } + +vectors = + [ Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = SHA512 + , vecSec = + "\x9d\x61\xb1\x9d\xef\xfd\x5a\x60\xba\x84\x4a\xf4\x92\xec\x2c\xc4\x44\x49\xc5\x69\x7b\x32\x69\x19\x70\x3b\xac\x03\x1c\xae\x7f\x60" + , vecPub = + "\xd7\x5a\x98\x01\x82\xb1\x0a\xb7\xd5\x4b\xfe\xd3\xc9\x64\x07\x3a\x0e\xe1\x72\xf3\xda\xa6\x23\x25\xaf\x02\x1a\x68\xf7\x07\x51\x1a" + , vecMsg = "" + , vecSig = + "\xe5\x56\x43\x00\xc3\x60\xac\x72\x90\x86\xe2\xcc\x80\x6e\x82\x8a\x84\x87\x7f\x1e\xb8\xe5\xd9\x74\xd8\x73\xe0\x65\x22\x49\x01\x55\x5f\xb8\x82\x15\x90\xa3\x3b\xac\xc6\x1e\x39\x70\x1c\xf9\xb4\x6b\xd2\x5b\xf5\xf0\x59\x5b\xbe\x24\x65\x51\x41\x43\x8e\x7a\x10\x0b" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = SHA512 + , vecSec = + "\x4c\xcd\x08\x9b\x28\xff\x96\xda\x9d\xb6\xc3\x46\xec\x11\x4e\x0f\x5b\x8a\x31\x9f\x35\xab\xa6\x24\xda\x8c\xf6\xed\x4f\xb8\xa6\xfb" + , vecPub = + "\x3d\x40\x17\xc3\xe8\x43\x89\x5a\x92\xb7\x0a\xa7\x4d\x1b\x7e\xbc\x9c\x98\x2c\xcf\x2e\xc4\x96\x8c\xc0\xcd\x55\xf1\x2a\xf4\x66\x0c" + , vecMsg = "\x72" + , vecSig = + "\x92\xa0\x09\xa9\xf0\xd4\xca\xb8\x72\x0e\x82\x0b\x5f\x64\x25\x40\xa2\xb2\x7b\x54\x16\x50\x3f\x8f\xb3\x76\x22\x23\xeb\xdb\x69\xda\x08\x5a\xc1\xe4\x3e\x15\x99\x6e\x45\x8f\x36\x13\xd0\xf1\x1d\x8c\x38\x7b\x2e\xae\xb4\x30\x2a\xee\xb0\x0d\x29\x16\x12\xbb\x0c\x00" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = SHA512 + , vecSec = + "\xc5\xaa\x8d\xf4\x3f\x9f\x83\x7b\xed\xb7\x44\x2f\x31\xdc\xb7\xb1\x66\xd3\x85\x35\x07\x6f\x09\x4b\x85\xce\x3a\x2e\x0b\x44\x58\xf7" + , vecPub = + "\xfc\x51\xcd\x8e\x62\x18\xa1\xa3\x8d\xa4\x7e\xd0\x02\x30\xf0\x58\x08\x16\xed\x13\xba\x33\x03\xac\x5d\xeb\x91\x15\x48\x90\x80\x25" + , vecMsg = "\xaf\x82" + , vecSig = + "\x62\x91\xd6\x57\xde\xec\x24\x02\x48\x27\xe6\x9c\x3a\xbe\x01\xa3\x0c\xe5\x48\xa2\x84\x74\x3a\x44\x5e\x36\x80\xd7\xdb\x5a\xc3\xac\x18\xff\x9b\x53\x8d\x16\xf2\x90\xae\x67\xf7\x60\x98\x4d\xc6\x59\x4a\x7c\x15\xe9\x71\x6e\xd2\x8d\xc0\x27\xbe\xce\xea\x1e\xc4\x0a" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = SHA512 + , vecSec = + "\xf5\xe5\x76\x7c\xf1\x53\x31\x95\x17\x63\x0f\x22\x68\x76\xb8\x6c\x81\x60\xcc\x58\x3b\xc0\x13\x74\x4c\x6b\xf2\x55\xf5\xcc\x0e\xe5" + , vecPub = + "\x27\x81\x17\xfc\x14\x4c\x72\x34\x0f\x67\xd0\xf2\x31\x6e\x83\x86\xce\xff\xbf\x2b\x24\x28\xc9\xc5\x1f\xef\x7c\x59\x7f\x1d\x42\x6e" + , vecMsg = + "\x08\xb8\xb2\xb7\x33\x42\x42\x43\x76\x0f\xe4\x26\xa4\xb5\x49\x08\x63\x21\x10\xa6\x6c\x2f\x65\x91\xea\xbd\x33\x45\xe3\xe4\xeb\x98\xfa\x6e\x26\x4b\xf0\x9e\xfe\x12\xee\x50\xf8\xf5\x4e\x9f\x77\xb1\xe3\x55\xf6\xc5\x05\x44\xe2\x3f\xb1\x43\x3d\xdf\x73\xbe\x84\xd8\x79\xde\x7c\x00\x46\xdc\x49\x96\xd9\xe7\x73\xf4\xbc\x9e\xfe\x57\x38\x82\x9a\xdb\x26\xc8\x1b\x37\xc9\x3a\x1b\x27\x0b\x20\x32\x9d\x65\x86\x75\xfc\x6e\xa5\x34\xe0\x81\x0a\x44\x32\x82\x6b\xf5\x8c\x94\x1e\xfb\x65\xd5\x7a\x33\x8b\xbd\x2e\x26\x64\x0f\x89\xff\xbc\x1a\x85\x8e\xfc\xb8\x55\x0e\xe3\xa5\xe1\x99\x8b\xd1\x77\xe9\x3a\x73\x63\xc3\x44\xfe\x6b\x19\x9e\xe5\xd0\x2e\x82\xd5\x22\xc4\xfe\xba\x15\x45\x2f\x80\x28\x8a\x82\x1a\x57\x91\x16\xec\x6d\xad\x2b\x3b\x31\x0d\xa9\x03\x40\x1a\xa6\x21\x00\xab\x5d\x1a\x36\x55\x3e\x06\x20\x3b\x33\x89\x0c\xc9\xb8\x32\xf7\x9e\xf8\x05\x60\xcc\xb9\xa3\x9c\xe7\x67\x96\x7e\xd6\x28\xc6\xad\x57\x3c\xb1\x16\xdb\xef\xef\xd7\x54\x99\xda\x96\xbd\x68\xa8\xa9\x7b\x92\x8a\x8b\xbc\x10\x3b\x66\x21\xfc\xde\x2b\xec\xa1\x23\x1d\x20\x6b\xe6\xcd\x9e\xc7\xaf\xf6\xf6\xc9\x4f\xcd\x72\x04\xed\x34\x55\xc6\x8c\x83\xf4\xa4\x1d\xa4\xaf\x2b\x74\xef\x5c\x53\xf1\xd8\xac\x70\xbd\xcb\x7e\xd1\x85\xce\x81\xbd\x84\x35\x9d\x44\x25\x4d\x95\x62\x9e\x98\x55\xa9\x4a\x7c\x19\x58\xd1\xf8\xad\xa5\xd0\x53\x2e\xd8\xa5\xaa\x3f\xb2\xd1\x7b\xa7\x0e\xb6\x24\x8e\x59\x4e\x1a\x22\x97\xac\xbb\xb3\x9d\x50\x2f\x1a\x8c\x6e\xb6\xf1\xce\x22\xb3\xde\x1a\x1f\x40\xcc\x24\x55\x41\x19\xa8\x31\xa9\xaa\xd6\x07\x9c\xad\x88\x42\x5d\xe6\xbd\xe1\xa9\x18\x7e\xbb\x60\x92\xcf\x67\xbf\x2b\x13\xfd\x65\xf2\x70\x88\xd7\x8b\x7e\x88\x3c\x87\x59\xd2\xc4\xf5\xc6\x5a\xdb\x75\x53\x87\x8a\xd5\x75\xf9\xfa\xd8\x78\xe8\x0a\x0c\x9b\xa6\x3b\xcb\xcc\x27\x32\xe6\x94\x85\xbb\xc9\xc9\x0b\xfb\xd6\x24\x81\xd9\x08\x9b\xec\xcf\x80\xcf\xe2\xdf\x16\xa2\xcf\x65\xbd\x92\xdd\x59\x7b\x07\x07\xe0\x91\x7a\xf4\x8b\xbb\x75\xfe\xd4\x13\xd2\x38\xf5\x55\x5a\x7a\x56\x9d\x80\xc3\x41\x4a\x8d\x08\x59\xdc\x65\xa4\x61\x28\xba\xb2\x7a\xf8\x7a\x71\x31\x4f\x31\x8c\x78\x2b\x23\xeb\xfe\x80\x8b\x82\xb0\xce\x26\x40\x1d\x2e\x22\xf0\x4d\x83\xd1\x25\x5d\xc5\x1a\xdd\xd3\xb7\x5a\x2b\x1a\xe0\x78\x45\x04\xdf\x54\x3a\xf8\x96\x9b\xe3\xea\x70\x82\xff\x7f\xc9\x88\x8c\x14\x4d\xa2\xaf\x58\x42\x9e\xc9\x60\x31\xdb\xca\xd3\xda\xd9\xaf\x0d\xcb\xaa\xaf\x26\x8c\xb8\xfc\xff\xea\xd9\x4f\x3c\x7c\xa4\x95\xe0\x56\xa9\xb4\x7a\xcd\xb7\x51\xfb\x73\xe6\x66\xc6\xc6\x55\xad\xe8\x29\x72\x97\xd0\x7a\xd1\xba\x5e\x43\xf1\xbc\xa3\x23\x01\x65\x13\x39\xe2\x29\x04\xcc\x8c\x42\xf5\x8c\x30\xc0\x4a\xaf\xdb\x03\x8d\xda\x08\x47\xdd\x98\x8d\xcd\xa6\xf3\xbf\xd1\x5c\x4b\x4c\x45\x25\x00\x4a\xa0\x6e\xef\xf8\xca\x61\x78\x3a\xac\xec\x57\xfb\x3d\x1f\x92\xb0\xfe\x2f\xd1\xa8\x5f\x67\x24\x51\x7b\x65\xe6\x14\xad\x68\x08\xd6\xf6\xee\x34\xdf\xf7\x31\x0f\xdc\x82\xae\xbf\xd9\x04\xb0\x1e\x1d\xc5\x4b\x29\x27\x09\x4b\x2d\xb6\x8d\x6f\x90\x3b\x68\x40\x1a\xde\xbf\x5a\x7e\x08\xd7\x8f\xf4\xef\x5d\x63\x65\x3a\x65\x04\x0c\xf9\xbf\xd4\xac\xa7\x98\x4a\x74\xd3\x71\x45\x98\x67\x80\xfc\x0b\x16\xac\x45\x16\x49\xde\x61\x88\xa7\xdb\xdf\x19\x1f\x64\xb5\xfc\x5e\x2a\xb4\x7b\x57\xf7\xf7\x27\x6c\xd4\x19\xc1\x7a\x3c\xa8\xe1\xb9\x39\xae\x49\xe4\x88\xac\xba\x6b\x96\x56\x10\xb5\x48\x01\x09\xc8\xb1\x7b\x80\xe1\xb7\xb7\x50\xdf\xc7\x59\x8d\x5d\x50\x11\xfd\x2d\xcc\x56\x00\xa3\x2e\xf5\xb5\x2a\x1e\xcc\x82\x0e\x30\x8a\xa3\x42\x72\x1a\xac\x09\x43\xbf\x66\x86\xb6\x4b\x25\x79\x37\x65\x04\xcc\xc4\x93\xd9\x7e\x6a\xed\x3f\xb0\xf9\xcd\x71\xa4\x3d\xd4\x97\xf0\x1f\x17\xc0\xe2\xcb\x37\x97\xaa\x2a\x2f\x25\x66\x56\x16\x8e\x6c\x49\x6a\xfc\x5f\xb9\x32\x46\xf6\xb1\x11\x63\x98\xa3\x46\xf1\xa6\x41\xf3\xb0\x41\xe9\x89\xf7\x91\x4f\x90\xcc\x2c\x7f\xff\x35\x78\x76\xe5\x06\xb5\x0d\x33\x4b\xa7\x7c\x22\x5b\xc3\x07\xba\x53\x71\x52\xf3\xf1\x61\x0e\x4e\xaf\xe5\x95\xf6\xd9\xd9\x0d\x11\xfa\xa9\x33\xa1\x5e\xf1\x36\x95\x46\x86\x8a\x7f\x3a\x45\xa9\x67\x68\xd4\x0f\xd9\xd0\x34\x12\xc0\x91\xc6\x31\x5c\xf4\xfd\xe7\xcb\x68\x60\x69\x37\x38\x0d\xb2\xea\xaa\x70\x7b\x4c\x41\x85\xc3\x2e\xdd\xcd\xd3\x06\x70\x5e\x4d\xc1\xff\xc8\x72\xee\xee\x47\x5a\x64\xdf\xac\x86\xab\xa4\x1c\x06\x18\x98\x3f\x87\x41\xc5\xef\x68\xd3\xa1\x01\xe8\xa3\xb8\xca\xc6\x0c\x90\x5c\x15\xfc\x91\x08\x40\xb9\x4c\x00\xa0\xb9\xd0" + , vecSig = + "\x0a\xab\x4c\x90\x05\x01\xb3\xe2\x4d\x7c\xdf\x46\x63\x32\x6a\x3a\x87\xdf\x5e\x48\x43\xb2\xcb\xdb\x67\xcb\xf6\xe4\x60\xfe\xc3\x50\xaa\x53\x71\xb1\x50\x8f\x9f\x45\x28\xec\xea\x23\xc4\x36\xd9\x4b\x5e\x8f\xcd\x4f\x68\x1e\x30\xa6\xac\x00\xa9\x70\x4a\x18\x8a\x03" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = SHA512 + , vecSec = + "\x83\x3f\xe6\x24\x09\x23\x7b\x9d\x62\xec\x77\x58\x75\x20\x91\x1e\x9a\x75\x9c\xec\x1d\x19\x75\x5b\x7d\xa9\x01\xb9\x6d\xca\x3d\x42" + , vecPub = + "\xec\x17\x2b\x93\xad\x5e\x56\x3b\xf4\x93\x2c\x70\xe1\x24\x50\x34\xc3\x54\x67\xef\x2e\xfd\x4d\x64\xeb\xf8\x19\x68\x34\x67\xe2\xbf" + , vecMsg = + "\xdd\xaf\x35\xa1\x93\x61\x7a\xba\xcc\x41\x73\x49\xae\x20\x41\x31\x12\xe6\xfa\x4e\x89\xa9\x7e\xa2\x0a\x9e\xee\xe6\x4b\x55\xd3\x9a\x21\x92\x99\x2a\x27\x4f\xc1\xa8\x36\xba\x3c\x23\xa3\xfe\xeb\xbd\x45\x4d\x44\x23\x64\x3c\xe8\x0e\x2a\x9a\xc9\x4f\xa5\x4c\xa4\x9f" + , vecSig = + "\xdc\x2a\x44\x59\xe7\x36\x96\x33\xa5\x2b\x1b\xf2\x77\x83\x9a\x00\x20\x10\x09\xa3\xef\xbf\x3e\xcb\x69\xbe\xa2\x18\x6c\x26\xb5\x89\x09\x35\x1f\xc9\xac\x90\xb3\xec\xfd\xfb\xc7\xc6\x64\x31\xe0\x30\x3d\xca\x17\x9c\x13\x8a\xc1\x7a\xd9\xbe\xf1\x17\x73\x31\xa7\x04" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = Blake2b_512 + , vecSec = + "\x9d\x61\xb1\x9d\xef\xfd\x5a\x60\xba\x84\x4a\xf4\x92\xec\x2c\xc4\x44\x49\xc5\x69\x7b\x32\x69\x19\x70\x3b\xac\x03\x1c\xae\x7f\x60" + , vecPub = + "\x78\xe6\x5b\xf3\x0f\x89\x3d\x32\xfc\x57\xef\x05\x1c\x34\x1b\xde\xde\x24\x25\x44\xfc\x2a\x21\x12\xf0\xfa\x2c\x7a\xfd\xeb\xc0\x2f" + , vecMsg = "" + , vecSig = + "\x99\xa5\x23\xbd\x46\x16\xc8\x16\x11\x44\xd6\xa9\x9d\x3c\x32\x40\x0c\xb4\xa3\x26\xf4\xd7\x9e\x30\x73\x40\xf6\xaf\xa1\x17\x50\xa0\x08\x5d\x7d\x84\x62\x6b\xc9\xe4\xb1\x53\xfc\x0e\x39\x6d\x15\xce\x44\xc3\x9b\xae\x45\x33\x80\x4d\xb1\xfe\x5b\x52\xf2\xb1\xb8\x05" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = Blake2b_512 + , vecSec = + "\x4c\xcd\x08\x9b\x28\xff\x96\xda\x9d\xb6\xc3\x46\xec\x11\x4e\x0f\x5b\x8a\x31\x9f\x35\xab\xa6\x24\xda\x8c\xf6\xed\x4f\xb8\xa6\xfb" + , vecPub = + "\x5e\x71\x39\x2d\x91\xe6\xa5\x8f\xed\xeb\x08\x50\x36\x4f\x56\xcd\x15\x8a\x60\x44\x75\x57\xd7\x89\x03\x89\xc9\xb3\xd4\x57\x6d\x4d" + , vecMsg = "\x72" + , vecSig = + "\x6d\xa7\x5e\x15\xb5\x70\x7f\x4d\xe5\xa1\x53\xc4\x8a\x5d\x83\x9f\xb8\x50\x74\xc3\x8a\xeb\x62\x85\x97\x7f\x03\xa1\x39\x77\x59\x7f\x97\x60\x69\xfd\xb9\x03\xf1\x83\x47\x4a\xaa\x5e\xd0\xcf\xe8\x78\xba\x8e\xf8\x68\xc5\xe4\x7c\xa3\xf9\x6c\xcf\xb3\xa8\x9b\x2a\x06" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = Blake2b_512 + , vecSec = + "\xc5\xaa\x8d\xf4\x3f\x9f\x83\x7b\xed\xb7\x44\x2f\x31\xdc\xb7\xb1\x66\xd3\x85\x35\x07\x6f\x09\x4b\x85\xce\x3a\x2e\x0b\x44\x58\xf7" + , vecPub = + "\x8d\x53\xca\x70\xf0\xea\xb2\x3b\x91\x78\x34\x57\x85\xfc\xdb\x69\xed\x67\x23\xf8\x14\x8f\x7e\x33\x9e\x88\x65\x37\x00\xb7\x18\xda" + , vecMsg = "\xaf\x82" + , vecSig = + "\x7c\xc3\xc1\x38\x52\xbd\x12\xab\xf3\xce\x4c\xa8\xca\x28\x36\xcb\xf8\x6d\xa9\x6c\x46\x34\xc5\x0d\xf3\xfb\x80\xdc\x80\x9e\x29\xdb\x0e\x10\x9c\x36\x13\x53\x40\x7c\x12\x36\xa9\x04\xf6\x36\x86\x8a\xa3\x39\x77\xa9\x9d\x3f\x84\x45\x98\xdb\x15\x38\xb4\x29\x52\x03" + } + , Vec + { vecPrx = Just Curve_Edwards25519 + , vecAlg = Blake2b_512 + , vecSec = + "\xf5\xe5\x76\x7c\xf1\x53\x31\x95\x17\x63\x0f\x22\x68\x76\xb8\x6c\x81\x60\xcc\x58\x3b\xc0\x13\x74\x4c\x6b\xf2\x55\xf5\xcc\x0e\xe5" + , vecPub = + "\x9e\x3c\xa4\x9b\xb2\xd9\xe3\x6b\x8f\x0c\x94\x4a\x7b\x1c\x29\x26\x45\xda\x87\xce\x6f\xa6\xb4\x28\x86\xe5\xd7\xc8\x68\x33\xa7\x14" + , vecMsg = + "\x08\xb8\xb2\xb7\x33\x42\x42\x43\x76\x0f\xe4\x26\xa4\xb5\x49\x08\x63\x21\x10\xa6\x6c\x2f\x65\x91\xea\xbd\x33\x45\xe3\xe4\xeb\x98\xfa\x6e\x26\x4b\xf0\x9e\xfe\x12\xee\x50\xf8\xf5\x4e\x9f\x77\xb1\xe3\x55\xf6\xc5\x05\x44\xe2\x3f\xb1\x43\x3d\xdf\x73\xbe\x84\xd8\x79\xde\x7c\x00\x46\xdc\x49\x96\xd9\xe7\x73\xf4\xbc\x9e\xfe\x57\x38\x82\x9a\xdb\x26\xc8\x1b\x37\xc9\x3a\x1b\x27\x0b\x20\x32\x9d\x65\x86\x75\xfc\x6e\xa5\x34\xe0\x81\x0a\x44\x32\x82\x6b\xf5\x8c\x94\x1e\xfb\x65\xd5\x7a\x33\x8b\xbd\x2e\x26\x64\x0f\x89\xff\xbc\x1a\x85\x8e\xfc\xb8\x55\x0e\xe3\xa5\xe1\x99\x8b\xd1\x77\xe9\x3a\x73\x63\xc3\x44\xfe\x6b\x19\x9e\xe5\xd0\x2e\x82\xd5\x22\xc4\xfe\xba\x15\x45\x2f\x80\x28\x8a\x82\x1a\x57\x91\x16\xec\x6d\xad\x2b\x3b\x31\x0d\xa9\x03\x40\x1a\xa6\x21\x00\xab\x5d\x1a\x36\x55\x3e\x06\x20\x3b\x33\x89\x0c\xc9\xb8\x32\xf7\x9e\xf8\x05\x60\xcc\xb9\xa3\x9c\xe7\x67\x96\x7e\xd6\x28\xc6\xad\x57\x3c\xb1\x16\xdb\xef\xef\xd7\x54\x99\xda\x96\xbd\x68\xa8\xa9\x7b\x92\x8a\x8b\xbc\x10\x3b\x66\x21\xfc\xde\x2b\xec\xa1\x23\x1d\x20\x6b\xe6\xcd\x9e\xc7\xaf\xf6\xf6\xc9\x4f\xcd\x72\x04\xed\x34\x55\xc6\x8c\x83\xf4\xa4\x1d\xa4\xaf\x2b\x74\xef\x5c\x53\xf1\xd8\xac\x70\xbd\xcb\x7e\xd1\x85\xce\x81\xbd\x84\x35\x9d\x44\x25\x4d\x95\x62\x9e\x98\x55\xa9\x4a\x7c\x19\x58\xd1\xf8\xad\xa5\xd0\x53\x2e\xd8\xa5\xaa\x3f\xb2\xd1\x7b\xa7\x0e\xb6\x24\x8e\x59\x4e\x1a\x22\x97\xac\xbb\xb3\x9d\x50\x2f\x1a\x8c\x6e\xb6\xf1\xce\x22\xb3\xde\x1a\x1f\x40\xcc\x24\x55\x41\x19\xa8\x31\xa9\xaa\xd6\x07\x9c\xad\x88\x42\x5d\xe6\xbd\xe1\xa9\x18\x7e\xbb\x60\x92\xcf\x67\xbf\x2b\x13\xfd\x65\xf2\x70\x88\xd7\x8b\x7e\x88\x3c\x87\x59\xd2\xc4\xf5\xc6\x5a\xdb\x75\x53\x87\x8a\xd5\x75\xf9\xfa\xd8\x78\xe8\x0a\x0c\x9b\xa6\x3b\xcb\xcc\x27\x32\xe6\x94\x85\xbb\xc9\xc9\x0b\xfb\xd6\x24\x81\xd9\x08\x9b\xec\xcf\x80\xcf\xe2\xdf\x16\xa2\xcf\x65\xbd\x92\xdd\x59\x7b\x07\x07\xe0\x91\x7a\xf4\x8b\xbb\x75\xfe\xd4\x13\xd2\x38\xf5\x55\x5a\x7a\x56\x9d\x80\xc3\x41\x4a\x8d\x08\x59\xdc\x65\xa4\x61\x28\xba\xb2\x7a\xf8\x7a\x71\x31\x4f\x31\x8c\x78\x2b\x23\xeb\xfe\x80\x8b\x82\xb0\xce\x26\x40\x1d\x2e\x22\xf0\x4d\x83\xd1\x25\x5d\xc5\x1a\xdd\xd3\xb7\x5a\x2b\x1a\xe0\x78\x45\x04\xdf\x54\x3a\xf8\x96\x9b\xe3\xea\x70\x82\xff\x7f\xc9\x88\x8c\x14\x4d\xa2\xaf\x58\x42\x9e\xc9\x60\x31\xdb\xca\xd3\xda\xd9\xaf\x0d\xcb\xaa\xaf\x26\x8c\xb8\xfc\xff\xea\xd9\x4f\x3c\x7c\xa4\x95\xe0\x56\xa9\xb4\x7a\xcd\xb7\x51\xfb\x73\xe6\x66\xc6\xc6\x55\xad\xe8\x29\x72\x97\xd0\x7a\xd1\xba\x5e\x43\xf1\xbc\xa3\x23\x01\x65\x13\x39\xe2\x29\x04\xcc\x8c\x42\xf5\x8c\x30\xc0\x4a\xaf\xdb\x03\x8d\xda\x08\x47\xdd\x98\x8d\xcd\xa6\xf3\xbf\xd1\x5c\x4b\x4c\x45\x25\x00\x4a\xa0\x6e\xef\xf8\xca\x61\x78\x3a\xac\xec\x57\xfb\x3d\x1f\x92\xb0\xfe\x2f\xd1\xa8\x5f\x67\x24\x51\x7b\x65\xe6\x14\xad\x68\x08\xd6\xf6\xee\x34\xdf\xf7\x31\x0f\xdc\x82\xae\xbf\xd9\x04\xb0\x1e\x1d\xc5\x4b\x29\x27\x09\x4b\x2d\xb6\x8d\x6f\x90\x3b\x68\x40\x1a\xde\xbf\x5a\x7e\x08\xd7\x8f\xf4\xef\x5d\x63\x65\x3a\x65\x04\x0c\xf9\xbf\xd4\xac\xa7\x98\x4a\x74\xd3\x71\x45\x98\x67\x80\xfc\x0b\x16\xac\x45\x16\x49\xde\x61\x88\xa7\xdb\xdf\x19\x1f\x64\xb5\xfc\x5e\x2a\xb4\x7b\x57\xf7\xf7\x27\x6c\xd4\x19\xc1\x7a\x3c\xa8\xe1\xb9\x39\xae\x49\xe4\x88\xac\xba\x6b\x96\x56\x10\xb5\x48\x01\x09\xc8\xb1\x7b\x80\xe1\xb7\xb7\x50\xdf\xc7\x59\x8d\x5d\x50\x11\xfd\x2d\xcc\x56\x00\xa3\x2e\xf5\xb5\x2a\x1e\xcc\x82\x0e\x30\x8a\xa3\x42\x72\x1a\xac\x09\x43\xbf\x66\x86\xb6\x4b\x25\x79\x37\x65\x04\xcc\xc4\x93\xd9\x7e\x6a\xed\x3f\xb0\xf9\xcd\x71\xa4\x3d\xd4\x97\xf0\x1f\x17\xc0\xe2\xcb\x37\x97\xaa\x2a\x2f\x25\x66\x56\x16\x8e\x6c\x49\x6a\xfc\x5f\xb9\x32\x46\xf6\xb1\x11\x63\x98\xa3\x46\xf1\xa6\x41\xf3\xb0\x41\xe9\x89\xf7\x91\x4f\x90\xcc\x2c\x7f\xff\x35\x78\x76\xe5\x06\xb5\x0d\x33\x4b\xa7\x7c\x22\x5b\xc3\x07\xba\x53\x71\x52\xf3\xf1\x61\x0e\x4e\xaf\xe5\x95\xf6\xd9\xd9\x0d\x11\xfa\xa9\x33\xa1\x5e\xf1\x36\x95\x46\x86\x8a\x7f\x3a\x45\xa9\x67\x68\xd4\x0f\xd9\xd0\x34\x12\xc0\x91\xc6\x31\x5c\xf4\xfd\xe7\xcb\x68\x60\x69\x37\x38\x0d\xb2\xea\xaa\x70\x7b\x4c\x41\x85\xc3\x2e\xdd\xcd\xd3\x06\x70\x5e\x4d\xc1\xff\xc8\x72\xee\xee\x47\x5a\x64\xdf\xac\x86\xab\xa4\x1c\x06\x18\x98\x3f\x87\x41\xc5\xef\x68\xd3\xa1\x01\xe8\xa3\xb8\xca\xc6\x0c\x90\x5c\x15\xfc\x91\x08\x40\xb9\x4c\x00\xa0\xb9\xd0" + , vecSig = + "\xd0\x39\x65\xac\x31\x6a\x20\xf5\xa4\x7a\xb2\xd6\x18\x5e\xb3\xf0\xae\xea\x9c\x2e\xb8\xab\xe9\x22\xe9\x6d\x31\x7b\x3b\xd0\xef\x02\xe8\xd4\x7f\xd9\x23\x84\xe2\x86\x15\xeb\x33\x14\xad\xbc\x71\xc4\x67\x59\x96\x09\x9e\x48\x4c\xeb\x16\x28\x47\xc4\x0c\x32\x44\x0e" + } + ] + +doPublicKeyTest :: Int -> Vec -> Spec +doPublicKeyTest i Vec{..} = + it (show i) (EdDSA.toPublic vecPrx vecAlg sec `shouldBe` pub) + where + !pub = throwCryptoError $ EdDSA.publicKey vecPrx vecAlg vecPub + !sec = throwCryptoError $ EdDSA.secretKey vecPrx vecSec + +doSignatureTest :: Int -> Vec -> Spec +doSignatureTest i Vec{..} = + it (show i) (EdDSA.sign vecPrx sec pub vecMsg `shouldBe` sig) + where + !sig = throwCryptoError $ EdDSA.signature vecPrx vecAlg vecSig + !pub = throwCryptoError $ EdDSA.publicKey vecPrx vecAlg vecPub + !sec = throwCryptoError $ EdDSA.secretKey vecPrx vecSec + +doVerifyTest :: Int -> Vec -> Spec +doVerifyTest i Vec{..} = + it (show i) (EdDSA.verify vecPrx pub vecMsg sig `shouldBe` True) + where + !sig = throwCryptoError $ EdDSA.signature vecPrx vecAlg vecSig + !pub = throwCryptoError $ EdDSA.publicKey vecPrx vecAlg vecPub + +unhex :: ByteString -> ByteString +unhex = either error id . convertFromBase Base16 + +-- | Invalid signatures from Wycheproof's ed25519_test.json. +data NegVec + = forall curve hash. + ( EdDSA.EllipticCurveEdDSA curve + , HashAlgorithm hash + , HashDigestSize hash ~ EdDSA.CurveDigestSize curve + ) => + NegVec + { negPrx :: Maybe curve + , negAlg :: hash + , negTc :: Int + , negWhy :: String + , negPub :: ByteString + , negMsg :: ByteString + , negSig :: ByteString + } + +negVectors = + [ ed25519Neg + 63 + "s replaced by s + L" + "7d4d0e7f6153a69b6242b522abbee685fda4420f8834b108c3bdae369ef549fa" + "54657374" + "7c38e026f29e14aabd059a0f2db8b0cd783040609a8be684db12f82a27774ab067654bce3832c2d76f8f6f5dafc08d9339d4eef676573336a5c51eb6f946b31d" + , ed25519Neg + 64 + "s replaced by s + 2L" + "7d4d0e7f6153a69b6242b522abbee685fda4420f8834b108c3bdae369ef549fa" + "54657374" + "7c38e026f29e14aabd059a0f2db8b0cd783040609a8be684db12f82a27774ab05439412b5395d42f462c67008eba6ca839d4eef676573336a5c51eb6f946b32d" + , ed25519Neg + 65 + "s replaced by s + 4L" + "7d4d0e7f6153a69b6242b522abbee685fda4420f8834b108c3bdae369ef549fa" + "54657374" + "7c38e026f29e14aabd059a0f2db8b0cd783040609a8be684db12f82a27774ab02ee12ce5875bf9dff26556464bae2ad239d4eef676573336a5c51eb6f946b34d" + , ed25519Neg + 66 + "s replaced by s + 8L" + "7d4d0e7f6153a69b6242b522abbee685fda4420f8834b108c3bdae369ef549fa" + "54657374" + "7c38e026f29e14aabd059a0f2db8b0cd783040609a8be684db12f82a27774ab0e2300459f1e742404cd934d2c595a6253ad4eef676573336a5c51eb6f946b38d" + , ed25519Neg + 85 + "s just above the bound" + "100fdf47fb94f1536a4f7c3fda27383fa03375a8f527c537e6f1703c47f94f86" + "6a0bc2b0057cedfc0fa2e3f7f7d39279b30f454a69dfd1117c758d86b19d85e0" + "0971f86d2c9c78582524a103cb9cf949522ae528f8054dc20107d999be673ff4e25ebf2f2928766b1248bec6e91697775f8446639ede46ad4df4053000000010" + , ed25519Neg + 151 + "R encodes y = 1 with the sign bit of x set" + "d75a980182b10ab7d54bfed3c964073a0ee172f3daa62325af021a68f707511a" + "313233343030" + "0100000000000000000000000000000000000000000000000000000000000080c803ee1f2342aa96ff698a393d1ab5e66f3eda101d6d120b394c3fd32c117d0a" + ] + where + ed25519Neg tc why pub msg sig = + NegVec + { negPrx = Just Curve_Edwards25519 + , negAlg = SHA512 + , negTc = tc + , negWhy = why + , negPub = unhex pub + , negMsg = unhex msg + , negSig = unhex sig + } + +doNegVerifyTest :: NegVec -> Spec +doNegVerifyTest NegVec{..} = + it + (show negTc ++ ": " ++ negWhy) + (EdDSA.verify negPrx pub negMsg sig `shouldBe` False) + where + !sig = throwCryptoError $ EdDSA.signature negPrx negAlg negSig + !pub = throwCryptoError $ EdDSA.publicKey negPrx negAlg negPub + +spec :: Spec +spec = do + describe "gen publickey" $ zipWithM_ doPublicKeyTest [katZero ..] vectors + describe "gen signature" $ zipWithM_ doSignatureTest [katZero ..] vectors + describe "verify sig" $ zipWithM_ doVerifyTest [katZero ..] vectors + describe "reject non-canonical encoding" $ mapM_ doNegVerifyTest negVectors diff --git a/tests/Hash.hs b/tests/Hash.hs deleted file mode 100644 --- a/tests/Hash.hs +++ /dev/null @@ -1,509 +0,0 @@ -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE ExistentialQuantification #-} -{-# LANGUAGE OverloadedStrings #-} - -module Hash ( - tests, -) where - -import Crypto.Hash - -import Data.ByteArray (convert) -import qualified Data.ByteArray.Encoding as B (Base (..), convertToBase) -import qualified Data.ByteString as B -import GHC.TypeLits -import Imports - -v0, v1, v2 :: ByteString -v0 = "" -v1 = "The quick brown fox jumps over the lazy dog" -v2 = "The quick brown fox jumps over the lazy cog" -vectors = [v0, v1, v2] - -instance Arbitrary ByteString where - arbitrary = B.pack `fmap` arbitrary - -data HashAlg = forall alg. HashAlgorithm alg => HashAlg alg - -expected :: [(String, HashAlg, [ByteString])] -expected = - [ - ( "MD2" - , HashAlg MD2 - , - [ "8350e5a3e24c153df2275c9f80692773" - , "03d85a0d629d2c442e987525319fc471" - , "6b890c9292668cdbbfda00a4ebf31f05" - ] - ) - , - ( "MD4" - , HashAlg MD4 - , - [ "31d6cfe0d16ae931b73c59d7e0c089c0" - , "1bee69a46ba811185c194762abaeae90" - , "b86e130ce7028da59e672d56ad0113df" - ] - ) - , - ( "MD5" - , HashAlg MD5 - , - [ "d41d8cd98f00b204e9800998ecf8427e" - , "9e107d9d372bb6826bd81d3542a419d6" - , "1055d3e698d289f2af8663725127bd4b" - ] - ) - , - ( "SHA1" - , HashAlg SHA1 - , - [ "da39a3ee5e6b4b0d3255bfef95601890afd80709" - , "2fd4e1c67a2d28fced849ee1bb76e7391b93eb12" - , "de9f2c7fd25e1b3afad3e85a0bd17d9b100db4b3" - ] - ) - , - ( "SHA224" - , HashAlg SHA224 - , - [ "d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f" - , "730e109bd7a8a32b1cb9d9a09aa2325d2430587ddbc0c38bad911525" - , "fee755f44a55f20fb3362cdc3c493615b3cb574ed95ce610ee5b1e9b" - ] - ) - , - ( "SHA256" - , HashAlg SHA256 - , - [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" - , "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592" - , "e4c4d8f3bf76b692de791a173e05321150f7a345b46484fe427f6acc7ecc81be" - ] - ) - , - ( "SHA384" - , HashAlg SHA384 - , - [ "38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b" - , "ca737f1014a48f4c0b6dd43cb177b0afd9e5169367544c494011e3317dbf9a509cb1e5dc1e85a941bbee3d7f2afbc9b1" - , "098cea620b0978caa5f0befba6ddcf22764bea977e1c70b3483edfdf1de25f4b40d6cea3cadf00f809d422feb1f0161b" - ] - ) - , - ( "SHA512" - , HashAlg SHA512 - , - [ "cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e" - , "07e547d9586f6a73f73fbac0435ed76951218fb7d0c8d788a309d785436bbb642e93a252a954f23912547d1e8a3b5ed6e1bfd7097821233fa0538f3db854fee6" - , "3eeee1d0e11733ef152a6c29503b3ae20c4f1f3cda4cb26f1bc1a41f91c7fe4ab3bd86494049e201c4bd5155f31ecb7a3c8606843c4cc8dfcab7da11c8ae5045" - ] - ) - , - ( "SHA512/224" - , HashAlg SHA512t_224 - , - [ "6ed0dd02806fa89e25de060c19d3ac86cabb87d6a0ddd05c333b84f4" - , "944cd2847fb54558d4775db0485a50003111c8e5daa63fe722c6aa37" - , "2b9d6565a7e40f780ba8ab7c8dcf41e3ed3b77997f4c55aa987eede5" - ] - ) - , - ( "SHA512/256" - , HashAlg SHA512t_256 - , - [ "c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a" - , "dd9d67b371519c339ed8dbd25af90e976a1eeefd4ad3d889005e532fc5bef04d" - , "cc8d255a7f2f38fd50388fd1f65ea7910835c5c1e73da46fba01ea50d5dd76fb" - ] - ) - , - ( "RIPEMD160" - , HashAlg RIPEMD160 - , - [ "9c1185a5c5e9fc54612808977ee8f548b2258d31" - , "37f332f68db77bd9d7edd4969571ad671cf9dd3b" - , "132072df690933835eb8b6ad0b77e7b6f14acad7" - ] - ) - , - ( "Tiger" - , HashAlg Tiger - , - [ "3293ac630c13f0245f92bbb1766e16167a4e58492dde73f3" - , "6d12a41e72e644f017b6f0e2f7b44c6285f06dd5d2c5b075" - , "a8f04b0f7201a0d728101c9d26525b31764a3493fcd8458f" - ] - ) - , {- - , ("Skein256-160", HashAlg Skein256_160, [ - "ff800bed6d2044ee9d604a674e3fda50d9b24a72", - "3265703c166aa3e0d7da070b9cf1b1a5953f0a77", - "17b29aa1424b3ec022505bd215ff73fd2e6d1e5a" ]) - -} - - ( "Skein256-256" - , HashAlg Skein256_256 - , - [ "c8877087da56e072870daa843f176e9453115929094c3a40c463a196c29bf7ba" - , "c0fbd7d779b20f0a4614a66697f9e41859eaf382f14bf857e8cdb210adb9b3fe" - , "fb2f2f2deed0e1dd7ee2b91cee34e2d1c22072e1f5eaee288c35a0723eb653cd" - ] - ) - , {- - , ("Skein512-160", HashAlg Skein512_160, [ - "49daf1ccebb3544bc93cb5019ba91b0eea8876ee", - "826325ee55a6dd18c3b2dbbc9c10420f5475975e", - "7544ec7a35712ec953f02b0d0c86641cae4eb6e5" ]) - -} - - ( "Skein512-384" - , HashAlg Skein512_384 - , - [ "dd5aaf4589dc227bd1eb7bc68771f5baeaa3586ef6c7680167a023ec8ce26980f06c4082c488b4ac9ef313f8cbe70808" - , "f814c107f3465e7c54048a5503547deddc377264f05c706b0d19db4847b354855ee52ab6a785c238c9e710d848542041" - , "e06520eeadc1d0a44fee1d2492547499c1e58526387c8b9c53905e5edb79f9840575cbf844e21b1ad1ea126dd8a8ca6f" - ] - ) - , - ( "Skein512-512" - , HashAlg Skein512_512 - , - [ "bc5b4c50925519c290cc634277ae3d6257212395cba733bbad37a4af0fa06af41fca7903d06564fea7a2d3730dbdb80c1f85562dfcc070334ea4d1d9e72cba7a" - , "94c2ae036dba8783d0b3f7d6cc111ff810702f5c77707999be7e1c9486ff238a7044de734293147359b4ac7e1d09cd247c351d69826b78dcddd951f0ef912713" - , "7f81113575e4b4d3441940e87aca331e6d63d103fe5107f29cd877af0d0f5e0ea34164258c60da5190189d0872e63a96596d2ef25e709099842da71d64111e0f" - ] - ) - , {- - , ("Skein512-896", HashAlg Skein512_896, [ - "b95175236c83a459ce7ec6c12b761a838b22d750e765b3fdaa892201b2aa714bc3d1d887dd64028bbf177c1dd11baa09c6c4ddb598fd07d6a8c131a09fc5b958e2999a8006754b25abe3bf8492b7eabec70e52e04e5ac867df2393c573f16eee3244554f1d2b724f2c0437c62007f770", - "3265708553e7d146e5c7bcbc97b3e9e9f5b53a5e4af53612bdd6454da4fa7b13d413184fe34ed57b6574be10e389d0ec4b1d2b1dd2c80e0257d5a76b2cd86a19a27b1bcb3cc24d911b5dc5ee74d19ad558fd85b5f024e99f56d1d3199f1f9f88ed85fab9f945f11cf9fc00e94e3ca4c7", - "3d23d3db9be719bbd2119f8402a28f38d8225faa79d5b68b80738c64a82004aafc7a840cd6dd9bced6644fa894a3d8d7d2ee89525fd1956a2db052c4c2f8d2111c91ef46b0997540d42bcf384826af1a5ef6510077f52d0574cf2b46f1b6a5dad07ed40f3d21a13ca2d079fa602ff02d" ]) - -} - - ( "Whirlpool" - , HashAlg Whirlpool - , - [ "19fa61d75522a4669b44e39c1d2e1726c530232130d407f89afee0964997f7a73e83be698b288febcf88e3e03c4f0757ea8964e59b63d93708b138cc42a66eb3" - , "b97de512e91e3828b40d2b0fdce9ceb3c4a71f9bea8d88e75c4fa854df36725fd2b52eb6544edcacd6f8beddfea403cb55ae31f03ad62a5ef54e42ee82c3fb35" - , "dce81fc695cfea3d7e1446509238daf89f24cc61896f2d265927daa70f2108f8902f0dfd68be085d5abb9fcd2e482c1dc24f2fabf81f40b73495cad44d7360d3" - ] - ) - , - ( "Keccak-224" - , HashAlg Keccak_224 - , - [ "f71837502ba8e10837bdd8d365adb85591895602fc552b48b7390abd" - , "310aee6b30c47350576ac2873fa89fd190cdc488442f3ef654cf23fe" - , "0b27ff3b732133287f6831e2af47cf342b7ef1f3fcdee248811090cd" - ] - ) - , - ( "Keccak-256" - , HashAlg Keccak_256 - , - [ "c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470" - , "4d741b6f1eb29cb2a9b9911c82f56fa8d73b04959d3d9d222895df6c0b28aa15" - , "ed6c07f044d7573cc53bf1276f8cba3dac497919597a45b4599c8f73e22aa334" - ] - ) - , - ( "Keccak-384" - , HashAlg Keccak_384 - , - [ "2c23146a63a29acf99e73b88f8c24eaa7dc60aa771780ccc006afbfa8fe2479b2dd2b21362337441ac12b515911957ff" - , "283990fa9d5fb731d786c5bbee94ea4db4910f18c62c03d173fc0a5e494422e8a0b3da7574dae7fa0baf005e504063b3" - , "1cc515e1812491058d8b8b226fd85045e746b4937a58b0111b6b7a39dd431b6295bd6b6d05e01e225586b4dab3cbb87a" - ] - ) - , - ( "Keccak-512" - , HashAlg Keccak_512 - , - [ "0eab42de4c3ceb9235fc91acffe746b29c29a8c366b7c60e4e67c466f36a4304c00fa9caf9d87976ba469bcbe06713b435f091ef2769fb160cdab33d3670680e" - , "d135bb84d0439dbac432247ee573a23ea7d3c9deb2a968eb31d47c4fb45f1ef4422d6c531b5b9bd6f449ebcc449ea94d0a8f05f62130fda612da53c79659f609" - , "10f8caabb5b179861da5e447d34b84d604e3eb81830880e1c2135ffc94580a47cb21f6243ec0053d58b1124d13af2090033659075ee718e0f111bb3f69fb24cf" - ] - ) - , - ( "SHA3-224" - , HashAlg SHA3_224 - , - [ "6b4e03423667dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7" - , "d15dadceaa4d5d7bb3b48f446421d542e08ad8887305e28d58335795" - , "b770eb6ac3ac52bd2f9e8dc186d6b604e7c3b7ffc8bd9220b0078ced" - ] - ) - , - ( "SHA3-256" - , HashAlg SHA3_256 - , - [ "a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a" - , "69070dda01975c8c120c3aada1b282394e7f032fa9cf32f4cb2259a0897dfc04" - , "cc80b0b13ba89613d93f02ee7ccbe72ee26c6edfe577f22e63a1380221caedbc" - ] - ) - , - ( "SHA3-384" - , HashAlg SHA3_384 - , - [ "0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2ac3713831264adb47fb6bd1e058d5f004" - , "7063465e08a93bce31cd89d2e3ca8f602498696e253592ed26f07bf7e703cf328581e1471a7ba7ab119b1a9ebdf8be41" - , "e414797403c7d01ab64b41e90df4165d59b7f147e4292ba2da336acba242fd651949eb1cfff7e9012e134b40981842e1" - ] - ) - , - ( "SHA3-512" - , HashAlg SHA3_512 - , - [ "a69f73cca23a9ac5c8b567dc185a756e97c982164fe25859e0d1dcc1475c80a615b2123af1f5f94c11e3e9402c3ac558f500199d95b6d3e301758586281dcd26" - , "01dedd5de4ef14642445ba5f5b97c15e47b9ad931326e4b0727cd94cefc44fff23f07bf543139939b49128caf436dc1bdee54fcb24023a08d9403f9b4bf0d450" - , "28e361fe8c56e617caa56c28c7c36e5c13be552b77081be82b642f08bb7ef085b9a81910fe98269386b9aacfd2349076c9506126e198f6f6ad44c12017ca77b1" - ] - ) - , - ( "Blake2b-160" - , HashAlg Blake2b_160 - , - [ "3345524abf6bbe1809449224b5972c41790b6cf2" - , "3c523ed102ab45a37d54f5610d5a983162fde84f" - , "a3d365b5fba5d36fbb19c03b7fde496058969c5a" - ] - ) - , - ( "Blake2b-224" - , HashAlg Blake2b_224 - , - [ "836cc68931c2e4e3e838602eca1902591d216837bafddfe6f0c8cb07" - , "477c3985751dd4d1b8c93827ea5310b33bb02a26463a050dffd3e857" - , "a4a1b6851be66891a3deff406c4d7556879ebf952407450755f90eb6" - ] - ) - , - ( "Blake2b-256" - , HashAlg Blake2b_256 - , - [ "0e5751c026e543b2e8ab2eb06099daa1d1e5df47778f7787faab45cdf12fe3a8" - , "01718cec35cd3d796dd00020e0bfecb473ad23457d063b75eff29c0ffa2e58a9" - , "036c13096926b3dfccfe3f233bd1b2f583b818b8b15c01be65af69238e900b2c" - ] - ) - , - ( "Blake2b-384" - , HashAlg Blake2b_384 - , - [ "b32811423377f52d7862286ee1a72ee540524380fda1724a6f25d7978c6fd3244a6caf0498812673c5e05ef583825100" - , "b7c81b228b6bd912930e8f0b5387989691c1cee1e65aade4da3b86a3c9f678fc8018f6ed9e2906720c8d2a3aeda9c03d" - , "927a1f297873cbe887a93b2183c4e2eba53966ba92c6db8b87029a1d8c673471d09740676cced79c5016838973f630c3" - ] - ) - , - ( "Blake2b-512" - , HashAlg Blake2b_512 - , - [ "786a02f742015903c6c6fd852552d272912f4740e15847618a86e217f71f5419d25e1031afee585313896444934eb04b903a685b1448b755d56f701afe9be2ce" - , "a8add4bdddfd93e4877d2746e62817b116364a1fa7bc148d95090bc7333b3673f82401cf7aa2e4cb1ecd90296e3f14cb5413f8ed77be73045b13914cdcd6a918" - , "af438eea5d8cdb209336a7e85bf58090dc21b49d823f89a7d064c119f127bd361af9c7d109edda0f0e91bdce078d1d86b8e6f25727c98f6d3bb6f50acb2dd376" - ] - ) - , - ( "Blake2s-160" - , HashAlg Blake2s_160 - , - [ "354c9c33f735962418bdacb9479873429c34916f" - , "5a604fec9713c369e84b0ed68daed7d7504ef240" - , "759bef6d041bcbd861b8b51baaece6c8fffd0acf" - ] - ) - , - ( "Blake2s-224" - , HashAlg Blake2s_224 - , - [ "1fa1291e65248b37b3433475b2a0dd63d54a11ecc4e3e034e7bc1ef4" - , "e4e5cb6c7cae41982b397bf7b7d2d9d1949823ae78435326e8db4912" - , "e220025fd46a9a635c3f7f60bb96a84c01019ac0817f5901e7eeaa2c" - ] - ) - , - ( "Blake2s-256" - , HashAlg Blake2s_256 - , - [ "69217a3079908094e11121d042354a7c1f55b6482ca1a51e1b250dfd1ed0eef9" - , "606beeec743ccbeff6cbcdf5d5302aa855c256c29b88c8ed331ea1a6bf3c8812" - , "94662583a600a12dff357c0a6f1b514a710ef0f587a38e8d2e4d7f67e9c81667" - ] - ) - , - ( "SHAKE128_4096" - , HashAlg (SHAKE128 :: SHAKE128 4096) - , - [ "7f9c2ba4e88f827d616045507605853ed73b8093f6efbc88eb1a6eacfa66ef263cb1eea988004b93103cfb0aeefd2a686e01fa4a58e8a3639ca8a1e3f9ae57e235b8cc873c23dc62b8d260169afa2f75ab916a58d974918835d25e6a435085b2badfd6dfaac359a5efbb7bcc4b59d538df9a04302e10c8bc1cbf1a0b3a5120ea17cda7cfad765f5623474d368ccca8af0007cd9f5e4c849f167a580b14aabdefaee7eef47cb0fca9767be1fda69419dfb927e9df07348b196691abaeb580b32def58538b8d23f87732ea63b02b4fa0f4873360e2841928cd60dd4cee8cc0d4c922a96188d032675c8ac850933c7aff1533b94c834adbb69c6115bad4692d8619f90b0cdf8a7b9c264029ac185b70b83f2801f2f4b3f70c593ea3aeeb613a7f1b1de33fd75081f592305f2e4526edc09631b10958f464d889f31ba010250fda7f1368ec2967fc84ef2ae9aff268e0b1700affc6820b523a3d917135f2dff2ee06bfe72b3124721d4a26c04e53a75e30e73a7a9c4a95d91c55d495e9f51dd0b5e9d83c6d5e8ce803aa62b8d654db53d09b8dcff273cdfeb573fad8bcd45578bec2e770d01efde86e721a3f7c6cce275dabe6e2143f1af18da7efddc4c7b70b5e345db93cc936bea323491ccb38a388f546a9ff00dd4e1300b9b2153d2041d205b443e41b45a653f2a5c4492c1add544512dda2529833462b71a41a45be97290b6f" - , "f4202e3c5852f9182a0430fd8144f0a74b95e7417ecae17db0f8cfeed0e3e66eb5585ec6f86021cacf272c798bcf97d368b886b18fec3a571f096086a523717a3732d50db2b0b7998b4117ae66a761ccf1847a1616f4c07d5178d0d965f9feba351420f8bfb6f5ab9a0cb102568eabf3dfa4e22279f8082dce8143eb78235a1a54914ab71abb07f2f3648468370b9fbb071e074f1c030a4030225f40c39480339f3dc71d0f04f71326de1381674cc89e259e219927fae8ea2799a03da862a55afafe670957a2af3318d919d0a3358f3b891236d6a8e8d19999d1076b529968faefbd880d77bb300829dca87e9c8e4c28e0800ff37490a5bd8c36c0b0bdb2701a5d58d03378b9dbd384389e3ef0fd4003b08998fd3f32fe1a0810fc0eccaad94bca8dd83b34559c333f0b16dfc2896ed87b30ba14c81f87cd8b4bb6317db89b0e7e94c0616f9a665fba5b0e6fb3549c9d7b68e66d08a86eb2faec05cc462a771806b93cc38b0a4feb9935c6c8945da6a589891ba5ee99753cfdd38e1abc7147fd74b7c7d1ce0609b6680a2e18888d84949b6e6cf6a2aa4113535aaee079459e3f257b569a9450523c41f5b5ba4b79b3ba5949140a74bb048de0657d04954bdd71dae76f61e2a1f88aecb91cfa5b36c1bf3350a798dc4dcf48628effe3a0c5340c756bd922f78d0e36ef7df12ce78c179cc721ad087e15ea496bf5f60b21b5822d" - , "22fd225fb8f2c8d0d2097e1f8d38b6a9e619d39664dad3795f0336fda544d305e1be56a9953ecd6e4bec14b622f53da492eccbe257b9af84775a6bdf81aab6b1ba3492149b7ce4ea402c56e343939f78b9a9727193269798420c9323a9eb1fa63006e1482fcf15e3696aa1ae5cb66eb8aad4ac6041ca0b576b78785ad95bc5fb6f8a420ba9e1726552c0f2b97050ae69fc018d3f63b3a812a2d39ef64b8ae368472dec4341511b02cf77363c74f216055d5905412bc2bf57b4010b1bdce2882cb28fc7e4d470ed05219fc4d1ce11d11e10db369c807cd71891653638956b2f9d176e116188aff2fbb8519d9ad8c3fb52fa8bb4a0413364e89d9f9d7db627f2a2288babedcc314a19e45c6b7fb93b16a15d9953ff619ab4d523c481552588f711b450f854c858ebcb8cf49492d6240a753bde2109c486cac666bdba767c6e9254cc723f67855534c34d08ed486c67c1b8dd288c6010ce8e6c1c9b7f927acf4d71ee99729cee55ecec544245d0b51b31dd78eb99c2723e8ba5cf92ac7720e8933d9fd3596b90073f6980c5ed3f1cbfada26bdb6946e72391198e3d1cedebdba092324500e32b8e04e32550f6dd6c4befafa95acc206c5708300d2a8df2765751102f9738a1449c4c0d588f0076d0c1a5ee445eb85c97ada5837d5dc1d34ea081c8411d64a4d9ce9279bfe9feb25696cf741c705ed46f171e2239216d6c45dc8d15" - ] - ) - , - ( "SHAKE256_4096" - , HashAlg (SHAKE256 :: SHAKE256 4096) - , - [ "46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762fd75dc4ddd8c0f200cb05019d67b592f6fc821c49479ab48640292eacb3b7c4be141e96616fb13957692cc7edd0b45ae3dc07223c8e92937bef84bc0eab862853349ec75546f58fb7c2775c38462c5010d846c185c15111e595522a6bcd16cf86f3d122109e3b1fdd943b6aec468a2d621a7c06c6a957c62b54dafc3be87567d677231395f6147293b68ceab7a9e0c58d864e8efde4e1b9a46cbe854713672f5caaae314ed9083dab4b099f8e300f01b8650f1f4b1d8fcf3f3cb53fb8e9eb2ea203bdc970f50ae55428a91f7f53ac266b28419c3778a15fd248d339ede785fb7f5a1aaa96d313eacc890936c173cdcd0fab882c45755feb3aed96d477ff96390bf9a66d1368b208e21f7c10d04a3dbd4e360633e5db4b602601c14cea737db3dcf722632cc77851cbdde2aaf0a33a07b373445df490cc8fc1e4160ff118378f11f0477de055a81a9eda57a4a2cfb0c83929d310912f729ec6cfa36c6ac6a75837143045d791cc85eff5b21932f23861bcf23a52b5da67eaf7baae0f5fb1369db78f3ac45f8c4ac5671d85735cdddb09d2b1e34a1fc066ff4a162cb263d6541274ae2fcc865f618abe27c124cd8b074ccd516301b91875824d09958f341ef274bdab0bae316339894304e35877b0c28a9b1fd166c796b9cc258a064a8f57e27f2a" - , "2f671343d9b2e1604dc9dcf0753e5fe15c7c64a0d283cbbf722d411a0e36f6ca1d01d1369a23539cd80f7c054b6e5daf9c962cad5b8ed5bd11998b40d5734442bed798f6e5c915bd8bb07e0188d0a55c1290074f1c287af06352299184492cbdec9acba737ee292e5adaa445547355e72a03a3bac3aac770fe5d6b66600ff15d37d5b4789994ea2aeb097f550aa5e88e4d8ff0ba07b88c1c88573063f5d96df820abc2abd177ab037f351c375e553af917132cf2f563c79a619e1bb76e8e2266b0c5617d695f2c496a25f4073b6840c1833757ebb386f16757a8e16a21e9355e9b248f3b33be672da700266be99b8f8725e8ab06075f0219e655ebc188976364b7db139390d34a6ea67b4b223229183a94cf455ece91fdaf5b9c707fa4b40ec39816c1120c7aaaf47920977be900e6b9ca4b8940e192b927c475bd58e836f512ae3e52924e36ff8e9b1d0251047770a5e465905622b1f159be121ab93819c5e5c6dae299ac73bf1c4ed4a1e2c7fa3caa1039b05e94c9f993d04feb272b6e00bb0276939cf746c42936831fc8f2b4cb0cf94808ae0af405ce4bc67d1e7acfc6fd6590d3de91f795df5aaf57e2cee1845a303d0ea564be3f1299acdce67efe0d62cfc6d6829ff4ecc0a05153c24696c4d34c076453827e796f3062f94f62f4528b7cfc870f0dcd615b7c97b95da4b9be5830e8b3f66cce71e0f622c771994443e2" - , "fffcaac0606c0edb7bc0d15f033accb68538159016e5ae8470bf9ebea89fa6c9fcc3e027d94f7f967b7246346bd9f6b8084e45a057b976847c4db03bf383c834054866f6a8282a497368c46e1852fc09e20f22c45607a27c8b2a4798ebefada54f8d3795b9f07606b1cd6e41f90d765480ef5c0d5790659cf1d210adfd412378b92e1dd9bd7fd95a1a66677fc6baa0e3a53c9031c1fb59cbad9f5dc5881a3c8e25c80ecb1abf0971488ada1f533dcbf8d37031335378574b8d3fad61159c9fae28caa543b3072ce308d369be340e78c6edc664cc6dde9b2f0a4ad2e60ce9c8b1e5722b8d5b73d0962b74fb9ed86307a180f53933339f9d56d3b345c2a0e98fcf5de7754f3845f6be30089f0e142ad4602f18abdc750bda7c91c3f32872e66640db46045ab4c276b379f1b834c2cbb1bd8601305649ec6b3bf20618695136dee6541492d1d985ea1fb765fd7a559e810eba30f2f710233ae5a411b94ddcaa01a08f1c31320d111c0714422cd5e987c9a76fc865de34003ab12664081be8017d23d977f2bf4ed9e3ce09ea3d64bb4ae8ebfa9d0721f57841008c297e2f455a0441a2bd618ca379dbd239a21e410defb4001b1e11f87e36bf894c222f76f12ddcc3771bbb17d5c0dfd86d89a3e13e084f6dc1c4762bcd393c1757db7afb1434221569e7ddaaffd6318253ec3df8cf5f826b81896d6474ee06a2e30ccc8c6a96bdd5" - ] - ) - , - ( "Blake2b 160" - , HashAlg (Blake2b :: Blake2b 160) - , - [ "3345524abf6bbe1809449224b5972c41790b6cf2" - , "3c523ed102ab45a37d54f5610d5a983162fde84f" - , "a3d365b5fba5d36fbb19c03b7fde496058969c5a" - ] - ) - , - ( "Blake2b 224" - , HashAlg (Blake2b :: Blake2b 224) - , - [ "836cc68931c2e4e3e838602eca1902591d216837bafddfe6f0c8cb07" - , "477c3985751dd4d1b8c93827ea5310b33bb02a26463a050dffd3e857" - , "a4a1b6851be66891a3deff406c4d7556879ebf952407450755f90eb6" - ] - ) - , - ( "Blake2b 256" - , HashAlg (Blake2b :: Blake2b 256) - , - [ "0e5751c026e543b2e8ab2eb06099daa1d1e5df47778f7787faab45cdf12fe3a8" - , "01718cec35cd3d796dd00020e0bfecb473ad23457d063b75eff29c0ffa2e58a9" - , "036c13096926b3dfccfe3f233bd1b2f583b818b8b15c01be65af69238e900b2c" - ] - ) - , - ( "Blake2b 384" - , HashAlg (Blake2b :: Blake2b 384) - , - [ "b32811423377f52d7862286ee1a72ee540524380fda1724a6f25d7978c6fd3244a6caf0498812673c5e05ef583825100" - , "b7c81b228b6bd912930e8f0b5387989691c1cee1e65aade4da3b86a3c9f678fc8018f6ed9e2906720c8d2a3aeda9c03d" - , "927a1f297873cbe887a93b2183c4e2eba53966ba92c6db8b87029a1d8c673471d09740676cced79c5016838973f630c3" - ] - ) - , - ( "Blake2b 512" - , HashAlg (Blake2b :: Blake2b 512) - , - [ "786a02f742015903c6c6fd852552d272912f4740e15847618a86e217f71f5419d25e1031afee585313896444934eb04b903a685b1448b755d56f701afe9be2ce" - , "a8add4bdddfd93e4877d2746e62817b116364a1fa7bc148d95090bc7333b3673f82401cf7aa2e4cb1ecd90296e3f14cb5413f8ed77be73045b13914cdcd6a918" - , "af438eea5d8cdb209336a7e85bf58090dc21b49d823f89a7d064c119f127bd361af9c7d109edda0f0e91bdce078d1d86b8e6f25727c98f6d3bb6f50acb2dd376" - ] - ) - , - ( "Blake2s 160" - , HashAlg (Blake2s :: Blake2s 160) - , - [ "354c9c33f735962418bdacb9479873429c34916f" - , "5a604fec9713c369e84b0ed68daed7d7504ef240" - , "759bef6d041bcbd861b8b51baaece6c8fffd0acf" - ] - ) - , - ( "Blake2s 224" - , HashAlg (Blake2s :: Blake2s 224) - , - [ "1fa1291e65248b37b3433475b2a0dd63d54a11ecc4e3e034e7bc1ef4" - , "e4e5cb6c7cae41982b397bf7b7d2d9d1949823ae78435326e8db4912" - , "e220025fd46a9a635c3f7f60bb96a84c01019ac0817f5901e7eeaa2c" - ] - ) - , - ( "Blake2s 256" - , HashAlg (Blake2s :: Blake2s 256) - , - [ "69217a3079908094e11121d042354a7c1f55b6482ca1a51e1b250dfd1ed0eef9" - , "606beeec743ccbeff6cbcdf5d5302aa855c256c29b88c8ed331ea1a6bf3c8812" - , "94662583a600a12dff357c0a6f1b514a710ef0f587a38e8d2e4d7f67e9c81667" - ] - ) - ] - -runhash :: HashAlg -> ByteString -> ByteString -runhash (HashAlg hashAlg) v = B.convertToBase B.Base16 $ hashWith hashAlg $ v - -runhashinc :: HashAlg -> [ByteString] -> ByteString -runhashinc (HashAlg hashAlg) v = B.convertToBase B.Base16 $ hashinc $ v - where - hashinc = hashFinalize . foldl hashUpdate (hashInitWith hashAlg) - -data HashPrefixAlg = forall alg. HashAlgorithmPrefix alg => HashPrefixAlg alg - -expectedPrefix :: [(String, HashPrefixAlg)] -expectedPrefix = - [ ("MD5", HashPrefixAlg MD5) - , ("SHA1", HashPrefixAlg SHA1) - , ("SHA224", HashPrefixAlg SHA224) - , ("SHA256", HashPrefixAlg SHA256) - , ("SHA384", HashPrefixAlg SHA384) - , ("SHA512", HashPrefixAlg SHA512) - ] - -runhashpfx :: HashPrefixAlg -> ByteString -> ByteString -runhashpfx (HashPrefixAlg hashAlg) v = B.convertToBase B.Base16 $ hashWith hashAlg v - -runhashpfxpfx :: HashPrefixAlg -> ByteString -> Int -> ByteString -runhashpfxpfx (HashPrefixAlg hashAlg) v len = B.convertToBase B.Base16 $ hashPrefixWith hashAlg v len - -makeTestAlg (name, hashAlg, results) = - testGroup name $ concatMap maketest (zip3 is vectors results) - where - is :: [Int] - is = [1 ..] - - maketest (i, v, r) = - [ testCase (show i) (r @=? runhash hashAlg v) - ] - -makeTestChunk (hashName, hashAlg, _) = - [ testProperty hashName $ \ckLen (ArbitraryBS0_2901 inp) -> - runhash hashAlg inp `propertyEq` runhashinc hashAlg (chunkS ckLen inp) - ] - -makeTestPrefix (hashName, hashAlg) = - [ testProperty hashName $ \(ArbitraryBS0_2901 inp) (Int0_2901 len) -> - runhashpfx hashAlg (B.take len inp) `propertyEq` runhashpfxpfx hashAlg inp len - ] - -makeTestHybrid (hashName, HashPrefixAlg alg) = - [ testProperty hashName $ \(ArbitraryBS0_2901 start) (ArbitraryBS0_2901 end) -> do - len <- choose (0, B.length end) - let ref = hashWith alg (start `B.append` B.take len end) - hyb = hashFinalizePrefix (hashUpdate (hashInitWith alg) start) end len - return (ref `propertyEq` hyb) - ] - --- SHAKE128 truncation example with expected byte at final position --- -shake128TruncationBytes = [0x01, 0x03, 0x07, 0x0f, 0x0f, 0x2f, 0x6f, 0x6f] - -makeTestSHAKE128Truncation i byte = - testCase (show i) $ xof 4088 `B.snoc` byte @=? xof (4088 + i) - where - hashEmpty :: KnownNat n => proxy n -> Digest (SHAKE128 n) - hashEmpty _ = hash B.empty - - xof n = case someNatVal n of - Nothing -> error ("invalid Nat: " ++ show n) - Just (SomeNat p) -> convert (hashEmpty p) - -tests = - testGroup - "hash" - [ testGroup "KATs" (map makeTestAlg expected) - , testGroup "Chunking" (concatMap makeTestChunk expected) - , testGroup "Prefix" (concatMap makeTestPrefix expectedPrefix) - , testGroup "Hybrid" (concatMap makeTestHybrid expectedPrefix) - , testGroup - "Truncating" - [ testGroup - "SHAKE128" - (zipWith makeTestSHAKE128Truncation [1 ..] shake128TruncationBytes) - ] - ] diff --git a/tests/HashSpec.hs b/tests/HashSpec.hs new file mode 100644 --- /dev/null +++ b/tests/HashSpec.hs @@ -0,0 +1,577 @@ +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE ExistentialQuantification #-} +{-# LANGUAGE OverloadedStrings #-} + +module HashSpec ( + spec, +) where + +import Crypto.Hash + +import Data.ByteArray (convert) +import qualified Data.ByteArray.Encoding as B (Base (..), convertToBase) +import qualified Data.ByteString as B +import GHC.TypeLits +import Imports + +v0, v1, v2 :: ByteString +v0 = "" +v1 = "The quick brown fox jumps over the lazy dog" +v2 = "The quick brown fox jumps over the lazy cog" +vectors = [v0, v1, v2] + +instance Arbitrary ByteString where + arbitrary = B.pack `fmap` arbitrary + +data HashAlg = forall alg. HashAlgorithm alg => HashAlg alg + +expected :: [(String, HashAlg, [ByteString])] +expected = + [ + ( "MD2" + , HashAlg MD2 + , + [ "8350e5a3e24c153df2275c9f80692773" + , "03d85a0d629d2c442e987525319fc471" + , "6b890c9292668cdbbfda00a4ebf31f05" + ] + ) + , + ( "MD4" + , HashAlg MD4 + , + [ "31d6cfe0d16ae931b73c59d7e0c089c0" + , "1bee69a46ba811185c194762abaeae90" + , "b86e130ce7028da59e672d56ad0113df" + ] + ) + , + ( "MD5" + , HashAlg MD5 + , + [ "d41d8cd98f00b204e9800998ecf8427e" + , "9e107d9d372bb6826bd81d3542a419d6" + , "1055d3e698d289f2af8663725127bd4b" + ] + ) + , + ( "SHA1" + , HashAlg SHA1 + , + [ "da39a3ee5e6b4b0d3255bfef95601890afd80709" + , "2fd4e1c67a2d28fced849ee1bb76e7391b93eb12" + , "de9f2c7fd25e1b3afad3e85a0bd17d9b100db4b3" + ] + ) + , + ( "SHA224" + , HashAlg SHA224 + , + [ "d14a028c2a3a2bc9476102bb288234c415a2b01f828ea62ac5b3e42f" + , "730e109bd7a8a32b1cb9d9a09aa2325d2430587ddbc0c38bad911525" + , "fee755f44a55f20fb3362cdc3c493615b3cb574ed95ce610ee5b1e9b" + ] + ) + , + ( "SHA256" + , HashAlg SHA256 + , + [ "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + , "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592" + , "e4c4d8f3bf76b692de791a173e05321150f7a345b46484fe427f6acc7ecc81be" + ] + ) + , + ( "SHA384" + , HashAlg SHA384 + , + [ "38b060a751ac96384cd9327eb1b1e36a21fdb71114be07434c0cc7bf63f6e1da274edebfe76f65fbd51ad2f14898b95b" + , "ca737f1014a48f4c0b6dd43cb177b0afd9e5169367544c494011e3317dbf9a509cb1e5dc1e85a941bbee3d7f2afbc9b1" + , "098cea620b0978caa5f0befba6ddcf22764bea977e1c70b3483edfdf1de25f4b40d6cea3cadf00f809d422feb1f0161b" + ] + ) + , + ( "SHA512" + , HashAlg SHA512 + , + [ "cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e" + , "07e547d9586f6a73f73fbac0435ed76951218fb7d0c8d788a309d785436bbb642e93a252a954f23912547d1e8a3b5ed6e1bfd7097821233fa0538f3db854fee6" + , "3eeee1d0e11733ef152a6c29503b3ae20c4f1f3cda4cb26f1bc1a41f91c7fe4ab3bd86494049e201c4bd5155f31ecb7a3c8606843c4cc8dfcab7da11c8ae5045" + ] + ) + , + ( "SHA512/224" + , HashAlg SHA512t_224 + , + [ "6ed0dd02806fa89e25de060c19d3ac86cabb87d6a0ddd05c333b84f4" + , "944cd2847fb54558d4775db0485a50003111c8e5daa63fe722c6aa37" + , "2b9d6565a7e40f780ba8ab7c8dcf41e3ed3b77997f4c55aa987eede5" + ] + ) + , + ( "SHA512/256" + , HashAlg SHA512t_256 + , + [ "c672b8d1ef56ed28ab87c3622c5114069bdd3ad7b8f9737498d0c01ecef0967a" + , "dd9d67b371519c339ed8dbd25af90e976a1eeefd4ad3d889005e532fc5bef04d" + , "cc8d255a7f2f38fd50388fd1f65ea7910835c5c1e73da46fba01ea50d5dd76fb" + ] + ) + , + ( "RIPEMD160" + , HashAlg RIPEMD160 + , + [ "9c1185a5c5e9fc54612808977ee8f548b2258d31" + , "37f332f68db77bd9d7edd4969571ad671cf9dd3b" + , "132072df690933835eb8b6ad0b77e7b6f14acad7" + ] + ) + , + ( "Tiger" + , HashAlg Tiger + , + [ "3293ac630c13f0245f92bbb1766e16167a4e58492dde73f3" + , "6d12a41e72e644f017b6f0e2f7b44c6285f06dd5d2c5b075" + , "a8f04b0f7201a0d728101c9d26525b31764a3493fcd8458f" + ] + ) + , {- + , ("Skein256-160", HashAlg Skein256_160, [ + "ff800bed6d2044ee9d604a674e3fda50d9b24a72", + "3265703c166aa3e0d7da070b9cf1b1a5953f0a77", + "17b29aa1424b3ec022505bd215ff73fd2e6d1e5a" ]) + -} + + ( "Skein256-256" + , HashAlg Skein256_256 + , + [ "c8877087da56e072870daa843f176e9453115929094c3a40c463a196c29bf7ba" + , "c0fbd7d779b20f0a4614a66697f9e41859eaf382f14bf857e8cdb210adb9b3fe" + , "fb2f2f2deed0e1dd7ee2b91cee34e2d1c22072e1f5eaee288c35a0723eb653cd" + ] + ) + , {- + , ("Skein512-160", HashAlg Skein512_160, [ + "49daf1ccebb3544bc93cb5019ba91b0eea8876ee", + "826325ee55a6dd18c3b2dbbc9c10420f5475975e", + "7544ec7a35712ec953f02b0d0c86641cae4eb6e5" ]) + -} + + ( "Skein512-384" + , HashAlg Skein512_384 + , + [ "dd5aaf4589dc227bd1eb7bc68771f5baeaa3586ef6c7680167a023ec8ce26980f06c4082c488b4ac9ef313f8cbe70808" + , "f814c107f3465e7c54048a5503547deddc377264f05c706b0d19db4847b354855ee52ab6a785c238c9e710d848542041" + , "e06520eeadc1d0a44fee1d2492547499c1e58526387c8b9c53905e5edb79f9840575cbf844e21b1ad1ea126dd8a8ca6f" + ] + ) + , + ( "Skein512-512" + , HashAlg Skein512_512 + , + [ "bc5b4c50925519c290cc634277ae3d6257212395cba733bbad37a4af0fa06af41fca7903d06564fea7a2d3730dbdb80c1f85562dfcc070334ea4d1d9e72cba7a" + , "94c2ae036dba8783d0b3f7d6cc111ff810702f5c77707999be7e1c9486ff238a7044de734293147359b4ac7e1d09cd247c351d69826b78dcddd951f0ef912713" + , "7f81113575e4b4d3441940e87aca331e6d63d103fe5107f29cd877af0d0f5e0ea34164258c60da5190189d0872e63a96596d2ef25e709099842da71d64111e0f" + ] + ) + , {- + , ("Skein512-896", HashAlg Skein512_896, [ + "b95175236c83a459ce7ec6c12b761a838b22d750e765b3fdaa892201b2aa714bc3d1d887dd64028bbf177c1dd11baa09c6c4ddb598fd07d6a8c131a09fc5b958e2999a8006754b25abe3bf8492b7eabec70e52e04e5ac867df2393c573f16eee3244554f1d2b724f2c0437c62007f770", + "3265708553e7d146e5c7bcbc97b3e9e9f5b53a5e4af53612bdd6454da4fa7b13d413184fe34ed57b6574be10e389d0ec4b1d2b1dd2c80e0257d5a76b2cd86a19a27b1bcb3cc24d911b5dc5ee74d19ad558fd85b5f024e99f56d1d3199f1f9f88ed85fab9f945f11cf9fc00e94e3ca4c7", + "3d23d3db9be719bbd2119f8402a28f38d8225faa79d5b68b80738c64a82004aafc7a840cd6dd9bced6644fa894a3d8d7d2ee89525fd1956a2db052c4c2f8d2111c91ef46b0997540d42bcf384826af1a5ef6510077f52d0574cf2b46f1b6a5dad07ed40f3d21a13ca2d079fa602ff02d" ]) + -} + + ( "Whirlpool" + , HashAlg Whirlpool + , + [ "19fa61d75522a4669b44e39c1d2e1726c530232130d407f89afee0964997f7a73e83be698b288febcf88e3e03c4f0757ea8964e59b63d93708b138cc42a66eb3" + , "b97de512e91e3828b40d2b0fdce9ceb3c4a71f9bea8d88e75c4fa854df36725fd2b52eb6544edcacd6f8beddfea403cb55ae31f03ad62a5ef54e42ee82c3fb35" + , "dce81fc695cfea3d7e1446509238daf89f24cc61896f2d265927daa70f2108f8902f0dfd68be085d5abb9fcd2e482c1dc24f2fabf81f40b73495cad44d7360d3" + ] + ) + , + ( "Keccak-224" + , HashAlg Keccak_224 + , + [ "f71837502ba8e10837bdd8d365adb85591895602fc552b48b7390abd" + , "310aee6b30c47350576ac2873fa89fd190cdc488442f3ef654cf23fe" + , "0b27ff3b732133287f6831e2af47cf342b7ef1f3fcdee248811090cd" + ] + ) + , + ( "Keccak-256" + , HashAlg Keccak_256 + , + [ "c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470" + , "4d741b6f1eb29cb2a9b9911c82f56fa8d73b04959d3d9d222895df6c0b28aa15" + , "ed6c07f044d7573cc53bf1276f8cba3dac497919597a45b4599c8f73e22aa334" + ] + ) + , + ( "Keccak-384" + , HashAlg Keccak_384 + , + [ "2c23146a63a29acf99e73b88f8c24eaa7dc60aa771780ccc006afbfa8fe2479b2dd2b21362337441ac12b515911957ff" + , "283990fa9d5fb731d786c5bbee94ea4db4910f18c62c03d173fc0a5e494422e8a0b3da7574dae7fa0baf005e504063b3" + , "1cc515e1812491058d8b8b226fd85045e746b4937a58b0111b6b7a39dd431b6295bd6b6d05e01e225586b4dab3cbb87a" + ] + ) + , + ( "Keccak-512" + , HashAlg Keccak_512 + , + [ "0eab42de4c3ceb9235fc91acffe746b29c29a8c366b7c60e4e67c466f36a4304c00fa9caf9d87976ba469bcbe06713b435f091ef2769fb160cdab33d3670680e" + , "d135bb84d0439dbac432247ee573a23ea7d3c9deb2a968eb31d47c4fb45f1ef4422d6c531b5b9bd6f449ebcc449ea94d0a8f05f62130fda612da53c79659f609" + , "10f8caabb5b179861da5e447d34b84d604e3eb81830880e1c2135ffc94580a47cb21f6243ec0053d58b1124d13af2090033659075ee718e0f111bb3f69fb24cf" + ] + ) + , + ( "SHA3-224" + , HashAlg SHA3_224 + , + [ "6b4e03423667dbb73b6e15454f0eb1abd4597f9a1b078e3f5b5a6bc7" + , "d15dadceaa4d5d7bb3b48f446421d542e08ad8887305e28d58335795" + , "b770eb6ac3ac52bd2f9e8dc186d6b604e7c3b7ffc8bd9220b0078ced" + ] + ) + , + ( "SHA3-256" + , HashAlg SHA3_256 + , + [ "a7ffc6f8bf1ed76651c14756a061d662f580ff4de43b49fa82d80a4b80f8434a" + , "69070dda01975c8c120c3aada1b282394e7f032fa9cf32f4cb2259a0897dfc04" + , "cc80b0b13ba89613d93f02ee7ccbe72ee26c6edfe577f22e63a1380221caedbc" + ] + ) + , + ( "SHA3-384" + , HashAlg SHA3_384 + , + [ "0c63a75b845e4f7d01107d852e4c2485c51a50aaaa94fc61995e71bbee983a2ac3713831264adb47fb6bd1e058d5f004" + , "7063465e08a93bce31cd89d2e3ca8f602498696e253592ed26f07bf7e703cf328581e1471a7ba7ab119b1a9ebdf8be41" + , "e414797403c7d01ab64b41e90df4165d59b7f147e4292ba2da336acba242fd651949eb1cfff7e9012e134b40981842e1" + ] + ) + , + ( "SHA3-512" + , HashAlg SHA3_512 + , + [ "a69f73cca23a9ac5c8b567dc185a756e97c982164fe25859e0d1dcc1475c80a615b2123af1f5f94c11e3e9402c3ac558f500199d95b6d3e301758586281dcd26" + , "01dedd5de4ef14642445ba5f5b97c15e47b9ad931326e4b0727cd94cefc44fff23f07bf543139939b49128caf436dc1bdee54fcb24023a08d9403f9b4bf0d450" + , "28e361fe8c56e617caa56c28c7c36e5c13be552b77081be82b642f08bb7ef085b9a81910fe98269386b9aacfd2349076c9506126e198f6f6ad44c12017ca77b1" + ] + ) + , + ( "Blake2b-160" + , HashAlg Blake2b_160 + , + [ "3345524abf6bbe1809449224b5972c41790b6cf2" + , "3c523ed102ab45a37d54f5610d5a983162fde84f" + , "a3d365b5fba5d36fbb19c03b7fde496058969c5a" + ] + ) + , + ( "Blake2b-224" + , HashAlg Blake2b_224 + , + [ "836cc68931c2e4e3e838602eca1902591d216837bafddfe6f0c8cb07" + , "477c3985751dd4d1b8c93827ea5310b33bb02a26463a050dffd3e857" + , "a4a1b6851be66891a3deff406c4d7556879ebf952407450755f90eb6" + ] + ) + , + ( "Blake2b-256" + , HashAlg Blake2b_256 + , + [ "0e5751c026e543b2e8ab2eb06099daa1d1e5df47778f7787faab45cdf12fe3a8" + , "01718cec35cd3d796dd00020e0bfecb473ad23457d063b75eff29c0ffa2e58a9" + , "036c13096926b3dfccfe3f233bd1b2f583b818b8b15c01be65af69238e900b2c" + ] + ) + , + ( "Blake2b-384" + , HashAlg Blake2b_384 + , + [ "b32811423377f52d7862286ee1a72ee540524380fda1724a6f25d7978c6fd3244a6caf0498812673c5e05ef583825100" + , "b7c81b228b6bd912930e8f0b5387989691c1cee1e65aade4da3b86a3c9f678fc8018f6ed9e2906720c8d2a3aeda9c03d" + , "927a1f297873cbe887a93b2183c4e2eba53966ba92c6db8b87029a1d8c673471d09740676cced79c5016838973f630c3" + ] + ) + , + ( "Blake2b-512" + , HashAlg Blake2b_512 + , + [ "786a02f742015903c6c6fd852552d272912f4740e15847618a86e217f71f5419d25e1031afee585313896444934eb04b903a685b1448b755d56f701afe9be2ce" + , "a8add4bdddfd93e4877d2746e62817b116364a1fa7bc148d95090bc7333b3673f82401cf7aa2e4cb1ecd90296e3f14cb5413f8ed77be73045b13914cdcd6a918" + , "af438eea5d8cdb209336a7e85bf58090dc21b49d823f89a7d064c119f127bd361af9c7d109edda0f0e91bdce078d1d86b8e6f25727c98f6d3bb6f50acb2dd376" + ] + ) + , + ( "Blake2s-160" + , HashAlg Blake2s_160 + , + [ "354c9c33f735962418bdacb9479873429c34916f" + , "5a604fec9713c369e84b0ed68daed7d7504ef240" + , "759bef6d041bcbd861b8b51baaece6c8fffd0acf" + ] + ) + , + ( "Blake2s-224" + , HashAlg Blake2s_224 + , + [ "1fa1291e65248b37b3433475b2a0dd63d54a11ecc4e3e034e7bc1ef4" + , "e4e5cb6c7cae41982b397bf7b7d2d9d1949823ae78435326e8db4912" + , "e220025fd46a9a635c3f7f60bb96a84c01019ac0817f5901e7eeaa2c" + ] + ) + , + ( "Blake2s-256" + , HashAlg Blake2s_256 + , + [ "69217a3079908094e11121d042354a7c1f55b6482ca1a51e1b250dfd1ed0eef9" + , "606beeec743ccbeff6cbcdf5d5302aa855c256c29b88c8ed331ea1a6bf3c8812" + , "94662583a600a12dff357c0a6f1b514a710ef0f587a38e8d2e4d7f67e9c81667" + ] + ) + , + ( "SHAKE128_4096" + , HashAlg (SHAKE128 :: SHAKE128 4096) + , + [ "7f9c2ba4e88f827d616045507605853ed73b8093f6efbc88eb1a6eacfa66ef263cb1eea988004b93103cfb0aeefd2a686e01fa4a58e8a3639ca8a1e3f9ae57e235b8cc873c23dc62b8d260169afa2f75ab916a58d974918835d25e6a435085b2badfd6dfaac359a5efbb7bcc4b59d538df9a04302e10c8bc1cbf1a0b3a5120ea17cda7cfad765f5623474d368ccca8af0007cd9f5e4c849f167a580b14aabdefaee7eef47cb0fca9767be1fda69419dfb927e9df07348b196691abaeb580b32def58538b8d23f87732ea63b02b4fa0f4873360e2841928cd60dd4cee8cc0d4c922a96188d032675c8ac850933c7aff1533b94c834adbb69c6115bad4692d8619f90b0cdf8a7b9c264029ac185b70b83f2801f2f4b3f70c593ea3aeeb613a7f1b1de33fd75081f592305f2e4526edc09631b10958f464d889f31ba010250fda7f1368ec2967fc84ef2ae9aff268e0b1700affc6820b523a3d917135f2dff2ee06bfe72b3124721d4a26c04e53a75e30e73a7a9c4a95d91c55d495e9f51dd0b5e9d83c6d5e8ce803aa62b8d654db53d09b8dcff273cdfeb573fad8bcd45578bec2e770d01efde86e721a3f7c6cce275dabe6e2143f1af18da7efddc4c7b70b5e345db93cc936bea323491ccb38a388f546a9ff00dd4e1300b9b2153d2041d205b443e41b45a653f2a5c4492c1add544512dda2529833462b71a41a45be97290b6f" + , "f4202e3c5852f9182a0430fd8144f0a74b95e7417ecae17db0f8cfeed0e3e66eb5585ec6f86021cacf272c798bcf97d368b886b18fec3a571f096086a523717a3732d50db2b0b7998b4117ae66a761ccf1847a1616f4c07d5178d0d965f9feba351420f8bfb6f5ab9a0cb102568eabf3dfa4e22279f8082dce8143eb78235a1a54914ab71abb07f2f3648468370b9fbb071e074f1c030a4030225f40c39480339f3dc71d0f04f71326de1381674cc89e259e219927fae8ea2799a03da862a55afafe670957a2af3318d919d0a3358f3b891236d6a8e8d19999d1076b529968faefbd880d77bb300829dca87e9c8e4c28e0800ff37490a5bd8c36c0b0bdb2701a5d58d03378b9dbd384389e3ef0fd4003b08998fd3f32fe1a0810fc0eccaad94bca8dd83b34559c333f0b16dfc2896ed87b30ba14c81f87cd8b4bb6317db89b0e7e94c0616f9a665fba5b0e6fb3549c9d7b68e66d08a86eb2faec05cc462a771806b93cc38b0a4feb9935c6c8945da6a589891ba5ee99753cfdd38e1abc7147fd74b7c7d1ce0609b6680a2e18888d84949b6e6cf6a2aa4113535aaee079459e3f257b569a9450523c41f5b5ba4b79b3ba5949140a74bb048de0657d04954bdd71dae76f61e2a1f88aecb91cfa5b36c1bf3350a798dc4dcf48628effe3a0c5340c756bd922f78d0e36ef7df12ce78c179cc721ad087e15ea496bf5f60b21b5822d" + , "22fd225fb8f2c8d0d2097e1f8d38b6a9e619d39664dad3795f0336fda544d305e1be56a9953ecd6e4bec14b622f53da492eccbe257b9af84775a6bdf81aab6b1ba3492149b7ce4ea402c56e343939f78b9a9727193269798420c9323a9eb1fa63006e1482fcf15e3696aa1ae5cb66eb8aad4ac6041ca0b576b78785ad95bc5fb6f8a420ba9e1726552c0f2b97050ae69fc018d3f63b3a812a2d39ef64b8ae368472dec4341511b02cf77363c74f216055d5905412bc2bf57b4010b1bdce2882cb28fc7e4d470ed05219fc4d1ce11d11e10db369c807cd71891653638956b2f9d176e116188aff2fbb8519d9ad8c3fb52fa8bb4a0413364e89d9f9d7db627f2a2288babedcc314a19e45c6b7fb93b16a15d9953ff619ab4d523c481552588f711b450f854c858ebcb8cf49492d6240a753bde2109c486cac666bdba767c6e9254cc723f67855534c34d08ed486c67c1b8dd288c6010ce8e6c1c9b7f927acf4d71ee99729cee55ecec544245d0b51b31dd78eb99c2723e8ba5cf92ac7720e8933d9fd3596b90073f6980c5ed3f1cbfada26bdb6946e72391198e3d1cedebdba092324500e32b8e04e32550f6dd6c4befafa95acc206c5708300d2a8df2765751102f9738a1449c4c0d588f0076d0c1a5ee445eb85c97ada5837d5dc1d34ea081c8411d64a4d9ce9279bfe9feb25696cf741c705ed46f171e2239216d6c45dc8d15" + ] + ) + , + ( "SHAKE256_4096" + , HashAlg (SHAKE256 :: SHAKE256 4096) + , + [ "46b9dd2b0ba88d13233b3feb743eeb243fcd52ea62b81b82b50c27646ed5762fd75dc4ddd8c0f200cb05019d67b592f6fc821c49479ab48640292eacb3b7c4be141e96616fb13957692cc7edd0b45ae3dc07223c8e92937bef84bc0eab862853349ec75546f58fb7c2775c38462c5010d846c185c15111e595522a6bcd16cf86f3d122109e3b1fdd943b6aec468a2d621a7c06c6a957c62b54dafc3be87567d677231395f6147293b68ceab7a9e0c58d864e8efde4e1b9a46cbe854713672f5caaae314ed9083dab4b099f8e300f01b8650f1f4b1d8fcf3f3cb53fb8e9eb2ea203bdc970f50ae55428a91f7f53ac266b28419c3778a15fd248d339ede785fb7f5a1aaa96d313eacc890936c173cdcd0fab882c45755feb3aed96d477ff96390bf9a66d1368b208e21f7c10d04a3dbd4e360633e5db4b602601c14cea737db3dcf722632cc77851cbdde2aaf0a33a07b373445df490cc8fc1e4160ff118378f11f0477de055a81a9eda57a4a2cfb0c83929d310912f729ec6cfa36c6ac6a75837143045d791cc85eff5b21932f23861bcf23a52b5da67eaf7baae0f5fb1369db78f3ac45f8c4ac5671d85735cdddb09d2b1e34a1fc066ff4a162cb263d6541274ae2fcc865f618abe27c124cd8b074ccd516301b91875824d09958f341ef274bdab0bae316339894304e35877b0c28a9b1fd166c796b9cc258a064a8f57e27f2a" + , "2f671343d9b2e1604dc9dcf0753e5fe15c7c64a0d283cbbf722d411a0e36f6ca1d01d1369a23539cd80f7c054b6e5daf9c962cad5b8ed5bd11998b40d5734442bed798f6e5c915bd8bb07e0188d0a55c1290074f1c287af06352299184492cbdec9acba737ee292e5adaa445547355e72a03a3bac3aac770fe5d6b66600ff15d37d5b4789994ea2aeb097f550aa5e88e4d8ff0ba07b88c1c88573063f5d96df820abc2abd177ab037f351c375e553af917132cf2f563c79a619e1bb76e8e2266b0c5617d695f2c496a25f4073b6840c1833757ebb386f16757a8e16a21e9355e9b248f3b33be672da700266be99b8f8725e8ab06075f0219e655ebc188976364b7db139390d34a6ea67b4b223229183a94cf455ece91fdaf5b9c707fa4b40ec39816c1120c7aaaf47920977be900e6b9ca4b8940e192b927c475bd58e836f512ae3e52924e36ff8e9b1d0251047770a5e465905622b1f159be121ab93819c5e5c6dae299ac73bf1c4ed4a1e2c7fa3caa1039b05e94c9f993d04feb272b6e00bb0276939cf746c42936831fc8f2b4cb0cf94808ae0af405ce4bc67d1e7acfc6fd6590d3de91f795df5aaf57e2cee1845a303d0ea564be3f1299acdce67efe0d62cfc6d6829ff4ecc0a05153c24696c4d34c076453827e796f3062f94f62f4528b7cfc870f0dcd615b7c97b95da4b9be5830e8b3f66cce71e0f622c771994443e2" + , "fffcaac0606c0edb7bc0d15f033accb68538159016e5ae8470bf9ebea89fa6c9fcc3e027d94f7f967b7246346bd9f6b8084e45a057b976847c4db03bf383c834054866f6a8282a497368c46e1852fc09e20f22c45607a27c8b2a4798ebefada54f8d3795b9f07606b1cd6e41f90d765480ef5c0d5790659cf1d210adfd412378b92e1dd9bd7fd95a1a66677fc6baa0e3a53c9031c1fb59cbad9f5dc5881a3c8e25c80ecb1abf0971488ada1f533dcbf8d37031335378574b8d3fad61159c9fae28caa543b3072ce308d369be340e78c6edc664cc6dde9b2f0a4ad2e60ce9c8b1e5722b8d5b73d0962b74fb9ed86307a180f53933339f9d56d3b345c2a0e98fcf5de7754f3845f6be30089f0e142ad4602f18abdc750bda7c91c3f32872e66640db46045ab4c276b379f1b834c2cbb1bd8601305649ec6b3bf20618695136dee6541492d1d985ea1fb765fd7a559e810eba30f2f710233ae5a411b94ddcaa01a08f1c31320d111c0714422cd5e987c9a76fc865de34003ab12664081be8017d23d977f2bf4ed9e3ce09ea3d64bb4ae8ebfa9d0721f57841008c297e2f455a0441a2bd618ca379dbd239a21e410defb4001b1e11f87e36bf894c222f76f12ddcc3771bbb17d5c0dfd86d89a3e13e084f6dc1c4762bcd393c1757db7afb1434221569e7ddaaffd6318253ec3df8cf5f826b81896d6474ee06a2e30ccc8c6a96bdd5" + ] + ) + , + ( "Blake2b 160" + , HashAlg (Blake2b :: Blake2b 160) + , + [ "3345524abf6bbe1809449224b5972c41790b6cf2" + , "3c523ed102ab45a37d54f5610d5a983162fde84f" + , "a3d365b5fba5d36fbb19c03b7fde496058969c5a" + ] + ) + , + ( "Blake2b 224" + , HashAlg (Blake2b :: Blake2b 224) + , + [ "836cc68931c2e4e3e838602eca1902591d216837bafddfe6f0c8cb07" + , "477c3985751dd4d1b8c93827ea5310b33bb02a26463a050dffd3e857" + , "a4a1b6851be66891a3deff406c4d7556879ebf952407450755f90eb6" + ] + ) + , + ( "Blake2b 256" + , HashAlg (Blake2b :: Blake2b 256) + , + [ "0e5751c026e543b2e8ab2eb06099daa1d1e5df47778f7787faab45cdf12fe3a8" + , "01718cec35cd3d796dd00020e0bfecb473ad23457d063b75eff29c0ffa2e58a9" + , "036c13096926b3dfccfe3f233bd1b2f583b818b8b15c01be65af69238e900b2c" + ] + ) + , + ( "Blake2b 384" + , HashAlg (Blake2b :: Blake2b 384) + , + [ "b32811423377f52d7862286ee1a72ee540524380fda1724a6f25d7978c6fd3244a6caf0498812673c5e05ef583825100" + , "b7c81b228b6bd912930e8f0b5387989691c1cee1e65aade4da3b86a3c9f678fc8018f6ed9e2906720c8d2a3aeda9c03d" + , "927a1f297873cbe887a93b2183c4e2eba53966ba92c6db8b87029a1d8c673471d09740676cced79c5016838973f630c3" + ] + ) + , + ( "Blake2b 512" + , HashAlg (Blake2b :: Blake2b 512) + , + [ "786a02f742015903c6c6fd852552d272912f4740e15847618a86e217f71f5419d25e1031afee585313896444934eb04b903a685b1448b755d56f701afe9be2ce" + , "a8add4bdddfd93e4877d2746e62817b116364a1fa7bc148d95090bc7333b3673f82401cf7aa2e4cb1ecd90296e3f14cb5413f8ed77be73045b13914cdcd6a918" + , "af438eea5d8cdb209336a7e85bf58090dc21b49d823f89a7d064c119f127bd361af9c7d109edda0f0e91bdce078d1d86b8e6f25727c98f6d3bb6f50acb2dd376" + ] + ) + , + ( "Blake2s 160" + , HashAlg (Blake2s :: Blake2s 160) + , + [ "354c9c33f735962418bdacb9479873429c34916f" + , "5a604fec9713c369e84b0ed68daed7d7504ef240" + , "759bef6d041bcbd861b8b51baaece6c8fffd0acf" + ] + ) + , + ( "Blake2s 224" + , HashAlg (Blake2s :: Blake2s 224) + , + [ "1fa1291e65248b37b3433475b2a0dd63d54a11ecc4e3e034e7bc1ef4" + , "e4e5cb6c7cae41982b397bf7b7d2d9d1949823ae78435326e8db4912" + , "e220025fd46a9a635c3f7f60bb96a84c01019ac0817f5901e7eeaa2c" + ] + ) + , + ( "Blake2s 256" + , HashAlg (Blake2s :: Blake2s 256) + , + [ "69217a3079908094e11121d042354a7c1f55b6482ca1a51e1b250dfd1ed0eef9" + , "606beeec743ccbeff6cbcdf5d5302aa855c256c29b88c8ed331ea1a6bf3c8812" + , "94662583a600a12dff357c0a6f1b514a710ef0f587a38e8d2e4d7f67e9c81667" + ] + ) + ] + +runhash :: HashAlg -> ByteString -> ByteString +runhash (HashAlg hashAlg) v = B.convertToBase B.Base16 $ hashWith hashAlg $ v + +runhashinc :: HashAlg -> [ByteString] -> ByteString +runhashinc (HashAlg hashAlg) v = B.convertToBase B.Base16 $ hashinc $ v + where + hashinc = hashFinalize . foldl hashUpdate (hashInitWith hashAlg) + +-- | Messages that take more than one block, which none of the vectors above +-- do: the longest of them is 43 bytes and a block is 64, so a compression +-- function that is wrong only in how one block carries into the next -- which +-- is what the paths written for a processor's hashing instructions can get +-- wrong -- passes every KAT above. Two of these cross a block boundary and +-- the third is the million letters FIPS 180-4 uses; the digests are what +-- FIPS 180-4 and openssl give. +longVectors :: [ByteString] +longVectors = + [ "abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq" + , "abcdefghbcdefghicdefghijdefghijkefghijklfghijklmghijklmnhijklmno" + , B.replicate 1000000 0x61 + ] + +expectedLong :: [(String, HashAlg, [ByteString])] +expectedLong = + [ + ( "MD5" + , HashAlg MD5 + , + [ "8215ef0796a20bcaaae116d3876c664a" + , "2782e38354c31d1b1d6dfb6f4ccb2d2e" + , "7707d6ae4e027c70eea2a935c2296f21" + ] + ) + , + ( "SHA1" + , HashAlg SHA1 + , + [ "84983e441c3bd26ebaae4aa1f95129e5e54670f1" + , "b85d6468bd3a73794bceaf812239cc1fe460ab95" + , "34aa973cd4c4daa4f61eeb2bdbad27316534016f" + ] + ) + , + ( "SHA224" + , HashAlg SHA224 + , + [ "75388b16512776cc5dba5da1fd890150b0c6455cb4f58b1952522525" + , "4176f330539b0ed8b0b6b5dea7c8e47a18fc4daf3f53920355b0926a" + , "20794655980c91d8bbb4c1ea97618a4bf03f42581948b2ee4ee7ad67" + ] + ) + , + ( "SHA256" + , HashAlg SHA256 + , + [ "248d6a61d20638b8e5c026930c3e6039a33ce45964ff2167f6ecedd419db06c1" + , "2ff100b36c386c65a1afc462ad53e25479bec9498ed00aa5a04de584bc25301b" + , "cdc76e5c9914fb9281a1c7e284d73e67f1809a48a497200e046d39ccc7112cd0" + ] + ) + , + ( "SHA384" + , HashAlg SHA384 + , + [ "3391fdddfc8dc7393707a65b1b4709397cf8b1d162af05abfe8f450de5f36bc6b0455a8520bc4e6f5fe95b1fe3c8452b" + , "bdc0f4a6e0d7de88f374e6c2562441d856aeabed3f52553103f55eca811f64b422c7cb47a8067f123e45c1a8ee303635" + , "9d0e1809716474cb086e834e310a4a1ced149e9c00f248527972cec5704c2a5b07b8b3dc38ecc4ebae97ddd87f3d8985" + ] + ) + , + ( "SHA512" + , HashAlg SHA512 + , + [ "204a8fc6dda82f0a0ced7beb8e08a41657c16ef468b228a8279be331a703c33596fd15c13b1b07f9aa1d3bea57789ca031ad85c7a71dd70354ec631238ca3445" + , "90d1bdb9a6cbf9cb0d4a7f185ee0870456f440b81f13f514f4561a08112763523033245875b68209bb1f5d5215bac81e0d69f77374cc44d1be30f58c8b615141" + , "e718483d0ce769644e2e42c7bc15b4638e1f98b13b2044285632a803afa973ebde0ff244877ea60a4cb0432ce577c31beb009c5c2c49aa2e4eadb217ad8cc09b" + ] + ) + ] + +data HashPrefixAlg = forall alg. HashAlgorithmPrefix alg => HashPrefixAlg alg + +expectedPrefix :: [(String, HashPrefixAlg)] +expectedPrefix = + [ ("MD5", HashPrefixAlg MD5) + , ("SHA1", HashPrefixAlg SHA1) + , ("SHA224", HashPrefixAlg SHA224) + , ("SHA256", HashPrefixAlg SHA256) + , ("SHA384", HashPrefixAlg SHA384) + , ("SHA512", HashPrefixAlg SHA512) + ] + +runhashpfx :: HashPrefixAlg -> ByteString -> ByteString +runhashpfx (HashPrefixAlg hashAlg) v = B.convertToBase B.Base16 $ hashWith hashAlg v + +runhashpfxpfx :: HashPrefixAlg -> ByteString -> Int -> ByteString +runhashpfxpfx (HashPrefixAlg hashAlg) v len = B.convertToBase B.Base16 $ hashPrefixWith hashAlg v len + +makeTestAlg = makeTestAlgWith vectors + +makeTestAlgWith vs (name, hashAlg, results) = + describe name $ mapM_ maketest (zip3 is vs results) + where + is :: [Int] + is = [1 ..] + + maketest (i, v, r) = do + it (show i) (runhash hashAlg v `shouldBe` r) + +makeTestChunk (hashName, hashAlg, _) = do + prop hashName $ \ckLen (ArbitraryBS0_2901 inp) -> + runhash hashAlg inp `propertyEq` runhashinc hashAlg (chunkS ckLen inp) + +makeTestPrefix (hashName, hashAlg) = do + prop hashName $ \(ArbitraryBS0_2901 inp) (Int0_2901 len) -> + runhashpfx hashAlg (B.take len inp) `propertyEq` runhashpfxpfx hashAlg inp len + +makeTestHybrid (hashName, HashPrefixAlg alg) = do + prop hashName $ \(ArbitraryBS0_2901 start) (ArbitraryBS0_2901 end) -> do + len <- choose (0, B.length end) + let ref = hashWith alg (start `B.append` B.take len end) + hyb = hashFinalizePrefix (hashUpdate (hashInitWith alg) start) end len + return (ref `propertyEq` hyb) + +-- SHAKE128 truncation example with expected byte at final position +-- +shake128TruncationBytes = [0x01, 0x03, 0x07, 0x0f, 0x0f, 0x2f, 0x6f, 0x6f] + +makeTestSHAKE128Truncation i byte = + it (show i) $ xof (4088 + i) `shouldBe` xof 4088 `B.snoc` byte + where + hashEmpty :: KnownNat n => proxy n -> Digest (SHAKE128 n) + hashEmpty _ = hash B.empty + + xof n = case someNatVal n of + Nothing -> error ("invalid Nat: " ++ show n) + Just (SomeNat p) -> convert (hashEmpty p) + +spec :: Spec +spec = do + describe "KATs" $ mapM_ makeTestAlg expected + describe "KATs over several blocks" $ + mapM_ (makeTestAlgWith longVectors) expectedLong + describe "Chunking" $ mapM_ makeTestChunk expected + describe "Prefix" $ mapM_ makeTestPrefix expectedPrefix + describe "Hybrid" $ mapM_ makeTestHybrid expectedPrefix + describe "Truncating" $ do + describe "SHAKE128" $ + sequence_ $ + (zipWith makeTestSHAKE128Truncation [1 ..] shake128TruncationBytes) diff --git a/tests/Imports.hs b/tests/Imports.hs --- a/tests/Imports.hs +++ b/tests/Imports.hs @@ -5,6 +5,9 @@ Word64, ByteString, + -- * Test vectors + firstVector, + -- * Modules module X, ) where @@ -18,7 +21,15 @@ import Data.Foldable as X (foldl') import Data.Monoid as X -import Test.Tasty as X -import Test.Tasty.HUnit as X -import Test.Tasty.QuickCheck as X hiding (vector) +import Test.Hspec as X +import Test.Hspec.QuickCheck as X (modifyMaxSuccess, prop) +import Test.QuickCheck as X hiding (vector) import Utils as X + +-- | The first of a list of test vectors. The lists these are taken from are +-- literals in the modules that hold them and are never empty, so this says so +-- once, with a name and a message, rather than leaving a partial 'head' at +-- every use. +firstVector :: [a] -> a +firstVector (v : _) = v +firstVector [] = error "firstVector: the vector list is empty" diff --git a/tests/KAT_AES.hs b/tests/KAT_AES.hs deleted file mode 100644 --- a/tests/KAT_AES.hs +++ /dev/null @@ -1,174 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AES (tests) where - -import BlockCipher -import qualified Crypto.Cipher.AES as AES -import Crypto.Cipher.Types -import Crypto.Error -import qualified Data.ByteArray as BA -import qualified Data.ByteString as B -import Data.Maybe -import Imports - -import qualified KAT_AES.KATCBC as KATCBC -import qualified KAT_AES.KATCCM as KATCCM -import qualified KAT_AES.KATECB as KATECB -import qualified KAT_AES.KATGCM as KATGCM -import qualified KAT_AES.KATOCB3 as KATOCB3 -import qualified KAT_AES.KATXTS as KATXTS - -{- -instance Show AES.AES where - show _ = "AES" -instance Arbitrary AES.AESIV where - arbitrary = AES.aesIV_ . B.pack <$> replicateM 16 arbitrary -instance Arbitrary AES.AES where - arbitrary = AES.initAES . B.pack <$> replicateM 16 arbitrary --} - -toKatECB (k, p, c) = KAT_ECB{ecbKey = k, ecbPlaintext = p, ecbCiphertext = c} -toKatCBC (k, iv, p, c) = KAT_CBC{cbcKey = k, cbcIV = iv, cbcPlaintext = p, cbcCiphertext = c} -toKatXTS (k1, k2, iv, p, _, c) = - KAT_XTS - { xtsKey1 = k1 - , xtsKey2 = k2 - , xtsIV = iv - , xtsPlaintext = p - , xtsCiphertext = c - } -toKatAEAD mode (k, iv, h, p, c, taglen, tag) = - KAT_AEAD - { aeadMode = mode - , aeadKey = k - , aeadIV = iv - , aeadHeader = h - , aeadPlaintext = p - , aeadCiphertext = c - , aeadTaglen = taglen - , aeadTag = tag - } -toKatGCM = toKatAEAD AEAD_GCM -toKatOCB = toKatAEAD AEAD_OCB - -toKatCCM (k, iv, h, i, o, m) = - KAT_AEAD - { aeadMode = AEAD_CCM (B.length i) (ccmMVal m) CCM_L2 - , aeadKey = k - , aeadIV = iv - , aeadHeader = h - , aeadPlaintext = i - , aeadCiphertext = ct - , aeadTaglen = m - , aeadTag = at - } - where - ccmMVal x = - fromMaybe (error $ "unsupported CCM tag length: " ++ show x) $ - lookup - x - [ (4, CCM_M4) - , (6, CCM_M6) - , (8, CCM_M8) - , (10, CCM_M10) - , (12, CCM_M12) - , (14, CCM_M14) - , (16, CCM_M16) - ] - ctWithTag = B.drop (B.length h) o - (ct, at) = B.splitAt (B.length ctWithTag - m) ctWithTag - -kats128 = - defaultKATs - { kat_ECB = map toKatECB KATECB.vectors_aes128_enc - , kat_CBC = map toKatCBC KATCBC.vectors_aes128_enc - , kat_CFB = - [ KAT_CFB - { cfbKey = - "\x2b\x7e\x15\x16\x28\xae\xd2\xa6\xab\xf7\x15\x88\x09\xcf\x4f\x3c" - , cfbIV = - "\xC8\xA6\x45\x37\xA0\xB3\xA9\x3F\xCD\xE3\xCD\xAD\x9F\x1C\xE5\x8B" - , cfbPlaintext = - "\x30\xc8\x1c\x46\xa3\x5c\xe4\x11\xe5\xfb\xc1\x19\x1a\x0a\x52\xef" - , cfbCiphertext = - "\x26\x75\x1f\x67\xa3\xcb\xb1\x40\xb1\x80\x8c\xf1\x87\xa4\xf4\xdf" - } - ] - , kat_XTS = map toKatXTS KATXTS.vectors_aes128_enc - , kat_AEAD = - map toKatGCM KATGCM.vectors_aes128_enc - ++ map toKatOCB KATOCB3.vectors_aes128_enc - ++ map toKatCCM KATCCM.vectors_aes128_enc - } - -kats192 = - defaultKATs - { kat_ECB = map toKatECB KATECB.vectors_aes192_enc - , kat_CBC = map toKatCBC KATCBC.vectors_aes192_enc - } - -kats256 = - defaultKATs - { kat_ECB = map toKatECB KATECB.vectors_aes256_enc - , kat_CBC = map toKatCBC KATCBC.vectors_aes256_enc - , kat_XTS = map toKatXTS KATXTS.vectors_aes256_enc - , kat_AEAD = map toKatGCM KATGCM.vectors_aes256_enc - } - --- SP 800-38D 5.2.1.1: 1 <= len(IV) <= 2^64 - 1. A zero-length IV makes --- J0 the GHASH of the empty string, which leaks the authentication key. -aeadIVLengthTests :: TestTree -aeadIVLengthTests = - testGroup - "AEAD IV length" - [ testCase "96-bit IV accepted" $ - True @=? isRight (initWith (B.replicate 12 0)) - , testCase "8-bit IV accepted" $ - True @=? isRight (initWith (B.replicate 1 0)) - , testCase "empty IV rejected" $ - Left CryptoError_IvSizeInvalid @=? initWith B.empty - ] - where - ctx = throwCryptoError (cipherInit (B.replicate 16 0)) :: AES.AES128 - initWith iv = - eitherCryptoError (() <$ aeadInit AEAD_GCM ctx (iv :: ByteString)) - isRight = either (const False) (const True) - -aeadTagLengthTests :: TestTree -aeadTagLengthTests = - testGroup - "AEAD tag length" - [ testCase "full tag verifies" $ Just message @=? openWith fullTag - , testCase "empty tag rejected" $ Nothing @=? openWith B.empty - , testCase "1-byte tag rejected" $ Nothing @=? openWith (B.take 1 fullTag) - , testCase "3-byte tag rejected" $ Nothing @=? openWith (B.take 3 fullTag) - , testCase "wrong tag rejected" $ - Nothing @=? openWith (B.map (+ 1) fullTag) - ] - where - key = B.replicate 16 0 - iv = B.replicate 12 0 - aad = "additional data" :: ByteString - message = "authenticated message" :: ByteString - ctx = throwCryptoError (cipherInit key) :: AES.AES128 - aead = throwCryptoError (aeadInit AEAD_GCM ctx iv) - (AuthTag tag, ciphertext) = aeadSimpleEncrypt aead aad message 16 - fullTag = BA.convert tag :: ByteString - openWith t = aeadSimpleDecrypt aead aad ciphertext (AuthTag (BA.convert t)) - -tests = - testGroup - "AES" - [ testBlockCipher kats128 (undefined :: AES.AES128) - , testBlockCipher kats192 (undefined :: AES.AES192) - , testBlockCipher kats256 (undefined :: AES.AES256) - , aeadIVLengthTests - , aeadTagLengthTests - {- - , testProperty "genCtr" $ \(key, iv1) -> - let (bs1, iv2) = AES.genCounter key iv1 32 - (bs2, iv3) = AES.genCounter key iv2 32 - (bsAll, iv3') = AES.genCounter key iv1 64 - in (B.concat [bs1,bs2] == bsAll && iv3 == iv3') - -} - ] diff --git a/tests/KAT_AES/KATCBC.hs b/tests/KAT_AES/KATCBC.hs deleted file mode 100644 --- a/tests/KAT_AES/KATCBC.hs +++ /dev/null @@ -1,460 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AES.KATCBC where - -import qualified Data.ByteString as B -import Data.ByteString.Char8 () - -type KATCBC = (B.ByteString, B.ByteString, B.ByteString, B.ByteString) - -vectors_aes128_enc - , vectors_aes128_dec - , vectors_aes192_enc - , vectors_aes192_dec - , vectors_aes256_enc - , vectors_aes256_dec - :: [KATCBC] -vectors_aes128_enc = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x66\xe9\x4b\xd4\xef\x8a\x2c\x3b\x88\x4c\xfa\x59\xca\x34\x2b\x2e" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xb6\xae\xaf\xfa\x75\x2d\xc0\x8b\x51\x63\x97\x31\x76\x1a\xed\x00" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xcb\x64\xcf\x3f\x42\x2a\xe8\x4b\xb9\x0e\x3a\xb4\xdb\xa7\xbd\x86" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xe5\xb5\x07\x7f\x93\x46\x46\x2c\x62\xa0\x75\xc0\xc7\x08\xee\x96" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xe1\x4d\x5d\x0e\xe2\x77\x15\xdf\x08\xb4\x15\x2b\xa2\x3d\xa8\xe0" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x5e\x77\xe5\x9f\x8f\x85\x94\x34\x89\xa2\x41\x49\xc7\x5f\x4e\xc9" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x8f\x42\xc2\x4b\xee\x6e\x63\x47\x2b\x16\x5a\xa9\x41\x31\x2f\x7c" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xb0\xea\x4a\xc0\xd2\x5c\xcd\x7c\x82\xcb\x8a\x30\x68\xc6\xfe\x2e" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\xe1\x4d\x5d\x0e\xe2\x77\x15\xdf\x08\xb4\x15\x2b\xa2\x3d\xa8\xe0" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x17\xd6\x14\xf3\x79\xa9\x35\x90\x77\xe9\x55\x77\xfd\x31\xc2\x0a" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x8f\x42\xc2\x4b\xee\x6e\x63\x47\x2b\x16\x5a\xa9\x41\x31\x2f\x7c" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\xe5\xb5\x07\x7f\x93\x46\x46\x2c\x62\xa0\x75\xc0\xc7\x08\xee\x96" - ) - ] -vectors_aes192_enc = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xaa\xe0\x69\x92\xac\xbf\x52\xa3\xe8\xf4\xa9\x6e\xc9\x30\x0b\xd7" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x5f\x88\xef\x3f\xbd\xeb\xf2\xe4\xe2\x66\x65\x12\xd3\xbc\xb7\x0f" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xdb\x42\xf5\x1c\xd2\x0e\xca\xd2\x9e\xb0\x13\x2b\x0f\xaa\x4b\x85" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xda\xb4\x01\x5f\x98\x70\x25\xeb\xb8\xa8\x5f\x3c\x7f\x73\x70\x19" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xcf\x1e\xce\x3c\x44\xb0\x78\xfb\x27\xcb\x0a\x3e\x07\x1b\x08\x20" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x98\xb8\x95\xa1\x45\xca\x4e\x0b\xf8\x3e\x69\x32\x81\xc1\xa0\x97" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xf2\xf0\xae\xd8\xcd\xc9\x21\xca\x4b\x55\x84\x5d\xa4\x15\x21\xc2" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x5e\xea\x4b\x13\xdd\xd9\x17\x12\xb0\x14\xe2\x82\x2d\x18\x76\xfb" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\xcf\x1e\xce\x3c\x44\xb0\x78\xfb\x27\xcb\x0a\x3e\x07\x1b\x08\x20" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\xeb\x8c\x17\x30\x90\xc7\x5b\x77\xd6\x72\xb4\x57\xa7\x78\xd9\xd0" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\xf2\xf0\xae\xd8\xcd\xc9\x21\xca\x4b\x55\x84\x5d\xa4\x15\x21\xc2" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\xda\xb4\x01\x5f\x98\x70\x25\xeb\xb8\xa8\x5f\x3c\x7f\x73\x70\x19" - ) - ] -vectors_aes256_enc = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xdc\x95\xc0\x78\xa2\x40\x89\x89\xad\x48\xa2\x14\x92\x84\x20\x87" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x72\x98\xca\xa5\x65\x03\x1e\xad\xc6\xce\x23\xd2\x3e\xa6\x63\x78" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xf4\x35\xa1\x11\xa3\xe4\xa1\x94\x49\x19\xf9\x12\xc5\xa2\x41\xde" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x91\xc0\x87\x62\x87\x6d\xcc\xf9\xba\x20\x4a\x33\x76\x8f\xa5\xfe" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x7b\xc3\x02\x6c\xd7\x37\x10\x3e\x62\x90\x2b\xcd\x18\xfb\x01\x63" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x9c\xac\x94\xc6\xb4\x85\x61\xf8\xff\xaa\xa7\x86\x16\xba\x48\x92" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\xf9\xc7\x44\x4b\xb0\xcc\x80\x6c\x7c\x39\xee\x22\x11\xf1\x46" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x6d\xed\xd0\xa3\xe6\x94\xa0\xde\x65\x1d\x68\xa6\xb5\x5a\x64\xa2" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x7b\xc3\x02\x6c\xd7\x37\x10\x3e\x62\x90\x2b\xcd\x18\xfb\x01\x63" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x62\xae\x12\xf3\x24\xbf\xea\x08\xd5\xf6\x75\xb5\x13\x02\x6b\xbf" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\xf9\xc7\x44\x4b\xb0\xcc\x80\x6c\x7c\x39\xee\x22\x11\xf1\x46" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x91\xc0\x87\x62\x87\x6d\xcc\xf9\xba\x20\x4a\x33\x76\x8f\xa5\xfe" - ) - ] -vectors_aes128_dec = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x14\x0f\x0f\x10\x11\xb5\x22\x3d\x79\x58\x77\x17\xff\xd9\xec\x3a" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x33\x08\x32\x40\xd6\x5c\xbc\x72\xaa\x0b\x44\xf3\xe1\x9e\xa9\x5a" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x65\x0a\x42\xa0\x3c\x4b\x93\xa4\xb7\x43\xdc\x9e\x9c\xf4\xc0\x9b" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x80\xcd\x20\xe1\xbd\x89\x3c\x5e\xe4\x20\x76\x85\xb0\x9a\x0e\x3e" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x15\x0e\x0e\x11\x10\xb4\x23\x3c\x78\x59\x76\x16\xfe\xd8\xed\x3b" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x32\x09\x33\x41\xd7\x5d\xbd\x73\xab\x0a\x45\xf2\xe0\x9f\xa8\x5b" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x64\x0b\x43\xa1\x3d\x4a\x92\xa5\xb6\x42\xdd\x9f\x9d\xf5\xc1\x9a" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x81\xcc\x21\xe0\xbc\x88\x3d\x5f\xe5\x21\x77\x84\xb1\x9b\x0f\x3f" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\xf5\x06\x41\x7e\x6a\x8f\xbc\x32\xdd\xa5\x52\x73\xbf\x9f\x4d\x5c" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\xbf\x6d\x28\xac\x20\xc9\x1d\x65\xa9\xd4\xb0\x96\xc2\xd5\xa5\x09" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x5f\x2a\x46\xab\x8d\xb9\x5b\x22\x15\xfe\x1a\xa4\xdd\x69\x59\x26" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x71\x9b\x21\xb5\x39\x7c\x2f\x16\x7c\x8b\x45\x22\xb5\x20\xec\x2e" - ) - ] -vectors_aes192_dec = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x13\x46\x0e\x87\xa8\xfc\x02\x3e\xf2\x50\x1a\xfe\x7f\xf5\x1c\x51" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x91\x75\x27\xfc\xd4\xa0\x6f\x32\x27\x29\x90\x14\xca\xde\xd4\x1a" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x29\x64\x80\xb6\xa5\xd6\xcf\xb3\x78\x3f\x21\x6b\x80\x31\x3d\xb3" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xbc\xa5\x06\x07\xd0\x67\x30\x85\x2d\x3a\x50\x4b\x68\x0a\x19\xcc" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x12\x47\x0f\x86\xa9\xfd\x03\x3f\xf3\x51\x1b\xff\x7e\xf4\x1d\x50" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x90\x74\x26\xfd\xd5\xa1\x6e\x33\x26\x28\x91\x15\xcb\xdf\xd5\x1b" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x28\x65\x81\xb7\xa4\xd7\xce\xb2\x79\x3e\x20\x6a\x81\x30\x3c\xb2" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xbd\xa4\x07\x06\xd1\x66\x31\x84\x2c\x3b\x51\x4a\x69\x0b\x18\xcd" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x38\xf9\xf9\xd1\x7e\x2c\x82\xaf\xdc\xed\x68\x03\xb6\x31\x46\x3e" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x35\x4e\xc1\x01\x0f\x17\x50\x5e\x63\x37\x40\x4b\x9a\xf2\xc0\x5c" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\xa7\x7c\xc9\xd1\x4f\x44\xf7\xf7\xcc\x45\x80\x83\x19\xb7\xa4\x71" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\xf9\x1d\xb1\x13\x0b\xd1\xc0\x66\x9f\xfa\xc2\x0e\xbe\xdd\xcb\xca" - ) - ] -vectors_aes256_dec = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x67\x67\x1c\xe1\xfa\x91\xdd\xeb\x0f\x8f\xbb\xb3\x66\xb5\x31\xb4" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x7b\xd3\xfb\x90\x65\x56\x9f\x39\x8b\x09\xcb\x93\x4b\x1e\x01\x23" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xda\xa8\xbf\x5c\xde\x2e\x52\x45\x5f\xa3\xb3\xfe\x33\x32\x47\xca" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x83\x24\xdc\xb4\x30\x12\x73\x6c\xed\x58\xab\x8f\x4b\x05\xca\x0b" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x66\x66\x1d\xe0\xfb\x90\xdc\xea\x0e\x8e\xba\xb2\x67\xb4\x30\xb5" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x7a\xd2\xfa\x91\x64\x57\x9e\x38\x8a\x08\xca\x92\x4a\x1f\x00\x22" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\xdb\xa9\xbe\x5d\xdf\x2f\x53\x44\x5e\xa2\xb2\xff\x32\x33\x46\xcb" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x82\x25\xdd\xb5\x31\x13\x72\x6d\xec\x59\xaa\x8e\x4a\x04\xcb\x0a" - ) - , - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x68\xe9\x07\x16\xe3\x66\x1b\x1d\xb1\x89\x74\xb0\x9c\x46\x47\xe4" - ) - , - ( "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , "\x7f\xb9\xeb\xa4\xd3\x5f\x70\x40\xab\x52\xec\xd2\x3b\x48\xb7\x6e" - ) - , - ( "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x6c\x58\x0f\x41\x82\x36\xbc\xff\x64\x1d\xac\xa7\x3e\x34\x11\x18" - ) - , - ( "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03\x03" - , "\x3f\x62\xd6\x8c\xb1\xf7\x62\x28\xa4\xc3\x82\x4f\x8b\x24\xe7\x4b" - ) - ] diff --git a/tests/KAT_AES/KATCCM.hs b/tests/KAT_AES/KATCCM.hs deleted file mode 100644 --- a/tests/KAT_AES/KATCCM.hs +++ /dev/null @@ -1,205 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AES.KATCCM where - -import qualified Data.ByteString as B - --- (key, iv, header, in, out+atag, taglen) -type KATCCM = - (B.ByteString, B.ByteString, B.ByteString, B.ByteString, B.ByteString, Int) - -vectors_aes128_enc :: [KATCCM] -vectors_aes128_enc = - [ - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x03\x02\x01\x00\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" - , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x58\x8c\x97\x9a\x61\xc6\x63\xd2\xf0\x66\xd0\xc2\xc0\xf9\x89\x80\x6d\x5f\x6b\x61\xda\xc3\x84\x17\xe8\xd1\x2c\xfd\xf9\x26\xe0" - , {- M = -} 8 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x04\x03\x02\x01\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" - , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x72\xc9\x1a\x36\xe1\x35\xf8\xcf\x29\x1c\xa8\x94\x08\x5c\x87\xe3\xcc\x15\xc4\x39\xc9\xe4\x3a\x3b\xa0\x91\xd5\x6e\x10\x40\x09\x16" - , {- M = -} 8 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x05\x04\x03\x02\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" - , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x51\xb1\xe5\xf4\x4a\x19\x7d\x1d\xa4\x6b\x0f\x8e\x2d\x28\x2a\xe8\x71\xe8\x38\xbb\x64\xda\x85\x96\x57\x4a\xda\xa7\x6f\xbd\x9f\xb0\xc5" - , {- M = -} 8 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x06\x05\x04\x03\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" - , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\xa2\x8c\x68\x65\x93\x9a\x9a\x79\xfa\xaa\x5c\x4c\x2a\x9d\x4a\x91\xcd\xac\x8c\x96\xc8\x61\xb9\xc9\xe6\x1e\xf1" - , {- M = -} 8 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x07\x06\x05\x04\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" - , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\xdc\xf1\xfb\x7b\x5d\x9e\x23\xfb\x9d\x4e\x13\x12\x53\x65\x8a\xd8\x6e\xbd\xca\x3e\x51\xe8\x3f\x07\x7d\x9c\x2d\x93" - , {- M = -} 8 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x08\x07\x06\x05\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" - , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x6f\xc1\xb0\x11\xf0\x06\x56\x8b\x51\x71\xa4\x2d\x95\x3d\x46\x9b\x25\x70\xa4\xbd\x87\x40\x5a\x04\x43\xac\x91\xcb\x94" - , {- M = -} 8 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x09\x08\x07\x06\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" - , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x01\x35\xd1\xb2\xc9\x5f\x41\xd5\xd1\xd4\xfe\xc1\x85\xd1\x66\xb8\x09\x4e\x99\x9d\xfe\xd9\x6c\x04\x8c\x56\x60\x2c\x97\xac\xbb\x74\x90" - , {- M = -} 10 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x0a\x09\x08\x07\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" - , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x7b\x75\x39\x9a\xc0\x83\x1d\xd2\xf0\xbb\xd7\x58\x79\xa2\xfd\x8f\x6c\xae\x6b\x6c\xd9\xb7\xdb\x24\xc1\x7b\x44\x33\xf4\x34\x96\x3f\x34\xb4" - , {- M = -} 10 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x0b\x0a\x09\x08\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07" - , {- in = -} "\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x82\x53\x1a\x60\xcc\x24\x94\x5a\x4b\x82\x79\x18\x1a\xb5\xc8\x4d\xf2\x1c\xe7\xf9\xb7\x3f\x42\xe1\x97\xea\x9c\x07\xe5\x6b\x5e\xb1\x7e\x5f\x4e" - , {- M = -} 10 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x0c\x0b\x0a\x09\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" - , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x07\x34\x25\x94\x15\x77\x85\x15\x2b\x07\x40\x98\x33\x0a\xbb\x14\x1b\x94\x7b\x56\x6a\xa9\x40\x6b\x4d\x99\x99\x88\xdd" - , {- M = -} 10 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x0d\x0c\x0b\x0a\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" - , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x67\x6b\xb2\x03\x80\xb0\xe3\x01\xe8\xab\x79\x59\x0a\x39\x6d\xa7\x8b\x83\x49\x34\xf5\x3a\xa2\xe9\x10\x7a\x8b\x6c\x02\x2c" - , {- M = -} 10 - ) - , - ( {- key = -} "\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf" - , {- iv = -} "\x00\x00\x00\x0e\x0d\x0c\x0b\xa0\xa1\xa2\xa3\xa4\xa5" - , {- hdr = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" - , {- in = -} "\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20" - , {- out = -} "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\xc0\xff\xa0\xd6\xf0\x5b\xdb\x67\xf2\x4d\x43\xa4\x33\x8d\x2a\xa4\xbe\xd7\xb2\x0e\x43\xcd\x1a\xa3\x16\x62\xe7\xad\x65\xd6\xdb" - , {- M = -} 10 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x41\x2b\x4e\xa9\xcd\xbe\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x0b\xe1\xa8\x8b\xac\xe0\x18\xb1" - , {- in = -} "\x08\xe8\xcf\x97\xd8\x20\xea\x25\x84\x60\xe9\x6a\xd9\xcf\x52\x89\x05\x4d\x89\x5c\xea\xc4\x7c" - , {- out = -} "\x0b\xe1\xa8\x8b\xac\xe0\x18\xb1\x4c\xb9\x7f\x86\xa2\xa4\x68\x9a\x87\x79\x47\xab\x80\x91\xef\x53\x86\xa6\xff\xbd\xd0\x80\xf8\xe7\x8c\xf7\xcb\x0c\xdd\xd7\xb3" - , {- M = -} 8 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x33\x56\x8e\xf7\xb2\x63\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x63\x01\x8f\x76\xdc\x8a\x1b\xcb" - , {- in = -} "\x90\x20\xea\x6f\x91\xbd\xd8\x5a\xfa\x00\x39\xba\x4b\xaf\xf9\xbf\xb7\x9c\x70\x28\x94\x9c\xd0\xec" - , {- out = -} "\x63\x01\x8f\x76\xdc\x8a\x1b\xcb\x4c\xcb\x1e\x7c\xa9\x81\xbe\xfa\xa0\x72\x6c\x55\xd3\x78\x06\x12\x98\xc8\x5c\x92\x81\x4a\xbc\x33\xc5\x2e\xe8\x1d\x7d\x77\xc0\x8a" - , {- M = -} 8 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x10\x3f\xe4\x13\x36\x71\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\xaa\x6c\xfa\x36\xca\xe8\x6b\x40" - , {- in = -} "\xb9\x16\xe0\xea\xcc\x1c\x00\xd7\xdc\xec\x68\xec\x0b\x3b\xbb\x1a\x02\xde\x8a\x2d\x1a\xa3\x46\x13\x2e" - , {- out = -} "\xaa\x6c\xfa\x36\xca\xe8\x6b\x40\xb1\xd2\x3a\x22\x20\xdd\xc0\xac\x90\x0d\x9a\xa0\x3c\x61\xfc\xf4\xa5\x59\xa4\x41\x77\x67\x08\x97\x08\xa7\x76\x79\x6e\xdb\x72\x35\x06" - , {- M = -} 8 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x76\x4c\x63\xb8\x05\x8e\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\xd0\xd0\x73\x5c\x53\x1e\x1b\xec\xf0\x49\xc2\x44" - , {- in = -} "\x12\xda\xac\x56\x30\xef\xa5\x39\x6f\x77\x0c\xe1\xa6\x6b\x21\xf7\xb2\x10\x1c" - , {- out = -} "\xd0\xd0\x73\x5c\x53\x1e\x1b\xec\xf0\x49\xc2\x44\x14\xd2\x53\xc3\x96\x7b\x70\x60\x9b\x7c\xbb\x7c\x49\x91\x60\x28\x32\x45\x26\x9a\x6f\x49\x97\x5b\xca\xde\xaf" - , {- M = -} 8 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\xf8\xb6\x78\x09\x4e\x3b\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x77\xb6\x0f\x01\x1c\x03\xe1\x52\x58\x99\xbc\xae" - , {- in = -} "\xe8\x8b\x6a\x46\xc7\x8d\x63\xe5\x2e\xb8\xc5\x46\xef\xb5\xde\x6f\x75\xe9\xcc\x0d" - , {- out = -} "\x77\xb6\x0f\x01\x1c\x03\xe1\x52\x58\x99\xbc\xae\x55\x45\xff\x1a\x08\x5e\xe2\xef\xbf\x52\xb2\xe0\x4b\xee\x1e\x23\x36\xc7\x3e\x3f\x76\x2c\x0c\x77\x44\xfe\x7e\x3c" - , {- M = -} 8 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\xd5\x60\x91\x2d\x3f\x70\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\xcd\x90\x44\xd2\xb7\x1f\xdb\x81\x20\xea\x60\xc0" - , {- in = -} "\x64\x35\xac\xba\xfb\x11\xa8\x2e\x2f\x07\x1d\x7c\xa4\xa5\xeb\xd9\x3a\x80\x3b\xa8\x7f" - , {- out = -} "\xcd\x90\x44\xd2\xb7\x1f\xdb\x81\x20\xea\x60\xc0\x00\x97\x69\xec\xab\xdf\x48\x62\x55\x94\xc5\x92\x51\xe6\x03\x57\x22\x67\x5e\x04\xc8\x47\x09\x9e\x5a\xe0\x70\x45\x51" - , {- M = -} 8 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x42\xff\xf8\xf1\x95\x1c\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\xd8\x5b\xc7\xe6\x9f\x94\x4f\xb8" - , {- in = -} "\x8a\x19\xb9\x50\xbc\xf7\x1a\x01\x8e\x5e\x67\x01\xc9\x17\x87\x65\x98\x09\xd6\x7d\xbe\xdd\x18" - , {- out = -} "\xd8\x5b\xc7\xe6\x9f\x94\x4f\xb8\xbc\x21\x8d\xaa\x94\x74\x27\xb6\xdb\x38\x6a\x99\xac\x1a\xef\x23\xad\xe0\xb5\x29\x39\xcb\x6a\x63\x7c\xf9\xbe\xc2\x40\x88\x97\xc6\xba" - , {- M = -} 10 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x92\x0f\x40\xe5\x6c\xdc\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x74\xa0\xeb\xc9\x06\x9f\x5b\x37" - , {- in = -} "\x17\x61\x43\x3c\x37\xc5\xa3\x5f\xc1\xf3\x9f\x40\x63\x02\xeb\x90\x7c\x61\x63\xbe\x38\xc9\x84\x37" - , {- out = -} "\x74\xa0\xeb\xc9\x06\x9f\x5b\x37\x58\x10\xe6\xfd\x25\x87\x40\x22\xe8\x03\x61\xa4\x78\xe3\xe9\xcf\x48\x4a\xb0\x4f\x44\x7e\xff\xf6\xf0\xa4\x77\xcc\x2f\xc9\xbf\x54\x89\x44" - , {- M = -} 10 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x27\xca\x0c\x71\x20\xbc\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x44\xa3\xaa\x3a\xae\x64\x75\xca" - , {- in = -} "\xa4\x34\xa8\xe5\x85\x00\xc6\xe4\x15\x30\x53\x88\x62\xd6\x86\xea\x9e\x81\x30\x1b\x5a\xe4\x22\x6b\xfa" - , {- out = -} "\x44\xa3\xaa\x3a\xae\x64\x75\xca\xf2\xbe\xed\x7b\xc5\x09\x8e\x83\xfe\xb5\xb3\x16\x08\xf8\xe2\x9c\x38\x81\x9a\x89\xc8\xe7\x76\xf1\x54\x4d\x41\x51\xa4\xed\x3a\x8b\x87\xb9\xce" - , {- M = -} 10 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x5b\x8c\xcb\xcd\x9a\xf8\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\xec\x46\xbb\x63\xb0\x25\x20\xc3\x3c\x49\xfd\x70" - , {- in = -} "\xb9\x6b\x49\xe2\x1d\x62\x17\x41\x63\x28\x75\xdb\x7f\x6c\x92\x43\xd2\xd7\xc2" - , {- out = -} "\xec\x46\xbb\x63\xb0\x25\x20\xc3\x3c\x49\xfd\x70\x31\xd7\x50\xa0\x9d\xa3\xed\x7f\xdd\xd4\x9a\x20\x32\xaa\xbf\x17\xec\x8e\xbf\x7d\x22\xc8\x08\x8c\x66\x6b\xe5\xc1\x97" - , {- M = -} 10 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x3e\xbe\x94\x04\x4b\x9a\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x47\xa6\x5a\xc7\x8b\x3d\x59\x42\x27\xe8\x5e\x71" - , {- in = -} "\xe2\xfc\xfb\xb8\x80\x44\x2c\x73\x1b\xf9\x51\x67\xc8\xff\xd7\x89\x5e\x33\x70\x76" - , {- out = -} "\x47\xa6\x5a\xc7\x8b\x3d\x59\x42\x27\xe8\x5e\x71\xe8\x82\xf1\xdb\xd3\x8c\xe3\xed\xa7\xc2\x3f\x04\xdd\x65\x07\x1e\xb4\x13\x42\xac\xdf\x7e\x00\xdc\xce\xc7\xae\x52\x98\x7d" - , {- M = -} 10 - ) - , - ( {- key = -} "\xd7\x82\x8d\x13\xb2\xb0\xbd\xc3\x25\xa7\x62\x36\xdf\x93\xcc\x6b" - , {- iv = -} "\x00\x8d\x49\x3b\x30\xae\x8b\x3c\x96\x96\x76\x6c\xfa" - , {- hdr = -} "\x6e\x37\xa6\xef\x54\x6d\x95\x5d\x34\xab\x60\x59" - , {- in = -} "\xab\xf2\x1c\x0b\x02\xfe\xb8\x8f\x85\x6d\xf4\xa3\x73\x81\xbc\xe3\xcc\x12\x85\x17\xd4" - , {- out = -} "\x6e\x37\xa6\xef\x54\x6d\x95\x5d\x34\xab\x60\x59\xf3\x29\x05\xb8\x8a\x64\x1b\x04\xb9\xc9\xff\xb5\x8c\xc3\x90\x90\x0f\x3d\xa1\x2a\xb1\x6d\xce\x9e\x82\xef\xa1\x6d\xa6\x20\x59" - , {- M = -} 10 - ) - ] diff --git a/tests/KAT_AES/KATECB.hs b/tests/KAT_AES/KATECB.hs deleted file mode 100644 --- a/tests/KAT_AES/KATECB.hs +++ /dev/null @@ -1,720 +0,0 @@ -module KAT_AES.KATECB where - -import qualified Data.ByteString as B - -vectors_aes128_enc = - [ - ( B.pack - [ 0x10 - , 0xa5 - , 0x88 - , 0x69 - , 0xd7 - , 0x4b - , 0xe5 - , 0xa3 - , 0x74 - , 0xcf - , 0x86 - , 0x7c - , 0xfb - , 0x47 - , 0x38 - , 0x59 - ] - , B.replicate 16 0 - , B.pack - [ 0x6d - , 0x25 - , 0x1e - , 0x69 - , 0x44 - , 0xb0 - , 0x51 - , 0xe0 - , 0x4e - , 0xaa - , 0x6f - , 0xb4 - , 0xdb - , 0xf7 - , 0x84 - , 0x65 - ] - ) - , - ( B.replicate 16 0 - , B.replicate 16 0 - , B.pack - [ 0x66 - , 0xe9 - , 0x4b - , 0xd4 - , 0xef - , 0x8a - , 0x2c - , 0x3b - , 0x88 - , 0x4c - , 0xfa - , 0x59 - , 0xca - , 0x34 - , 0x2b - , 0x2e - ] - ) - , - ( B.replicate 16 0 - , B.replicate 16 1 - , B.pack - [ 0xe1 - , 0x4d - , 0x5d - , 0x0e - , 0xe2 - , 0x77 - , 0x15 - , 0xdf - , 0x08 - , 0xb4 - , 0x15 - , 0x2b - , 0xa2 - , 0x3d - , 0xa8 - , 0xe0 - ] - ) - , - ( B.replicate 16 1 - , B.replicate 16 2 - , B.pack - [ 0x17 - , 0xd6 - , 0x14 - , 0xf3 - , 0x79 - , 0xa9 - , 0x35 - , 0x90 - , 0x77 - , 0xe9 - , 0x55 - , 0x77 - , 0xfd - , 0x31 - , 0xc2 - , 0x0a - ] - ) - , - ( B.replicate 16 2 - , B.replicate 16 1 - , B.pack - [ 0x8f - , 0x42 - , 0xc2 - , 0x4b - , 0xee - , 0x6e - , 0x63 - , 0x47 - , 0x2b - , 0x16 - , 0x5a - , 0xa9 - , 0x41 - , 0x31 - , 0x2f - , 0x7c - ] - ) - , - ( B.replicate 16 3 - , B.replicate 16 2 - , B.pack - [ 0x90 - , 0x98 - , 0x85 - , 0xe4 - , 0x77 - , 0xbc - , 0x20 - , 0xf5 - , 0x8a - , 0x66 - , 0x97 - , 0x1d - , 0xa0 - , 0xbc - , 0x75 - , 0xe3 - ] - ) - ] - -vectors_aes192_enc = - [ - ( B.replicate 24 0 - , B.replicate 16 0 - , B.pack - [ 0xaa - , 0xe0 - , 0x69 - , 0x92 - , 0xac - , 0xbf - , 0x52 - , 0xa3 - , 0xe8 - , 0xf4 - , 0xa9 - , 0x6e - , 0xc9 - , 0x30 - , 0x0b - , 0xd7 - ] - ) - , - ( B.replicate 24 0 - , B.replicate 16 1 - , B.pack - [ 0xcf - , 0x1e - , 0xce - , 0x3c - , 0x44 - , 0xb0 - , 0x78 - , 0xfb - , 0x27 - , 0xcb - , 0x0a - , 0x3e - , 0x07 - , 0x1b - , 0x08 - , 0x20 - ] - ) - , - ( B.replicate 24 1 - , B.replicate 16 2 - , B.pack - [ 0xeb - , 0x8c - , 0x17 - , 0x30 - , 0x90 - , 0xc7 - , 0x5b - , 0x77 - , 0xd6 - , 0x72 - , 0xb4 - , 0x57 - , 0xa7 - , 0x78 - , 0xd9 - , 0xd0 - ] - ) - , - ( B.replicate 24 2 - , B.replicate 16 1 - , B.pack - [ 0xf2 - , 0xf0 - , 0xae - , 0xd8 - , 0xcd - , 0xc9 - , 0x21 - , 0xca - , 0x4b - , 0x55 - , 0x84 - , 0x5d - , 0xa4 - , 0x15 - , 0x21 - , 0xc2 - ] - ) - , - ( B.replicate 24 3 - , B.replicate 16 2 - , B.pack - [ 0xca - , 0xcc - , 0x30 - , 0x79 - , 0xe4 - , 0xb7 - , 0x95 - , 0x27 - , 0x63 - , 0xd2 - , 0x55 - , 0xd6 - , 0x34 - , 0x10 - , 0x46 - , 0x14 - ] - ) - ] - -vectors_aes256_enc = - [ - ( B.replicate 32 0 - , B.replicate 16 0 - , B.pack - [ 0xdc - , 0x95 - , 0xc0 - , 0x78 - , 0xa2 - , 0x40 - , 0x89 - , 0x89 - , 0xad - , 0x48 - , 0xa2 - , 0x14 - , 0x92 - , 0x84 - , 0x20 - , 0x87 - ] - ) - , - ( B.replicate 32 0 - , B.replicate 16 1 - , B.pack - [ 0x7b - , 0xc3 - , 0x02 - , 0x6c - , 0xd7 - , 0x37 - , 0x10 - , 0x3e - , 0x62 - , 0x90 - , 0x2b - , 0xcd - , 0x18 - , 0xfb - , 0x01 - , 0x63 - ] - ) - , - ( B.replicate 32 1 - , B.replicate 16 2 - , B.pack - [ 0x62 - , 0xae - , 0x12 - , 0xf3 - , 0x24 - , 0xbf - , 0xea - , 0x08 - , 0xd5 - , 0xf6 - , 0x75 - , 0xb5 - , 0x13 - , 0x02 - , 0x6b - , 0xbf - ] - ) - , - ( B.replicate 32 2 - , B.replicate 16 1 - , B.pack - [ 0x00 - , 0xf9 - , 0xc7 - , 0x44 - , 0x4b - , 0xb0 - , 0xcc - , 0x80 - , 0x6c - , 0x7c - , 0x39 - , 0xee - , 0x22 - , 0x11 - , 0xf1 - , 0x46 - ] - ) - , - ( B.replicate 32 3 - , B.replicate 16 2 - , B.pack - [ 0xb4 - , 0x05 - , 0x87 - , 0x3e - , 0xa0 - , 0x76 - , 0x1b - , 0x9c - , 0xa9 - , 0x9f - , 0x70 - , 0xb0 - , 0x16 - , 0x16 - , 0xce - , 0xb1 - ] - ) - ] - -vectors_aes128_dec = - [ - ( B.replicate 16 0 - , B.replicate 16 0 - , B.pack - [ 0x14 - , 0x0f - , 0x0f - , 0x10 - , 0x11 - , 0xb5 - , 0x22 - , 0x3d - , 0x79 - , 0x58 - , 0x77 - , 0x17 - , 0xff - , 0xd9 - , 0xec - , 0x3a - ] - ) - , - ( B.replicate 16 0 - , B.replicate 16 1 - , B.pack - [ 0x15 - , 0x6d - , 0x0f - , 0x85 - , 0x75 - , 0xd5 - , 0x33 - , 0x07 - , 0x52 - , 0xf8 - , 0x4a - , 0xf2 - , 0x72 - , 0xff - , 0x30 - , 0x50 - ] - ) - , - ( B.replicate 16 1 - , B.replicate 16 2 - , B.pack - [ 0x34 - , 0x37 - , 0xd6 - , 0xe2 - , 0x31 - , 0xd7 - , 0x02 - , 0x41 - , 0x9b - , 0x51 - , 0xb4 - , 0x94 - , 0x72 - , 0x71 - , 0xb6 - , 0x11 - ] - ) - , - ( B.replicate 16 2 - , B.replicate 16 1 - , B.pack - [ 0xe3 - , 0xcd - , 0xe2 - , 0x37 - , 0xc8 - , 0xf2 - , 0xd9 - , 0x7b - , 0x8d - , 0x79 - , 0xf9 - , 0x17 - , 0x1d - , 0x4b - , 0xda - , 0xc1 - ] - ) - , - ( B.replicate 16 3 - , B.replicate 16 2 - , B.pack - [ 0x5b - , 0x94 - , 0xaa - , 0xed - , 0xd7 - , 0x83 - , 0x99 - , 0x8c - , 0xd5 - , 0x15 - , 0x35 - , 0x35 - , 0x18 - , 0xcc - , 0x45 - , 0xe2 - ] - ) - ] - -vectors_aes192_dec = - [ - ( B.replicate 24 0 - , B.replicate 16 0 - , B.pack - [ 0x13 - , 0x46 - , 0x0e - , 0x87 - , 0xa8 - , 0xfc - , 0x02 - , 0x3e - , 0xf2 - , 0x50 - , 0x1a - , 0xfe - , 0x7f - , 0xf5 - , 0x1c - , 0x51 - ] - ) - , - ( B.replicate 24 0 - , B.replicate 16 1 - , B.pack - [ 0x92 - , 0x17 - , 0x07 - , 0xc3 - , 0x3d - , 0x1c - , 0xc5 - , 0x96 - , 0x7d - , 0xa5 - , 0x1d - , 0xbb - , 0xb0 - , 0x66 - , 0xb2 - , 0x6c - ] - ) - , - ( B.replicate 24 1 - , B.replicate 16 2 - , B.pack - [ 0xee - , 0x92 - , 0x97 - , 0xc6 - , 0xba - , 0xe8 - , 0x26 - , 0x4d - , 0xff - , 0x08 - , 0x0e - , 0xbb - , 0x1e - , 0x74 - , 0x11 - , 0xc1 - ] - ) - , - ( B.replicate 24 2 - , B.replicate 16 1 - , B.pack - [ 0x49 - , 0x67 - , 0xdf - , 0x70 - , 0xd2 - , 0x9e - , 0x9a - , 0x7f - , 0x5d - , 0x7c - , 0xb9 - , 0xc1 - , 0x20 - , 0xc3 - , 0x8a - , 0x71 - ] - ) - , - ( B.replicate 24 3 - , B.replicate 16 2 - , B.pack - [ 0x74 - , 0x38 - , 0x62 - , 0x42 - , 0x6b - , 0x56 - , 0x7f - , 0xd5 - , 0xf0 - , 0x1d - , 0x1b - , 0x59 - , 0x56 - , 0x01 - , 0x26 - , 0x29 - ] - ) - ] - -vectors_aes256_dec = - [ - ( B.replicate 32 0 - , B.replicate 16 0 - , B.pack - [ 0x67 - , 0x67 - , 0x1c - , 0xe1 - , 0xfa - , 0x91 - , 0xdd - , 0xeb - , 0x0f - , 0x8f - , 0xbb - , 0xb3 - , 0x66 - , 0xb5 - , 0x31 - , 0xb4 - ] - ) - , - ( B.replicate 32 0 - , B.replicate 16 1 - , B.pack - [ 0xcc - , 0x09 - , 0x21 - , 0xa3 - , 0xc5 - , 0xca - , 0x17 - , 0xf7 - , 0x48 - , 0xb7 - , 0xc2 - , 0x7b - , 0x73 - , 0xba - , 0x87 - , 0xa2 - ] - ) - , - ( B.replicate 32 1 - , B.replicate 16 2 - , B.pack - [ 0xc0 - , 0x4b - , 0x27 - , 0x90 - , 0x1a - , 0x50 - , 0xcf - , 0xfa - , 0xf1 - , 0xbb - , 0x88 - , 0x9f - , 0xc0 - , 0x92 - , 0x5e - , 0x14 - ] - ) - , - ( B.replicate 32 2 - , B.replicate 16 1 - , B.pack - [ 0x24 - , 0x61 - , 0x53 - , 0x5d - , 0x16 - , 0x1c - , 0x15 - , 0x39 - , 0x88 - , 0x32 - , 0x77 - , 0x29 - , 0xc5 - , 0x8c - , 0xc0 - , 0x3a - ] - ) - , - ( B.replicate 32 3 - , B.replicate 16 2 - , B.pack - [ 0x30 - , 0xc9 - , 0x1c - , 0xce - , 0xfe - , 0x89 - , 0x30 - , 0xcf - , 0xff - , 0x31 - , 0xdb - , 0xcc - , 0xfc - , 0x11 - , 0xc5 - , 0x23 - ] - ) - ] diff --git a/tests/KAT_AES/KATGCM.hs b/tests/KAT_AES/KATGCM.hs deleted file mode 100644 --- a/tests/KAT_AES/KATGCM.hs +++ /dev/null @@ -1,120 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AES.KATGCM where - -import qualified Data.ByteString as B -import Data.ByteString.Char8 () - --- (key, iv, aad, input, out, taglen, tag) -type KATGCM = - ( B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - , Int - , B.ByteString - ) - -vectors_aes128_enc :: [KATGCM] -vectors_aes128_enc = - [ -- vectors 0 - - ( {-key = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-aad = -} "" - , {-input = -} "" - , {-out = -} "" - , {-taglen = -} 16 - , {-tag = -} "\x58\xe2\xfc\xce\xfa\x7e\x30\x61\x36\x7f\x1d\x57\xa4\xe7\x45\x5a" - ) - , -- vectors 1 - - ( {-key = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-aad = -} "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , {-input = -} "\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a" - , {-out = -} "\x09\x82\xd0\xc4\x6a\xbc\xa9\x98\xf9\x22\xc8\xb3\x7b\xb8\xf4\x72\xfd\x9f\xa0\xa1\x43\x41\x53\x29\xfd\xf7\x83\xf5\x9e\x81\xcb\xea" - , {-taglen = -} 16 - , {-tag = -} "\x28\x50\x64\x2f\xa8\x8b\xab\x21\x2a\x67\x1a\x97\x48\x69\xa5\x6c" - ) - , -- vectors 2 - - ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-aad = -} "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , {-input = -} "\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a" - , {-out = -} "\x1c\xa3\xb5\x41\x39\x6f\x19\x7a\x91\x2d\x27\x15\x70\xd1\xf5\x76\xde\xf1\xbe\x84\x42\x2a\xbb\xbe\x0b\x2d\x91\x21\x82\xbf\x7f\x17" - , {-taglen = -} 16 - , {-tag = -} "\x15\x2a\x05\xbb\x7e\x13\x5d\xbe\x93\x7f\xa0\x54\x7a\x8e\x74\xb6" - ) - , -- vectors 3 - - ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-aad = -} "\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01\x01" - , {-input = -} "\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a\x0a" - , {-out = -} "\xda\x35\xf6\x0a\x65\xc2\xa4\x6c\xb6\x6e\xb6\xf8\x1f\x0b\x9c\x74\x53\x4c\x97\x70\x36\xf7\xdf\x05\x6d\x00\xfe\xbf\xb4\xcb\xf5\x27" - , {-taglen = -} 16 - , {-tag = -} "\xb7\x76\x7c\x3b\x9e\xf1\xe2\xcb\xc9\x11\xf1\x9a\xdc\xfa\x35\x0d" - ) - , - ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-aad = -} "\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76" - , {-input = -} "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" - , {-out = -} "\xe4\x42\xf8\xc4\xc6\x67\x84\x86\x4a\x5a\x6e\xc7\xe0\xca\x68\xac\x16\xbc\x5b\xbf\xf7\xd5\xf3\xfa\xf3\xb2\xcb\xb0\xa2\x14\xa1\x81" - , {-taglen = -} 16 - , {-tag = -} "\x5f\x63\xb8\xeb\x1d\x6f\xa8\x7a\xeb\x39\xa5\xf6\xd7\xed\xc3\x13" - ) - , - ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\xff\xfe\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-aad = -} "\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76\x76" - , {-input = -} "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" - , {-out = -} "\xe4\x42\xf8\xc4\xc6\x67\x84\x86\x4a\x5a\x6e\xc7\xe0\xca\x68\xac\x16\xbc\x5b\xbf\xf7\xd5\xf3\xfa\xf3\xb2\xcb\xb0\xa2\x14\xa1" - , {-taglen = -} 16 - , {-tag = -} "\x94\xd1\x47\xc3\xa2\xca\x93\xe9\x66\x93\x1e\x3b\xb3\xbb\x67\x01" - ) - , -- vector 6 tests 32-bit counter wrapping - - ( {-key = -} "\x01\x02\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , {-iv = -} "\xe8\x38\x84\x1d\x75\xae\x33\xb5\x4b\x51\x57\x89\xc9\x5f\xbe\x65" - , {-aad = -} "\x54\x68\x65\x20\x66\x69\x76\x65\x20\x62\x6f\x78\x69\x6e\x67\x20\x77\x69\x7a\x61\x72\x64\x73\x20\x6a\x75\x6d\x70\x20\x71\x75\x69\x63\x6b\x6c\x79\x2e" - , {-input = -} "\x54\x68\x65\x20\x71\x75\x69\x63\x6b\x20\x62\x72\x6f\x77\x6e\x20\x66\x6f\x78\x20\x6a\x75\x6d\x70\x73\x20\x6f\x76\x65\x72\x20\x74\x68\x65\x20\x6c\x61\x7a\x79\x20\x64\x6f\x67" - , {-out = -} "\x82\x31\x9e\x5a\x6a\x7f\x43\xd0\x42\x8c\xf1\x01\xcf\x0c\x75\xf1\x5d\xda\x4f\xa1\x28\x95\xcd\xd7\x7b\xd5\x42\x68\x2f\xcd\x10\x1b\x0c\x75\x05\x54\xf4\x2f\x2b\xf6\x69\x96\x29" - , {-taglen = -} 16 - , {-tag = -} "\x9a\xfa\xf4\xea\xae\x2e\x6f\x40\x00\xf4\x89\x77\xd0\x1e\xd5\x14" - ) - ] - -vectors_aes256_enc :: [KATGCM] -vectors_aes256_enc = - [ - ( "\xb5\x2c\x50\x5a\x37\xd7\x8e\xda\x5d\xd3\x4f\x20\xc2\x25\x40\xea\x1b\x58\x96\x3c\xf8\xe5\xbf\x8f\xfa\x85\xf9\xf2\x49\x25\x05\xb4" - , "\x51\x6c\x33\x92\x9d\xf5\xa3\x28\x4f\xf4\x63\xd7" - , "" - , "" - , "" - , 16 - , "\xbd\xc1\xac\x88\x4d\x33\x24\x57\xa1\xd2\x66\x4f\x16\x8c\x76\xf0" - ) - , - ( "\x78\xdc\x4e\x0a\xaf\x52\xd9\x35\xc3\xc0\x1e\xea\x57\x42\x8f\x00\xca\x1f\xd4\x75\xf5\xda\x86\xa4\x9c\x8d\xd7\x3d\x68\xc8\xe2\x23" - , "\xd7\x9c\xf2\x2d\x50\x4c\xc7\x93\xc3\xfb\x6c\x8a" - , "\xb9\x6b\xaa\x8c\x1c\x75\xa6\x71\xbf\xb2\xd0\x8d\x06\xbe\x5f\x36" - , "" - , "" - , 16 - , "\x3e\x5d\x48\x6a\xa2\xe3\x0b\x22\xe0\x40\xb8\x57\x23\xa0\x6e\x76" - ) - , - ( "\xc3\xf1\x05\x86\xf2\x46\xaa\xca\xdc\xce\x37\x01\x44\x17\x70\xc0\x3c\xfe\xc9\x40\xaf\xe1\x90\x8c\x4c\x53\x7d\xf4\xe0\x1c\x50\xa0" - , "\x4f\x52\xfa\xa1\xfa\x67\xa0\xe5\xf4\x19\x64\x52" - , "\x46\xf9\xa2\x2b\x4e\x52\xe1\x52\x65\x13\xa9\x52\xdb\xee\x3b\x91\xf6\x95\x95\x50\x1e\x01\x77\xd5\x0f\xf3\x64\x63\x85\x88\xc0\x8d\x92\xfa\xb8\xc5\x8a\x96\x9b\xdc\xc8\x4c\x46\x8d\x84\x98\xc4\xf0\x63\x92\xb9\x9e\xd5\xe0\xc4\x84\x50\x7f\xc4\x8d\xc1\x8d\x87\xc4\x0e\x2e\xd8\x48\xb4\x31\x50\xbe\x9d\x36\xf1\x4c\xf2\xce\xf1\x31\x0b\xa4\xa7\x45\xad\xcc\x7b\xdc\x41\xf6" - , "\x79\xd9\x7e\xa3\xa2\xed\xd6\x50\x45\x82\x1e\xa7\x45\xa4\x47\x42" - , "\x56\x0c\xf7\x16\xe5\x61\x90\xe9\x39\x7c\x2f\x10\x36\x29\xeb\x1f" - , 16 - , "\xff\x7c\x91\x24\x87\x96\x44\xe8\x05\x55\x68\x7d\x27\x3c\x55\xd8" - ) - ] diff --git a/tests/KAT_AES/KATOCB3.hs b/tests/KAT_AES/KATOCB3.hs deleted file mode 100644 --- a/tests/KAT_AES/KATOCB3.hs +++ /dev/null @@ -1,394 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AES.KATOCB3 where - -import qualified Data.ByteString as B -import Data.ByteString.Char8 () - --- (key, iv, aad, input, out, taglen, tag) -type KATOCB3 = - ( B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - , Int - , B.ByteString - ) - -key1 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" -nonce1 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b" -key2 = "\x0f\x0e\x0d\x0c\x0b\x0a\x09\x08\x07\x06\x05\x04\x03\x02\x01\x00" -nonce2 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0d" -nonce_rfc7253_00 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x00" -nonce_rfc7253_01 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x01" -nonce_rfc7253_02 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x02" -nonce_rfc7253_03 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x03" -nonce_rfc7253_04 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x04" -nonce_rfc7253_05 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x05" -nonce_rfc7253_06 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x06" -nonce_rfc7253_07 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x07" -nonce_rfc7253_08 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x08" -nonce_rfc7253_09 = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x09" -nonce_rfc7253_0a = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0a" -nonce_rfc7253_0b = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0b" -nonce_rfc7253_0c = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0c" -nonce_rfc7253_0d = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0d" -nonce_rfc7253_0e = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0e" -nonce_rfc7253_0f = "\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0f" -nonce_dkg_120_00 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x00" -nonce_dkg_120_01 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x01" -nonce_dkg_120_02 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x02" -nonce_dkg_120_03 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x03" -nonce_dkg_120_04 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x04" -nonce_dkg_120_05 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x05" -nonce_dkg_120_06 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x06" -nonce_dkg_120_07 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x07" -nonce_dkg_120_08 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x08" -nonce_dkg_120_09 = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x09" -nonce_dkg_120_0a = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0a" -nonce_dkg_120_0b = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0b" -nonce_dkg_120_0c = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0c" -nonce_dkg_120_0d = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0d" -nonce_dkg_120_0e = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0e" -nonce_dkg_120_0f = "\xee\xdd\xcc\xbb\xaa\x99\x88\x77\x66\x55\x44\x33\x22\x11\x0f" - -bytes8 = "\x00\x01\x02\x03\x04\x05\x06\x07" -bytes16 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" -bytes24 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17" -bytes32 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f" -bytes40 = "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" - -vectors_aes128_enc :: [KATOCB3] -vectors_aes128_enc = - [ - ( {-key = -} key1 - , {-iv = -} nonce1 - , {-aad = -} "" - , {-input = -} "" - , {-out = -} "" - , {-taglen = -} 16 - , {-tag = -} "\x19\x7b\x9c\x3c\x44\x1d\x3c\x83\xea\xfb\x2b\xef\x63\x3b\x91\x82" - ) - , - ( key1 - , nonce1 - , "\x00\x01\x02\x03\x04\x05\x06\x07" - , "\x00\x01\x02\x03\x04\x05\x06\x07" - , "\x92\xb6\x57\x13\x0a\x74\xb8\x5a" - , 16 - , "\x16\xdc\x76\xa4\x6d\x47\xe1\xea\xd5\x37\x20\x9e\x8a\x96\xd1\x4e" - ) - , - ( key1 - , nonce1 - , "\x00\x01\x02\x03\x04\x05\x06\x07" - , "" - , "" - , 16 - , "\x98\xb9\x15\x52\xc8\xc0\x09\x18\x50\x44\xe3\x0a\x6e\xb2\xfe\x21" - ) - , - ( key1 - , nonce1 - , "" - , "\x00\x01\x02\x03\x04\x05\x06\x07" - , "\x92\xb6\x57\x13\x0a\x74\xb8\x5a" - , 16 - , "\x97\x1e\xff\xca\xe1\x9a\xd4\x71\x6f\x88\xe8\x7b\x87\x1f\xbe\xed" - ) - , - ( key1 - , nonce1 - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" - , "\xbe\xa5\xe8\x79\x8d\xbe\x71\x10\x03\x1c\x14\x4d\xa0\xb2\x61\x22" - , 16 - , "\x77\x6c\x99\x24\xd6\x72\x3a\x1f\xc4\x52\x45\x32\xac\x3e\x5b\xeb" - ) - {- Disabled: 96-bit tag vector - , ( key2 - , nonce2 - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27" - , "\x17\x92\xa4\xe3\x1e\x07\x55\xfb\x03\xe3\x1b\x22\x11\x6e\x6c\x2d\xdf\x9e\xfd\x6e\x33\xd5\x36\xf1\xa0\x12\x4b\x0a\x55\xba\xe8\x84\xed\x93\x48\x15\x29\xc7\x6b\x6a" - , 12 - , "\xd0\xc5\x15\xf4\xd1\xcd\xd4\xfd\xac\x4f\x02\xaa" - ) - -} - ] ++ vectors_rfc7253_aes128_tag128 - ++ vectors_dkg_nonce120_aes128 - -vectors_rfc7253_aes128_tag128 :: [KATOCB3] -vectors_rfc7253_aes128_tag128 = - [ ( key1 - , nonce_rfc7253_00 - , "" - , "" - , "" - , 16 - , "\x78\x54\x07\xbf\xff\xc8\xad\x9e\xdc\xc5\x52\x0a\xc9\x11\x1e\xe6" - ) - , ( key1 - , nonce_rfc7253_01 - , bytes8 - , bytes8 - , "\x68\x20\xb3\x65\x7b\x6f\x61\x5a" - , 16 - , "\x57\x25\xbd\xa0\xd3\xb4\xeb\x3a\x25\x7c\x9a\xf1\xf8\xf0\x30\x09" - ) - , ( key1 - , nonce_rfc7253_02 - , bytes8 - , "" - , "" - , 16 - , "\x81\x01\x7f\x82\x03\xf0\x81\x27\x71\x52\xfa\xde\x69\x4a\x0a\x00" - ) - , ( key1 - , nonce_rfc7253_03 - , "" - , bytes8 - , "\x45\xdd\x69\xf8\xf5\xaa\xe7\x24" - , 16 - , "\x14\x05\x4c\xd1\xf3\x5d\x82\x76\x0b\x2c\xd0\x0d\x2f\x99\xbf\xa9" - ) - , ( key1 - , nonce_rfc7253_04 - , bytes16 - , bytes16 - , "\x57\x1d\x53\x5b\x60\xb2\x77\x18\x8b\xe5\x14\x71\x70\xa9\xa2\x2c" - , 16 - , "\x3a\xd7\xa4\xff\x38\x35\xb8\xc5\x70\x1c\x1c\xce\xc8\xfc\x33\x58" - ) - , ( key1 - , nonce_rfc7253_05 - , bytes16 - , "" - , "" - , 16 - , "\x8c\xf7\x61\xb6\x90\x2e\xf7\x64\x46\x2a\xd8\x64\x98\xca\x6b\x97" - ) - , ( key1 - , nonce_rfc7253_06 - , "" - , bytes16 - , "\x5c\xe8\x8e\xc2\xe0\x69\x27\x06\xa9\x15\xc0\x0a\xeb\x8b\x23\x96" - , 16 - , "\xf4\x0e\x1c\x74\x3f\x52\x43\x6b\xdf\x06\xd8\xfa\x1e\xca\x34\x3d" - ) - , ( key1 - , nonce_rfc7253_07 - , bytes24 - , bytes24 - , "\x1c\xa2\x20\x73\x08\xc8\x7c\x01\x07\x56\x10\x4d\x88\x40\xce\x19\x52\xf0\x96\x73\xa4\x48\xa1\x22" - , 16 - , "\xc9\x2c\x62\x24\x10\x51\xf5\x73\x56\xd7\xf3\xc9\x0b\xb0\xe0\x7f" - ) - , ( key1 - , nonce_rfc7253_08 - , bytes24 - , "" - , "" - , 16 - , "\x6d\xc2\x25\xa0\x71\xfc\x1b\x9f\x7c\x69\xf9\x3b\x0f\x1e\x10\xde" - ) - , ( key1 - , nonce_rfc7253_09 - , "" - , bytes24 - , "\x22\x1b\xd0\xde\x7f\xa6\xfe\x99\x3e\xcc\xd7\x69\x46\x0a\x0a\xf2\xd6\xcd\xed\x0c\x39\x5b\x1c\x3c" - , 16 - , "\xe7\x25\xf3\x24\x94\xb9\xf9\x14\xd8\x5c\x0b\x1e\xb3\x83\x57\xff" - ) - , ( key1 - , nonce_rfc7253_0a - , bytes32 - , bytes32 - , "\xbd\x6f\x6c\x49\x62\x01\xc6\x92\x96\xc1\x1e\xfd\x13\x8a\x46\x7a\xbd\x3c\x70\x79\x24\xb9\x64\xde\xaf\xfc\x40\x31\x9a\xf5\xa4\x85" - , 16 - , "\x40\xfb\xba\x18\x6c\x55\x53\xc6\x8a\xd9\xf5\x92\xa7\x9a\x42\x40" - ) - , ( key1 - , nonce_rfc7253_0b - , bytes32 - , "" - , "" - , 16 - , "\xfe\x80\x69\x0b\xee\x8a\x48\x5d\x11\xf3\x29\x65\xbc\x9d\x2a\x32" - ) - , ( key1 - , nonce_rfc7253_0c - , "" - , bytes32 - , "\x29\x42\xbf\xc7\x73\xbd\xa2\x3c\xab\xc6\xac\xfd\x9b\xfd\x58\x35\xbd\x30\x0f\x09\x73\x79\x2e\xf4\x60\x40\xc5\x3f\x14\x32\xbc\xdf" - , 16 - , "\xb5\xe1\xdd\xe3\xbc\x18\xa5\xf8\x40\xb5\x2e\x65\x34\x44\xd5\xdf" - ) - , ( key1 - , nonce_rfc7253_0d - , bytes40 - , bytes40 - , "\xd5\xca\x91\x74\x84\x10\xc1\x75\x1f\xf8\xa2\xf6\x18\x25\x5b\x68\xa0\xa1\x2e\x09\x3f\xf4\x54\x60\x6e\x59\xf9\xc1\xd0\xdd\xc5\x4b\x65\xe8\x62\x8e\x56\x8b\xad\x7a" - , 16 - , "\xed\x07\xba\x06\xa4\xa6\x94\x83\xa7\x03\x54\x90\xc5\x76\x9e\x60" - ) - , ( key1 - , nonce_rfc7253_0e - , bytes40 - , "" - , "" - , 16 - , "\xc5\xcd\x9d\x18\x50\xc1\x41\xe3\x58\x64\x99\x94\xee\x70\x1b\x68" - ) - , ( key1 - , nonce_rfc7253_0f - , "" - , bytes40 - , "\x44\x12\x92\x34\x93\xc5\x7d\x5d\xe0\xd7\x00\xf7\x53\xcc\xe0\xd1\xd2\xd9\x50\x60\x12\x2e\x9f\x15\xa5\xdd\xbf\xc5\x78\x7e\x50\xb5\xcc\x55\xee\x50\x7b\xcb\x08\x4e" - , 16 - , "\x47\x9a\xd3\x63\xac\x36\x6b\x95\xa9\x8c\xa5\xf3\x00\x0b\x14\x79" - ) - ] - -vectors_dkg_nonce120_aes128 :: [KATOCB3] -vectors_dkg_nonce120_aes128 = - [ ( key1 - , nonce_dkg_120_00 - , "" - , "" - , "" - , 16 - , "\x75\x2a\xcd\x21\x32\xc4\x1e\x02\x0e\x41\xfb\x22\x3e\xfd\x77\xb6" - ) - , ( key1 - , nonce_dkg_120_01 - , bytes8 - , bytes8 - , "\x20\x1f\xe4\xd8\x9e\xa7\xbd\x1e" - , 16 - , "\xb5\xb1\x57\x7d\xb1\x62\x83\xb8\xae\xd1\x71\x5a\xd6\xbe\x51\x49" - ) - , ( key1 - , nonce_dkg_120_02 - , bytes8 - , "" - , "" - , 16 - , "\x71\x09\x60\xb9\xee\x00\xb8\xf4\x4d\x2e\x81\x20\xaa\xba\x63\xae" - ) - , ( key1 - , nonce_dkg_120_03 - , "" - , bytes8 - , "\x08\x4e\x86\x95\x70\x19\x4b\xd2" - , 16 - , "\x50\x32\xfe\x9e\x53\x28\xe4\x5d\x50\x7e\x74\xf3\x36\x6e\x20\xd2" - ) - , ( key1 - , nonce_dkg_120_04 - , bytes16 - , bytes16 - , "\x96\x76\xee\x37\xfd\x64\x5c\x07\xc0\xd4\xf7\x0a\xab\xf6\x86\x68" - , 16 - , "\x8e\x39\xb2\xfb\x3f\xc4\xff\x30\xdc\xd1\x82\x7b\x36\xa2\x98\xd3" - ) - , ( key1 - , nonce_dkg_120_05 - , bytes16 - , "" - , "" - , 16 - , "\x9d\x51\x0f\x56\xed\xf7\x2f\xfa\x34\x96\x9b\xce\xf9\x1e\x6d\xe9" - ) - , ( key1 - , nonce_dkg_120_06 - , "" - , bytes16 - , "\xd5\xe1\x5a\xa1\xd2\x32\xab\x57\xf2\x34\x36\x6d\xff\xb2\x55\x74" - , 16 - , "\xa3\x63\x6a\x5f\x3e\x34\x33\xea\x45\x90\xcb\xf4\xf9\xac\x1f\x4d" - ) - , ( key1 - , nonce_dkg_120_07 - , bytes24 - , bytes24 - , "\x1c\x4b\x67\x77\xb7\xf1\x37\xc3\x09\x71\xa9\x3d\xe3\xc5\x6c\xc7\x35\x68\x6a\x6f\x77\x03\x14\x2f" - , 16 - , "\xab\x8a\xcc\x98\x7c\x14\x06\xdf\xf9\x62\x73\xc5\x37\x6e\x62\x10" - ) - , ( key1 - , nonce_dkg_120_08 - , bytes24 - , "" - , "" - , 16 - , "\x96\xe6\x70\xc0\x23\x8f\xb9\x69\xb7\xac\xe4\xab\xaf\x74\x38\xc7" - ) - , ( key1 - , nonce_dkg_120_09 - , "" - , bytes24 - , "\x12\x90\xa6\x86\xd8\x25\xf7\x12\xe5\x94\xbe\x40\x39\xc0\x4d\x3e\x44\xf7\xd1\x34\x2b\x84\xff\xca" - , 16 - , "\xd6\x8b\xbd\xfa\x04\xb5\x80\xea\x9a\x01\xe2\xf4\x56\x53\x99\xc3" - ) - , ( key1 - , nonce_dkg_120_0a - , bytes32 - , bytes32 - , "\xfb\xdf\xc1\x1f\x74\x92\x17\xbb\x7f\xae\x5d\x40\x36\xb8\xf2\x28\x03\x71\x2e\xff\x9e\xf9\x43\x42\xfe\x1b\x68\x49\x68\xd0\xe3\xe3" - , 16 - , "\x81\xa2\x77\xda\xab\x83\x57\x94\x06\xa0\x1e\x26\x75\xa0\x82\xc9" - ) - , ( key1 - , nonce_dkg_120_0b - , bytes32 - , "" - , "" - , 16 - , "\x90\xcd\xa8\xa0\x51\x61\xd2\x87\x33\x61\x37\x4b\x76\xf9\x54\x30" - ) - , ( key1 - , nonce_dkg_120_0c - , "" - , bytes32 - , "\xd1\x32\x0a\xf4\xb6\xff\x8a\xfe\xec\xee\x79\x21\x39\x5d\x4e\x86\x92\x71\x77\x53\xee\x15\xf5\x03\x8e\xb6\x74\xda\x43\xd6\xea\x8d" - , 16 - , "\xbe\x78\x31\xe7\x23\xbe\x47\x1f\x62\xd9\xe7\xf4\x9a\x7d\x3b\x32" - ) - , ( key1 - , nonce_dkg_120_0d - , bytes40 - , bytes40 - , "\x5c\x79\xf1\xc4\xb9\xa2\x04\xed\x33\x23\x61\x6d\x57\x6f\xc5\x00\xe4\xa7\x19\x39\xf0\x3a\x3c\x3d\xe2\xc0\x97\xaf\x2c\x6c\x81\xdc\x3f\x03\x09\xe7\x60\x82\xb1\xf5" - , 16 - , "\x0f\xf8\x52\x29\x59\xff\xe4\x1f\x37\xef\x50\x7e\x90\x76\xd3\x2c" - ) - , ( key1 - , nonce_dkg_120_0e - , bytes40 - , "" - , "" - , 16 - , "\x3b\xf1\x58\xb7\xde\x76\xc5\x15\x1e\xf6\x08\x6a\x82\x5d\x0c\xc4" - ) - , ( key1 - , nonce_dkg_120_0f - , "" - , bytes40 - , "\x34\xda\x59\xd2\xeb\x08\xf4\x78\x22\xd4\x8c\x85\xb6\xa1\xd2\x36\x94\xe1\xd3\xde\x68\x0d\x61\x6d\x7b\x1b\x59\x47\x2c\x13\xe3\x69\xc6\x8d\xca\x69\x9d\xa1\x68\x6a" - , 16 - , "\x33\x9d\x54\x52\x80\x36\x32\x81\x0b\x08\x40\xe6\x80\x4a\xb0\x20" - ) - {- Disabled: 96-bit tag vector - , ( key2 - , nonce_dkg_120_0d - , bytes40 - , bytes40 - , "\x07\xe9\x03\xbf\xc4\x95\x52\x41\x1a\xbc\x86\x5f\x5e\xce\x60\xf6\xfa\xd1\xf5\xa9\xf1\x4d\x30\x70\xfa\x2f\x13\x08\xa5\x63\x20\x7f\xfe\x14\xc1\xee\xa4\x4b\x22\x05" - , 12 - , "\x9c\x74\x84\x31\x9d\x8a\x2c\x53\xc2\x36\xa7\xb3" - ) - -} - ] diff --git a/tests/KAT_AES/KATXTS.hs b/tests/KAT_AES/KATXTS.hs deleted file mode 100644 --- a/tests/KAT_AES/KATXTS.hs +++ /dev/null @@ -1,68 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AES.KATXTS where - -import qualified Data.ByteString as B -import Data.ByteString.Char8 () - -type KATXTS = - ( B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - , B.ByteString - ) - -vectors_aes128_enc - , vectors_aes128_dec - , vectors_aes256_enc - , vectors_aes256_dec - :: [KATXTS] -vectors_aes128_enc = - [ - ( "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x66\xe9\x4b\xd4\xef\x8a\x2c\x3b\x88\x4c\xfa\x59\xca\x34\x2b\x2e\xcc\xd2\x97\xa8\xdf\x15\x59\x76\x10\x99\xf4\xb3\x94\x69\x56\x5c" - , "\x91\x7c\xf6\x9e\xbd\x68\xb2\xec\x9b\x9f\xe9\xa3\xea\xdd\xa6\x92\xcd\x43\xd2\xf5\x95\x98\xed\x85\x8c\x02\xc2\x65\x2f\xbf\x92\x2e" - ) - , - ( "\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11\x11" - , "\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22" - , "\x33\x33\x33\x33\x33\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44" - , "\x3f\x80\x3b\xcd\x0d\x7f\xd2\xb3\x75\x58\x41\x9f\x59\xd5\xcd\xa6\xf9\x00\x77\x9a\x1b\xfe\xa4\x67\xeb\xb0\x82\x3e\xb3\xaa\x9b\x4d" - , "\xc4\x54\x18\x5e\x6a\x16\x93\x6e\x39\x33\x40\x38\xac\xef\x83\x8b\xfb\x18\x6f\xff\x74\x80\xad\xc4\x28\x93\x82\xec\xd6\xd3\x94\xf0" - ) - , - ( "\xff\xfe\xfd\xfc\xfb\xfa\xf9\xf8\xf7\xf6\xf5\xf4\xf3\xf2\xf1\xf0" - , "\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22\x22" - , "\x33\x33\x33\x33\x33\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44\x44" - , "\x3f\x80\x3b\xcd\x0d\x7f\xd2\xb3\x75\x58\x41\x9f\x59\xd5\xcd\xa6\xf9\x00\x77\x9a\x1b\xfe\xa4\x67\xeb\xb0\x82\x3e\xb3\xaa\x9b\x4d" - , "\xaf\x85\x33\x6b\x59\x7a\xfc\x1a\x90\x0b\x2e\xb2\x1e\xc9\x49\xd2\x92\xdf\x4c\x04\x7e\x0b\x21\x53\x21\x86\xa5\x97\x1a\x22\x7a\x89" - ) - , - ( "\x27\x18\x28\x18\x28\x45\x90\x45\x23\x53\x60\x28\x74\x71\x35\x26" - , "\x31\x41\x59\x26\x53\x58\x97\x93\x23\x84\x62\x64\x33\x83\x27\x95" - , "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff" - , "" - , "\x27\xa7\x47\x9b\xef\xa1\xd4\x76\x48\x9f\x30\x8c\xd4\xcf\xa6\xe2\xa9\x6e\x4b\xbe\x32\x08\xff\x25\x28\x7d\xd3\x81\x96\x16\xe8\x9c\xc7\x8c\xf7\xf5\xe5\x43\x44\x5f\x83\x33\xd8\xfa\x7f\x56\x00\x00\x05\x27\x9f\xa5\xd8\xb5\xe4\xad\x40\xe7\x36\xdd\xb4\xd3\x54\x12\x32\x80\x63\xfd\x2a\xab\x53\xe5\xea\x1e\x0a\x9f\x33\x25\x00\xa5\xdf\x94\x87\xd0\x7a\x5c\x92\xcc\x51\x2c\x88\x66\xc7\xe8\x60\xce\x93\xfd\xf1\x66\xa2\x49\x12\xb4\x22\x97\x61\x46\xae\x20\xce\x84\x6b\xb7\xdc\x9b\xa9\x4a\x76\x7a\xae\xf2\x0c\x0d\x61\xad\x02\x65\x5e\xa9\x2d\xc4\xc4\xe4\x1a\x89\x52\xc6\x51\xd3\x31\x74\xbe\x51\xa1\x0c\x42\x11\x10\xe6\xd8\x15\x88\xed\xe8\x21\x03\xa2\x52\xd8\xa7\x50\xe8\x76\x8d\xef\xff\xed\x91\x22\x81\x0a\xae\xb9\x9f\x91\x72\xaf\x82\xb6\x04\xdc\x4b\x8e\x51\xbc\xb0\x82\x35\xa6\xf4\x34\x13\x32\xe4\xca\x60\x48\x2a\x4b\xa1\xa0\x3b\x3e\x65\x00\x8f\xc5\xda\x76\xb7\x0b\xf1\x69\x0d\xb4\xea\xe2\x9c\x5f\x1b\xad\xd0\x3c\x5c\xcf\x2a\x55\xd7\x05\xdd\xcd\x86\xd4\x49\x51\x1c\xeb\x7e\xc3\x0b\xf1\x2b\x1f\xa3\x5b\x91\x3f\x9f\x74\x7a\x8a\xfd\x1b\x13\x0e\x94\xbf\xf9\x4e\xff\xd0\x1a\x91\x73\x5c\xa1\x72\x6a\xcd\x0b\x19\x7c\x4e\x5b\x03\x39\x36\x97\xe1\x26\x82\x6f\xb6\xbb\xde\x8e\xcc\x1e\x08\x29\x85\x16\xe2\xc9\xed\x03\xff\x3c\x1b\x78\x60\xf6\xde\x76\xd4\xce\xcd\x94\xc8\x11\x98\x55\xef\x52\x97\xca\x67\xe9\xf3\xe7\xff\x72\xb1\xe9\x97\x85\xca\x0a\x7e\x77\x20\xc5\xb3\x6d\xc6\xd7\x2c\xac\x95\x74\xc8\xcb\xbc\x2f\x80\x1e\x23\xe5\x6f\xd3\x44\xb0\x7f\x22\x15\x4b\xeb\xa0\xf0\x8c\xe8\x89\x1e\x64\x3e\xd9\x95\xc9\x4d\x9a\x69\xc9\xf1\xb5\xf4\x99\x02\x7a\x78\x57\x2a\xee\xbd\x74\xd2\x0c\xc3\x98\x81\xc2\x13\xee\x77\x0b\x10\x10\xe4\xbe\xa7\x18\x84\x69\x77\xae\x11\x9f\x7a\x02\x3a\xb5\x8c\xca\x0a\xd7\x52\xaf\xe6\x56\xbb\x3c\x17\x25\x6a\x9f\x6e\x9b\xf1\x9f\xdd\x5a\x38\xfc\x82\xbb\xe8\x72\xc5\x53\x9e\xdb\x60\x9e\xf4\xf7\x9c\x20\x3e\xbb\x14\x0f\x2e\x58\x3c\xb2\xad\x15\xb4\xaa\x5b\x65\x50\x16\xa8\x44\x92\x77\xdb\xd4\x77\xef\x2c\x8d\x6c\x01\x7d\xb7\x38\xb1\x8d\xeb\x4a\x42\x7d\x19\x23\xce\x3f\xf2\x62\x73\x57\x79\xa4\x18\xf2\x0a\x28\x2d\xf9\x20\x14\x7b\xea\xbe\x42\x1e\xe5\x31\x9d\x05\x68" - ) - ] -vectors_aes128_dec = - [] -vectors_aes256_enc = - [ - ( "\x27\x18\x28\x18\x28\x45\x90\x45\x23\x53\x60\x28\x74\x71\x35\x26\x62\x49\x77\x57\x24\x70\x93\x69\x99\x59\x57\x49\x66\x96\x76\x27" - , "\x31\x41\x59\x26\x53\x58\x97\x93\x23\x84\x62\x64\x33\x83\x27\x95\x02\x88\x41\x97\x16\x93\x99\x37\x51\x05\x82\x09\x74\x94\x45\x92" - , "\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f\x20\x21\x22\x23\x24\x25\x26\x27\x28\x29\x2a\x2b\x2c\x2d\x2e\x2f\x30\x31\x32\x33\x34\x35\x36\x37\x38\x39\x3a\x3b\x3c\x3d\x3e\x3f\x40\x41\x42\x43\x44\x45\x46\x47\x48\x49\x4a\x4b\x4c\x4d\x4e\x4f\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f\x60\x61\x62\x63\x64\x65\x66\x67\x68\x69\x6a\x6b\x6c\x6d\x6e\x6f\x70\x71\x72\x73\x74\x75\x76\x77\x78\x79\x7a\x7b\x7c\x7d\x7e\x7f\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f\xa0\xa1\xa2\xa3\xa4\xa5\xa6\xa7\xa8\xa9\xaa\xab\xac\xad\xae\xaf\xb0\xb1\xb2\xb3\xb4\xb5\xb6\xb7\xb8\xb9\xba\xbb\xbc\xbd\xbe\xbf\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7\xc8\xc9\xca\xcb\xcc\xcd\xce\xcf\xd0\xd1\xd2\xd3\xd4\xd5\xd6\xd7\xd8\xd9\xda\xdb\xdc\xdd\xde\xdf\xe0\xe1\xe2\xe3\xe4\xe5\xe6\xe7\xe8\xe9\xea\xeb\xec\xed\xee\xef\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9\xfa\xfb\xfc\xfd\xfe\xff" - , "" - , "\x1c\x3b\x3a\x10\x2f\x77\x03\x86\xe4\x83\x6c\x99\xe3\x70\xcf\x9b\xea\x00\x80\x3f\x5e\x48\x23\x57\xa4\xae\x12\xd4\x14\xa3\xe6\x3b\x5d\x31\xe2\x76\xf8\xfe\x4a\x8d\x66\xb3\x17\xf9\xac\x68\x3f\x44\x68\x0a\x86\xac\x35\xad\xfc\x33\x45\xbe\xfe\xcb\x4b\xb1\x88\xfd\x57\x76\x92\x6c\x49\xa3\x09\x5e\xb1\x08\xfd\x10\x98\xba\xec\x70\xaa\xa6\x69\x99\xa7\x2a\x82\xf2\x7d\x84\x8b\x21\xd4\xa7\x41\xb0\xc5\xcd\x4d\x5f\xff\x9d\xac\x89\xae\xba\x12\x29\x61\xd0\x3a\x75\x71\x23\xe9\x87\x0f\x8a\xcf\x10\x00\x02\x08\x87\x89\x14\x29\xca\x2a\x3e\x7a\x7d\x7d\xf7\xb1\x03\x55\x16\x5c\x8b\x9a\x6d\x0a\x7d\xe8\xb0\x62\xc4\x50\x0d\xc4\xcd\x12\x0c\x0f\x74\x18\xda\xe3\xd0\xb5\x78\x1c\x34\x80\x3f\xa7\x54\x21\xc7\x90\xdf\xe1\xde\x18\x34\xf2\x80\xd7\x66\x7b\x32\x7f\x6c\x8c\xd7\x55\x7e\x12\xac\x3a\x0f\x93\xec\x05\xc5\x2e\x04\x93\xef\x31\xa1\x2d\x3d\x92\x60\xf7\x9a\x28\x9d\x6a\x37\x9b\xc7\x0c\x50\x84\x14\x73\xd1\xa8\xcc\x81\xec\x58\x3e\x96\x45\xe0\x7b\x8d\x96\x70\x65\x5b\xa5\xbb\xcf\xec\xc6\xdc\x39\x66\x38\x0a\xd8\xfe\xcb\x17\xb6\xba\x02\x46\x9a\x02\x0a\x84\xe1\x8e\x8f\x84\x25\x20\x70\xc1\x3e\x9f\x1f\x28\x9b\xe5\x4f\xbc\x48\x14\x57\x77\x8f\x61\x60\x15\xe1\x32\x7a\x02\xb1\x40\xf1\x50\x5e\xb3\x09\x32\x6d\x68\x37\x8f\x83\x74\x59\x5c\x84\x9d\x84\xf4\xc3\x33\xec\x44\x23\x88\x51\x43\xcb\x47\xbd\x71\xc5\xed\xae\x9b\xe6\x9a\x2f\xfe\xce\xb1\xbe\xc9\xde\x24\x4f\xbe\x15\x99\x2b\x11\xb7\x7c\x04\x0f\x12\xbd\x8f\x6a\x97\x5a\x44\xa0\xf9\x0c\x29\xa9\xab\xc3\xd4\xd8\x93\x92\x72\x84\xc5\x87\x54\xcc\xe2\x94\x52\x9f\x86\x14\xdc\xd2\xab\xa9\x91\x92\x5f\xed\xc4\xae\x74\xff\xac\x6e\x33\x3b\x93\xeb\x4a\xff\x04\x79\xda\x9a\x41\x0e\x44\x50\xe0\xdd\x7a\xe4\xc6\xe2\x91\x09\x00\x57\x5d\xa4\x01\xfc\x07\x05\x9f\x64\x5e\x8b\x7e\x9b\xfd\xef\x33\x94\x30\x54\xff\x84\x01\x14\x93\xc2\x7b\x34\x29\xea\xed\xb4\xed\x53\x76\x44\x1a\x77\xed\x43\x85\x1a\xd7\x7f\x16\xf5\x41\xdf\xd2\x69\xd5\x0d\x6a\x5f\x14\xfb\x0a\xab\x1c\xbb\x4c\x15\x50\xbe\x97\xf7\xab\x40\x66\x19\x3c\x4c\xaa\x77\x3d\xad\x38\x01\x4b\xd2\x09\x2f\xa7\x55\xc8\x24\xbb\x5e\x54\xc4\xf3\x6f\xfd\xa9\xfc\xea\x70\xb9\xc6\xe6\x93\xe1\x48\xc1\x51" - ) - ] -vectors_aes256_dec = [] diff --git a/tests/KAT_AESGCMSIV.hs b/tests/KAT_AESGCMSIV.hs deleted file mode 100644 --- a/tests/KAT_AESGCMSIV.hs +++ /dev/null @@ -1,673 +0,0 @@ -{-# LANGUAGE FlexibleInstances #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE Rank2Types #-} -{-# LANGUAGE RecordWildCards #-} - -module KAT_AESGCMSIV (tests) where - -import Imports - -import qualified Data.ByteArray as B -import Data.Proxy - -import Crypto.Cipher.AES -import Crypto.Cipher.AESGCMSIV -import Crypto.Cipher.Types -import Crypto.Error - -data Vector c = Vector - { vecPlaintext :: ByteString - , vecAAD :: ByteString - , vecKey :: ByteString - , vecNonce :: ByteString - , vecTag :: ByteString - , vecCiphertext :: ByteString - } - -vecCipher :: Cipher c => Vector c -> c -vecCipher = throwCryptoError . cipherInit . vecKey - -vectors128 :: [Vector AES128] -vectors128 = - [ Vector - { vecPlaintext = "" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xdc\x20\xe2\xd8\x3f\x25\x70\x5b\xb4\x9e\x43\x9e\xca\x56\xde\x25" - , vecCiphertext = "" - } - , Vector - { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x57\x87\x82\xff\xf6\x01\x3b\x81\x5b\x28\x7c\x22\x49\x3a\x36\x4c" - , vecCiphertext = "\xb5\xd8\x39\x33\x0a\xc7\xb7\x86" - } - , Vector - { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xa4\x97\x8d\xb3\x57\x39\x1a\x0b\xc4\xfd\xec\x8b\x0d\x10\x66\x39" - , vecCiphertext = "\x73\x23\xea\x61\xd0\x59\x32\x26\x00\x47\xd9\x42" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x30\x3a\xaf\x90\xf6\xfe\x21\x19\x9c\x60\x68\x57\x74\x37\xa0\xc4" - , vecCiphertext = - "\x74\x3f\x7c\x80\x77\xab\x25\xf8\x62\x4e\x2e\x94\x85\x79\xcf\x77" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x1a\x8e\x45\xdc\xd4\x57\x8c\x66\x7c\xd8\x68\x47\xbf\x61\x55\xff" - , vecCiphertext = - "\x84\xe0\x7e\x62\xba\x83\xa6\x58\x54\x17\x24\x5d\x7e\xc4\x13\xa9\xfe\x42\x7d\x63\x15\xc0\x9b\x57\xce\x45\xf2\xe3\x93\x6a\x94\x45" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x5e\x6e\x31\x1d\xbf\x39\x5d\x35\xb0\xfe\x39\xc2\x71\x43\x88\xf8" - , vecCiphertext = - "\x3f\xd2\x4c\xe1\xf5\xa6\x7b\x75\xbf\x23\x51\xf1\x81\xa4\x75\xc7\xb8\x00\xa5\xb4\xd3\xdc\xf7\x01\x06\xb1\xee\xa8\x2f\xa1\xd6\x4d\xf4\x2b\xf7\x22\x61\x22\xfa\x92\xe1\x7a\x40\xee\xaa\xc1\x20\x1b" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x8a\x26\x3d\xd3\x17\xaa\x88\xd5\x6b\xdf\x39\x36\xdb\xa7\x5b\xb8" - , vecCiphertext = - "\x24\x33\x66\x8f\x10\x58\x19\x0f\x6d\x43\xe3\x60\xf4\xf3\x5c\xd8\xe4\x75\x12\x7c\xfc\xa7\x02\x8e\xa8\xab\x5c\x20\xf7\xab\x2a\xf0\x25\x16\xa2\xbd\xcb\xc0\x8d\x52\x1b\xe3\x7f\xf2\x8c\x15\x2b\xba\x36\x69\x7f\x25\xb4\xcd\x16\x9c\x65\x90\xd1\xdd\x39\x56\x6d\x3f" - } - , Vector - { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x3b\x0a\x1a\x25\x60\x96\x9c\xdf\x79\x0d\x99\x75\x9a\xbd\x15\x08" - , vecCiphertext = "\x1e\x6d\xab\xa3\x56\x69\xf4\x27" - } - , Vector - { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x08\x29\x9c\x51\x02\x74\x5a\xaa\x3a\x0c\x46\x9f\xad\x9e\x07\x5a" - , vecCiphertext = "\x29\x6c\x78\x89\xfd\x99\xf4\x19\x17\xf4\x46\x20" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x8f\x89\x36\xec\x03\x9e\x4e\x4b\xb9\x7e\xbd\x8c\x44\x57\x44\x1f" - , vecCiphertext = - "\xe2\xb0\xc5\xda\x79\xa9\x01\xc1\x74\x5f\x70\x05\x25\xcb\x33\x5b" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xe6\xaf\x6a\x7f\x87\x28\x7d\xa0\x59\xa7\x16\x84\xed\x34\x98\xe1" - , vecCiphertext = - "\x62\x00\x48\xef\x3c\x1e\x73\xe5\x7e\x02\xbb\x85\x62\xc4\x16\xa3\x19\xe7\x3e\x4c\xaa\xc8\xe9\x6a\x1e\xcb\x29\x33\x14\x5a\x1d\x71" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x6a\x8c\xc3\x86\x5f\x76\x89\x7c\x2e\x4b\x24\x5c\xf3\x1c\x51\xf2" - , vecCiphertext = - "\x50\xc8\x30\x3e\xa9\x39\x25\xd6\x40\x90\xd0\x7b\xd1\x09\xdf\xd9\x51\x5a\x5a\x33\x43\x10\x19\xc1\x7d\x93\x46\x59\x99\xa8\xb0\x05\x32\x01\xd7\x23\x12\x0a\x85\x62\xb8\x38\xcd\xff\x25\xbf\x9d\x1e" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xcd\xc4\x6a\xe4\x75\x56\x3d\xe0\x37\x00\x1e\xf8\x4a\xe2\x17\x44" - , vecCiphertext = - "\x2f\x5c\x64\x05\x9d\xb5\x5e\xe0\xfb\x84\x7e\xd5\x13\x00\x37\x46\xac\xa4\xe6\x1c\x71\x1b\x5d\xe2\xe7\xa7\x7f\xfd\x02\xda\x42\xfe\xec\x60\x19\x10\xd3\x46\x7b\xb8\xb3\x6e\xbb\xae\xbc\xe5\xfb\xa3\x0d\x36\xc9\x5f\x48\xa3\xe7\x98\x0f\x0e\x7a\xc2\x99\x33\x2a\x80" - } - , Vector - { vecPlaintext = "\x02\x00\x00\x00" - , vecAAD = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x07\xeb\x1f\x84\xfb\x28\xf8\xcb\x73\xde\x8e\x99\xe2\xf4\x8a\x14" - , vecCiphertext = "\xa8\xfe\x3e\x87" - } - , Vector - { vecPlaintext = - "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00" - , vecAAD = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x24\xaf\xc9\x80\x5e\x97\x6f\x45\x1e\x6d\x87\xf6\xfe\x10\x65\x14" - , vecCiphertext = - "\x6b\xb0\xfe\xcf\x5d\xed\x9b\x77\xf9\x02\xc7\xd5\xda\x23\x6a\x43\x91\xdd\x02\x97" - } - , Vector - { vecPlaintext = - "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00" - , vecAAD = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xbf\xf9\xb2\xef\x00\xfb\x47\x92\x0c\xc7\x2a\x0c\x0f\x13\xb9\xfd" - , vecCiphertext = - "\x44\xd0\xaa\xf6\xfb\x2f\x1f\x34\xad\xd5\xe8\x06\x4e\x83\xe1\x2a\x2a\xda" - } - , Vector - { vecPlaintext = "" - , vecAAD = "" - , vecKey = - "\xe6\x60\x21\xd5\xeb\x8e\x4f\x40\x66\xd4\xad\xb9\xc3\x35\x60\xe4" - , vecNonce = "\xf4\x6e\x44\xbb\x3d\xa0\x01\x5c\x94\xf7\x08\x87" - , vecTag = - "\xa4\x19\x4b\x79\x07\x1b\x01\xa8\x7d\x65\xf7\x06\xe3\x94\x95\x78" - , vecCiphertext = "" - } - , Vector - { vecPlaintext = "\x7a\x80\x6c" - , vecAAD = "\x46\xbb\x91\xc3\xc5" - , vecKey = - "\x36\x86\x42\x00\xe0\xea\xf5\x28\x4d\x88\x4a\x0e\x77\xd3\x16\x46" - , vecNonce = "\xba\xe8\xe3\x7f\xc8\x34\x41\xb1\x60\x34\x56\x6b" - , vecTag = - "\x71\x1b\xd8\x5b\xc1\xe4\xd3\xe0\xa4\x62\xe0\x74\xee\xa4\x28\xa8" - , vecCiphertext = "\xaf\x60\xeb" - } - , Vector - { vecPlaintext = "\xbd\xc6\x6f\x14\x65\x45" - , vecAAD = "\xfc\x88\x0c\x94\xa9\x51\x98\x87\x42\x96" - , vecKey = - "\xae\xdb\x64\xa6\xc5\x90\xbc\x84\xd1\xa5\xe2\x69\xe4\xb4\x78\x01" - , vecNonce = "\xaf\xc0\x57\x7e\x34\x69\x9b\x9e\x67\x1f\xdd\x4f" - , vecTag = - "\xd6\xa9\xc4\x55\x45\xcf\xc1\x1f\x03\xad\x74\x3d\xba\x20\xf9\x66" - , vecCiphertext = "\xbb\x93\xa3\xe3\x4d\x3c" - } - , Vector - { vecPlaintext = "\x11\x77\x44\x1f\x19\x54\x95\x86\x0f" - , vecAAD = "\x04\x67\x87\xf3\xea\x22\xc1\x27\xaa\xf1\x95\xd1\x89\x47\x28" - , vecKey = - "\xd5\xcc\x1f\xd1\x61\x32\x0b\x69\x20\xce\x07\x78\x7f\x86\x74\x3b" - , vecNonce = "\x27\x5d\x1a\xb3\x2f\x6d\x1f\x04\x34\xd8\x84\x8c" - , vecTag = - "\x1d\x02\xfd\x0c\xd1\x74\xc8\x4f\xc5\xda\xe2\xf6\x0f\x52\xfd\x2b" - , vecCiphertext = "\x4f\x37\x28\x1f\x7a\xd1\x29\x49\xd0" - } - , Vector - { vecPlaintext = "\x9f\x57\x2c\x61\x4b\x47\x45\x91\x44\x74\xe7\xc7" - , vecAAD = - "\xc9\x88\x2e\x53\x86\xfd\x9f\x92\xec\x48\x9c\x8f\xde\x2b\xe2\xcf\x97\xe7\x4e\x93" - , vecKey = - "\xb3\xfe\xd1\x47\x3c\x52\x8b\x84\x26\xa5\x82\x99\x59\x29\xa1\x49" - , vecNonce = "\x9e\x9a\xd8\x78\x0c\x8d\x63\xd0\xab\x41\x49\xc0" - , vecTag = - "\xc1\xdc\x2f\x87\x1f\xb7\x56\x1d\xa1\x28\x6e\x65\x5e\x24\xb7\xb0" - , vecCiphertext = "\xf5\x46\x73\xc5\xdd\xf7\x10\xc7\x45\x64\x1c\x8b" - } - , Vector - { vecPlaintext = "\x0d\x8c\x84\x51\x17\x80\x82\x35\x5c\x9e\x94\x0f\xea\x2f\x58" - , vecAAD = - "\x29\x50\xa7\x0d\x5a\x1d\xb2\x31\x6f\xd5\x68\x37\x8d\xa1\x07\xb5\x2b\x0d\xa5\x52\x10\xcc\x1c\x1b\x0a" - , vecKey = - "\x2d\x4e\xd8\x7d\xa4\x41\x02\x95\x2e\xf9\x4b\x02\xb8\x05\x24\x9b" - , vecNonce = "\xac\x80\xe6\xf6\x14\x55\xbf\xac\x83\x08\xa2\xd4" - , vecTag = - "\x83\xb3\x44\x9b\x9f\x39\x55\x2d\xe9\x9d\xc2\x14\xa1\x19\x0b\x0b" - , vecCiphertext = "\xc9\xff\x54\x5e\x07\xb8\x8a\x01\x5f\x05\xb2\x74\x54\x0a\xa1" - } - , Vector - { vecPlaintext = - "\x6b\x3d\xb4\xda\x3d\x57\xaa\x94\x84\x2b\x98\x03\xa9\x6e\x07\xfb\x6d\xe7" - , vecAAD = - "\x18\x60\xf7\x62\xeb\xfb\xd0\x82\x84\xe4\x21\x70\x2d\xe0\xde\x18\xba\xa9\xc9\x59\x62\x91\xb0\x84\x66\xf3\x7d\xe2\x1c\x7f" - , vecKey = - "\xbd\xe3\xb2\xf2\x04\xd1\xe9\xf8\xb0\x6b\xc4\x7f\x97\x45\xb3\xd1" - , vecNonce = "\xae\x06\x55\x6f\xb6\xaa\x78\x90\xbe\xbc\x18\xfe" - , vecTag = - "\x3e\x37\x70\x94\xf0\x47\x09\xf6\x4d\x7b\x98\x53\x10\xa4\xdb\x84" - , vecCiphertext = - "\x62\x98\xb2\x96\xe2\x4e\x8c\xc3\x5d\xce\x0b\xed\x48\x4b\x7f\x30\xd5\x80" - } - , Vector - { vecPlaintext = - "\xe4\x2a\x3c\x02\xc2\x5b\x64\x86\x9e\x14\x6d\x7b\x23\x39\x87\xbd\xdf\xc2\x40\x87\x1d" - , vecAAD = - "\x75\x76\xf7\x02\x8e\xc6\xeb\x5e\xa7\xe2\x98\x34\x2a\x94\xd4\xb2\x02\xb3\x70\xef\x97\x68\xec\x65\x61\xc4\xfe\x6b\x7e\x72\x96\xfa\x85\x9c\x21" - , vecKey = - "\xf9\x01\xcf\xe8\xa6\x96\x15\xa9\x3f\xdf\x7a\x98\xca\xd4\x81\x79" - , vecNonce = "\x62\x45\x70\x9f\xb1\x88\x53\xf6\x8d\x83\x36\x40" - , vecTag = - "\x2d\x15\x50\x6c\x84\xa9\xed\xd6\x5e\x13\xe9\xd2\x4a\x2a\x6e\x70" - , vecCiphertext = - "\x39\x1c\xc3\x28\xd4\x84\xa4\xf4\x64\x06\x18\x1b\xcd\x62\xef\xd9\xb3\xee\x19\x7d\x05" - } - ] - -vectors256 :: [Vector AES256] -vectors256 = - [ Vector - { vecPlaintext = "" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x07\xf5\xf4\x16\x9b\xbf\x55\xa8\x40\x0c\xd4\x7e\xa6\xfd\x40\x0f" - , vecCiphertext = "" - } - , Vector - { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x84\x31\x22\x13\x0f\x73\x64\xb7\x61\xe0\xb9\x74\x27\xe3\xdf\x28" - , vecCiphertext = "\xc2\xef\x32\x8e\x5c\x71\xc8\x3b" - } - , Vector - { vecPlaintext = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x8c\xa5\x0d\xa9\xae\x65\x59\xe4\x8f\xd1\x0f\x6e\x5c\x9c\xa1\x7e" - , vecCiphertext = "\x9a\xab\x2a\xeb\x3f\xaa\x0a\x34\xae\xa8\xe2\xb1" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xc9\xea\xc6\xfa\x70\x09\x42\x70\x2e\x90\x86\x23\x83\xc6\xc3\x66" - , vecCiphertext = - "\x85\xa0\x1b\x63\x02\x5b\xa1\x9b\x7f\xd3\xdd\xfc\x03\x3b\x3e\x76" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xe8\x19\xe6\x3a\xbc\xd0\x20\xb0\x06\xa9\x76\x39\x76\x32\xeb\x5d" - , vecCiphertext = - "\x4a\x6a\x9d\xb4\xc8\xc6\x54\x92\x01\xb9\xed\xb5\x30\x06\xcb\xa8\x21\xec\x9c\xf8\x50\x94\x8a\x7c\x86\xc6\x8a\xc7\x53\x9d\x02\x7f" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x79\x0b\xc9\x68\x80\xa9\x9b\xa8\x04\xbd\x12\xc0\xe6\xa2\x2c\xc4" - , vecCiphertext = - "\xc0\x0d\x12\x18\x93\xa9\xfa\x60\x3f\x48\xcc\xc1\xca\x3c\x57\xce\x74\x99\x24\x5e\xa0\x04\x6d\xb1\x6c\x53\xc7\xc6\x6f\xe7\x17\xe3\x9c\xf6\xc7\x48\x83\x7b\x61\xf6\xee\x3a\xdc\xee\x17\x53\x4e\xd5" - } - , Vector - { vecPlaintext = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x11\x28\x64\xc2\x69\xfc\x0d\x9d\x88\xc6\x1f\xa4\x7e\x39\xaa\x08" - , vecCiphertext = - "\xc2\xd5\x16\x0a\x1f\x86\x83\x83\x49\x10\xac\xda\xfc\x41\xfb\xb1\x63\x2d\x4a\x35\x3e\x8b\x90\x5e\xc9\xa5\x49\x9a\xc3\x4f\x96\xc7\xe1\x04\x9e\xb0\x80\x88\x38\x91\xa4\xdb\x8c\xaa\xa1\xf9\x9d\xd0\x04\xd8\x04\x87\x54\x07\x35\x23\x4e\x37\x44\x51\x2c\x6f\x90\xce" - } - , Vector - { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x91\x21\x3f\x26\x7e\x3b\x45\x2f\x02\xd0\x1a\xe3\x3e\x4e\xc8\x54" - , vecCiphertext = "\x1d\xe2\x29\x67\x23\x7a\x81\x32" - } - , Vector - { vecPlaintext = "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xc1\xa4\xa1\x9a\xe8\x00\x94\x1c\xcd\xc5\x7c\xc8\x41\x3c\x27\x7f" - , vecCiphertext = "\x16\x3d\x6f\x9c\xc1\xb3\x46\xcd\x45\x3a\x2e\x4c" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xb2\x92\xd2\x8f\xf6\x11\x89\xe8\xe4\x9f\x38\x75\xef\x91\xaf\xf7" - , vecCiphertext = - "\xc9\x15\x45\x82\x3c\xc2\x4f\x17\xdb\xb0\xe9\xe8\x07\xd5\xec\x17" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xae\xa1\xba\xd1\x27\x02\xe1\x96\x56\x04\x37\x4a\xab\x96\xdb\xbc" - , vecCiphertext = - "\x07\xda\xd3\x64\xbf\xc2\xb9\xda\x89\x11\x6d\x7b\xef\x6d\xaa\xaf\x6f\x25\x55\x10\xaa\x65\x4f\x92\x0a\xc8\x1b\x94\xe8\xba\xd3\x65" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x03\x33\x27\x42\xb2\x28\xc6\x47\x17\x36\x16\xcf\xd4\x4c\x54\xeb" - , vecCiphertext = - "\xc6\x7a\x1f\x0f\x56\x7a\x51\x98\xaa\x1f\xcc\x8e\x3f\x21\x31\x43\x36\xf7\xf5\x1c\xa8\xb1\xaf\x61\xfe\xac\x35\xa8\x64\x16\xfa\x47\xfb\xca\x3b\x5f\x74\x9c\xdf\x56\x45\x27\xf2\x31\x4f\x42\xfe\x25" - } - , Vector - { vecPlaintext = - "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "\x01" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x5b\xde\x02\x85\x03\x7c\x5d\xe8\x1e\x5b\x57\x0a\x04\x9b\x62\xa0" - , vecCiphertext = - "\x67\xfd\x45\xe1\x26\xbf\xb9\xa7\x99\x30\xc4\x3a\xad\x2d\x36\x96\x7d\x3f\x0e\x4d\x21\x7c\x1e\x55\x1f\x59\x72\x78\x70\xbe\xef\xc9\x8c\xb9\x33\xa8\xfc\xe9\xde\x88\x7b\x1e\x40\x79\x99\x88\xdb\x1f\xc3\xf9\x18\x80\xed\x40\x5b\x2d\xd2\x98\x31\x88\x58\x46\x7c\x89" - } - , Vector - { vecPlaintext = "\x02\x00\x00\x00" - , vecAAD = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\x18\x35\xe5\x17\x74\x1d\xfd\xdc\xcf\xa0\x7f\xa4\x66\x1b\x74\xcf" - , vecCiphertext = "\x22\xb3\xf4\xcd" - } - , Vector - { vecPlaintext = - "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00\x00\x00" - , vecAAD = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xb8\x79\xad\x97\x6d\x82\x42\xac\xc1\x88\xab\x59\xca\xbf\xe3\x07" - , vecCiphertext = - "\x43\xdd\x01\x63\xcd\xb4\x8f\x9f\xe3\x21\x2b\xf6\x1b\x20\x19\x76\x06\x7f\x34\x2b" - } - , Vector - { vecPlaintext = - "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x04\x00" - , vecAAD = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x00\x00" - , vecKey = - "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xcf\xcd\xf5\x04\x21\x12\xaa\x29\x68\x5c\x91\x2f\xc2\x05\x65\x43" - , vecCiphertext = - "\x46\x24\x01\x72\x4b\x5c\xe6\x58\x8d\x5a\x54\xaa\xe5\x37\x55\x13\xa0\x75" - } - , Vector - { vecPlaintext = "" - , vecAAD = "" - , vecKey = - "\xe6\x60\x21\xd5\xeb\x8e\x4f\x40\x66\xd4\xad\xb9\xc3\x35\x60\xe4\xf4\x6e\x44\xbb\x3d\xa0\x01\x5c\x94\xf7\x08\x87\x36\x86\x42\x00" - , vecNonce = "\xe0\xea\xf5\x28\x4d\x88\x4a\x0e\x77\xd3\x16\x46" - , vecTag = - "\x16\x9f\xbb\x2f\xbf\x38\x9a\x99\x5f\x63\x90\xaf\x22\x22\x8a\x62" - , vecCiphertext = "" - } - , Vector - { vecPlaintext = "\x67\x1f\xdd" - , vecAAD = "\x4f\xbd\xc6\x6f\x14" - , vecKey = - "\xba\xe8\xe3\x7f\xc8\x34\x41\xb1\x60\x34\x56\x6b\x7a\x80\x6c\x46\xbb\x91\xc3\xc5\xae\xdb\x64\xa6\xc5\x90\xbc\x84\xd1\xa5\xe2\x69" - , vecNonce = "\xe4\xb4\x78\x01\xaf\xc0\x57\x7e\x34\x69\x9b\x9e" - , vecTag = - "\x93\xda\x9b\xb8\x13\x33\xae\xe0\xc7\x85\xb2\x40\xd3\x19\x71\x9d" - , vecCiphertext = "\x0e\xac\xcb" - } - , Vector - { vecPlaintext = "\x19\x54\x95\x86\x0f\x04" - , vecAAD = "\x67\x87\xf3\xea\x22\xc1\x27\xaa\xf1\x95" - , vecKey = - "\x65\x45\xfc\x88\x0c\x94\xa9\x51\x98\x87\x42\x96\xd5\xcc\x1f\xd1\x61\x32\x0b\x69\x20\xce\x07\x78\x7f\x86\x74\x3b\x27\x5d\x1a\xb3" - , vecNonce = "\x2f\x6d\x1f\x04\x34\xd8\x84\x8c\x11\x77\x44\x1f" - , vecTag = - "\x6b\x62\xb8\x4d\xc4\x0c\x84\x63\x6a\x5e\xc1\x20\x20\xec\x8c\x2c" - , vecCiphertext = "\xa2\x54\xda\xd4\xf3\xf9" - } - , Vector - { vecPlaintext = "\xc9\x88\x2e\x53\x86\xfd\x9f\x92\xec" - , vecAAD = "\x48\x9c\x8f\xde\x2b\xe2\xcf\x97\xe7\x4e\x93\x2d\x4e\xd8\x7d" - , vecKey = - "\xd1\x89\x47\x28\xb3\xfe\xd1\x47\x3c\x52\x8b\x84\x26\xa5\x82\x99\x59\x29\xa1\x49\x9e\x9a\xd8\x78\x0c\x8d\x63\xd0\xab\x41\x49\xc0" - , vecNonce = "\x9f\x57\x2c\x61\x4b\x47\x45\x91\x44\x74\xe7\xc7" - , vecTag = - "\xc0\xfd\x3d\xc6\x62\x8d\xfe\x55\xeb\xb0\xb9\xfb\x22\x95\xc8\xc2" - , vecCiphertext = "\x0d\xf9\xe3\x08\x67\x82\x44\xc4\x4b" - } - , Vector - { vecPlaintext = "\x1d\xb2\x31\x6f\xd5\x68\x37\x8d\xa1\x07\xb5\x2b" - , vecAAD = - "\x0d\xa5\x52\x10\xcc\x1c\x1b\x0a\xbd\xe3\xb2\xf2\x04\xd1\xe9\xf8\xb0\x6b\xc4\x7f" - , vecKey = - "\xa4\x41\x02\x95\x2e\xf9\x4b\x02\xb8\x05\x24\x9b\xac\x80\xe6\xf6\x14\x55\xbf\xac\x83\x08\xa2\xd4\x0d\x8c\x84\x51\x17\x80\x82\x35" - , vecNonce = "\x5c\x9e\x94\x0f\xea\x2f\x58\x29\x50\xa7\x0d\x5a" - , vecTag = - "\x40\x40\x99\xc2\x58\x7f\x64\x97\x9f\x21\x82\x67\x06\xd4\x97\xd5" - , vecCiphertext = "\x8d\xbe\xb9\xf7\x25\x5b\xf5\x76\x9d\xd5\x66\x92" - } - , Vector - { vecPlaintext = "\x21\x70\x2d\xe0\xde\x18\xba\xa9\xc9\x59\x62\x91\xb0\x84\x66" - , vecAAD = - "\xf3\x7d\xe2\x1c\x7f\xf9\x01\xcf\xe8\xa6\x96\x15\xa9\x3f\xdf\x7a\x98\xca\xd4\x81\x79\x62\x45\x70\x9f" - , vecKey = - "\x97\x45\xb3\xd1\xae\x06\x55\x6f\xb6\xaa\x78\x90\xbe\xbc\x18\xfe\x6b\x3d\xb4\xda\x3d\x57\xaa\x94\x84\x2b\x98\x03\xa9\x6e\x07\xfb" - , vecNonce = "\x6d\xe7\x18\x60\xf7\x62\xeb\xfb\xd0\x82\x84\xe4" - , vecTag = - "\xb3\x08\x0d\x28\xf6\xeb\xb5\xd3\x64\x8c\xe9\x7b\xd5\xba\x67\xfd" - , vecCiphertext = "\x79\x35\x76\xdf\xa5\xc0\xf8\x87\x29\xa7\xed\x3c\x2f\x1b\xff" - } - , Vector - { vecPlaintext = - "\xb2\x02\xb3\x70\xef\x97\x68\xec\x65\x61\xc4\xfe\x6b\x7e\x72\x96\xfa\x85" - , vecAAD = - "\x9c\x21\x59\x05\x8b\x1f\x0f\xe9\x14\x33\xa5\xbd\xc2\x0e\x21\x4e\xab\x7f\xec\xef\x44\x54\xa1\x0e\xf0\x65\x7d\xf2\x1a\xc7" - , vecKey = - "\xb1\x88\x53\xf6\x8d\x83\x36\x40\xe4\x2a\x3c\x02\xc2\x5b\x64\x86\x9e\x14\x6d\x7b\x23\x39\x87\xbd\xdf\xc2\x40\x87\x1d\x75\x76\xf7" - , vecNonce = "\x02\x8e\xc6\xeb\x5e\xa7\xe2\x98\x34\x2a\x94\xd4" - , vecTag = - "\x45\x4f\xc2\xa1\x54\xfe\xa9\x1f\x83\x63\xa3\x9f\xec\x7d\x0a\x49" - , vecCiphertext = - "\x85\x7e\x16\xa6\x49\x15\xa7\x87\x63\x76\x87\xdb\x4a\x95\x19\x63\x5c\xdd" - } - , Vector - { vecPlaintext = - "\xce\xd5\x32\xce\x41\x59\xb0\x35\x27\x7d\x4d\xfb\xb7\xdb\x62\x96\x8b\x13\xcd\x4e\xec" - , vecAAD = - "\x73\x43\x20\xcc\xc9\xd9\xbb\xbb\x19\xcb\x81\xb2\xaf\x4e\xcb\xc3\xe7\x28\x34\x32\x1f\x7a\xa0\xf7\x0b\x72\x82\xb4\xf3\x3d\xf2\x3f\x16\x75\x41" - , vecKey = - "\x3c\x53\x5d\xe1\x92\xea\xed\x38\x22\xa2\xfb\xbe\x2c\xa9\xdf\xc8\x82\x55\xe1\x4a\x66\x1b\x8a\xa8\x2c\xc5\x42\x36\x09\x3b\xbc\x23" - , vecNonce = "\x68\x80\x89\xe5\x55\x40\xdb\x18\x72\x50\x4e\x1c" - , vecTag = - "\x9d\x6c\x70\x29\x67\x5b\x89\xea\xf4\xba\x1d\xed\x1a\x28\x65\x94" - , vecCiphertext = - "\x62\x66\x60\xc2\x6e\xa6\x61\x2f\xb1\x7a\xd9\x1e\x8e\x76\x76\x39\xed\xd6\xc9\xfa\xee" - } - ] - -vectorsWrap256 :: [Vector AES256] -vectorsWrap256 = - [ Vector - { vecPlaintext = - "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x4d\xb9\x23\xdc\x79\x3e\xe6\x49\x7c\x76\xdc\xc0\x3a\x98\xe1\x08" - , vecAAD = "" - , vecKey = - "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecCiphertext = - "\xf3\xf8\x0f\x2c\xf0\xcb\x2d\xd9\xc5\x98\x4f\xcd\xa9\x08\x45\x6c\xc5\x37\x70\x3b\x5b\xa7\x03\x24\xa6\x79\x3a\x7b\xf2\x18\xd3\xea" - } - , Vector - { vecPlaintext = - "\xeb\x36\x40\x27\x7c\x7f\xfd\x13\x03\xc7\xa5\x42\xd0\x2d\x3e\x4c\x00\x00\x00\x00\x00\x00\x00\x00" - , vecAAD = "" - , vecKey = - "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecNonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecTag = - "\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" - , vecCiphertext = - "\x18\xce\x4f\x0b\x8c\xb4\xd0\xca\xc6\x5f\xea\x8f\x79\x25\x7b\x20\x88\x8e\x53\xe7\x22\x99\xe5\x6d" - } - ] - -makeEncryptionTest :: BlockCipher128 aes => Int -> Vector aes -> TestTree -makeEncryptionTest i vec@Vector{..} = - testCase (show i) $ - (t, vecCiphertext) @=? encrypt (vecCipher vec) n vecAAD vecPlaintext - where - t = AuthTag (B.convert vecTag) - n = throwCryptoError (nonce vecNonce) - -makeDecryptionTest :: BlockCipher128 aes => Int -> Vector aes -> TestTree -makeDecryptionTest i vec@Vector{..} = - testCase (show i) $ - Just vecPlaintext @=? decrypt (vecCipher vec) n vecAAD vecCiphertext t - where - t = AuthTag (B.convert vecTag) - n = throwCryptoError (nonce vecNonce) - -katTests - :: TestName - -> (forall c. BlockCipher128 c => Int -> Vector c -> TestTree) - -> TestTree -katTests name makeTest = - testGroup - name - [ testGroup "AES128" $ zipWith makeTest [1 ..] vectors128 - , testGroup "AES256" $ zipWith makeTest [1 ..] vectors256 - , testGroup "CounterWrap" $ zipWith makeTest [1 ..] vectorsWrap256 - ] - -newtype Key c = Key ByteString - deriving (Show, Eq) - -instance Arbitrary (Key AES128) where - arbitrary = Key <$> arbitraryBS 16 - -instance Arbitrary (Key AES256) where - arbitrary = Key <$> arbitraryBS 32 - -instance Arbitrary Nonce where - arbitrary = throwCryptoError . nonce <$> arbitraryBS 12 - -encDecTest - :: BlockCipher128 c - => Proxy c - -> Key c - -> Nonce - -> ArbitraryBS0_2901 - -> ArbitraryBS0_2901 - -> Property -encDecTest prx (Key key) iv (ArbitraryBS0_2901 aad) (ArbitraryBS0_2901 input) = - let c = throwCryptoError (cipherInit key) `asProxyTypeOf` prx - (tag, ciphertext) = encrypt c iv aad input - in decrypt c iv aad ciphertext tag === Just input - -tests :: TestTree -tests = - testGroup - "AES-GCM-SIV" - [ testGroup - "KATs" - [ katTests "encrypt" makeEncryptionTest - , katTests "decrypt" makeDecryptionTest - ] - , testGroup - "properties" - [ testProperty "AES128" $ encDecTest (Proxy :: Proxy AES128) - , testProperty "AES256" $ encDecTest (Proxy :: Proxy AES256) - ] - ] diff --git a/tests/KAT_AFIS.hs b/tests/KAT_AFIS.hs deleted file mode 100644 --- a/tests/KAT_AFIS.hs +++ /dev/null @@ -1,55 +0,0 @@ -{-# LANGUAGE ExistentialQuantification #-} -{-# LANGUAGE OverloadedStrings #-} - -module KAT_AFIS (tests) where - -import Imports - -import qualified Crypto.Data.AFIS as AFIS -import Crypto.Hash -import Crypto.Random -import qualified Data.ByteString as B - -mergeVec :: [(Int, SHA1, B.ByteString, B.ByteString)] -mergeVec = - [ - ( 3 - , SHA1 - , "\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02\x02" - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\xd4\x76\xc8\x58\xbd\xf0\x15\xbe\x9f\x40\xe3\x65\x20\x1c\x9c\xb8\xd8\x1c\x16\x64" - ) - , - ( 3 - , SHA1 - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17" - , "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\xd6\x75\xc8\x59\xbb\xf7\x11\xbb\x95\x4b\xeb\x6c\x2e\x13\x90\xb5\xca\x0f\x06\x75\x17\x70\x39\x28" - ) - ] - -mergeKATs = zipWith toProp mergeVec [(0 :: Int) ..] - where - toProp (nbExpands, hashAlg, expected, dat) i = - testCase ("merge " ++ show i) (expected @=? AFIS.merge hashAlg nbExpands dat) - -data AFISParams = AFISParams B.ByteString Int SHA1 ChaChaDRG - -instance Show AFISParams where - show (AFISParams dat expand _ _) = "data: " ++ show dat ++ " expanded: " ++ show expand - -instance Arbitrary AFISParams where - arbitrary = - AFISParams - <$> arbitraryBSof 3 46 - <*> choose (2, 2) - <*> elements [SHA1] - <*> arbitrary - -instance Arbitrary ChaChaDRG where - arbitrary = drgNewTest <$> arbitrary - -tests = - testGroup - "AFIS" - [ testGroup "KAT merge" mergeKATs - , testProperty "merge.split == id" $ \(AFISParams bs e hf rng) -> bs == (AFIS.merge hf e $ fst (AFIS.split hf rng e bs)) - ] diff --git a/tests/KAT_Argon2.hs b/tests/KAT_Argon2.hs deleted file mode 100644 --- a/tests/KAT_Argon2.hs +++ /dev/null @@ -1,52 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Argon2 (tests) where - -import Crypto.Error -import qualified Crypto.KDF.Argon2 as Argon2 -import qualified Data.ByteString as B -import Imports - -data KDFVector = KDFVector - { kdfPass :: ByteString - , kdfSalt :: ByteString - , kdfOptions :: Argon2.Options - , kdfResult :: ByteString - } - -argon2i_13 :: Argon2.TimeCost -> Argon2.MemoryCost -> Argon2.Options -argon2i_13 iters memory = - Argon2.Options - { Argon2.iterations = iters - , Argon2.memory = memory - , Argon2.parallelism = 1 - , Argon2.variant = Argon2.Argon2i - , Argon2.version = Argon2.Version13 - } - -vectors = - [ KDFVector - "password" - "somesalt" - (argon2i_13 2 65536) - "\xc1\x62\x88\x32\x14\x7d\x97\x20\xc5\xbd\x1c\xfd\x61\x36\x70\x78\x72\x9f\x6d\xfb\x6f\x8f\xea\x9f\xf9\x81\x58\xe0\xd7\x81\x6e\xd0" - ] - -kdfTests :: [TestTree] -kdfTests = zipWith toKDFTest is vectors - where - toKDFTest i v = - testCase - (show i) - ( CryptoPassed (kdfResult v) - @=? Argon2.hash (kdfOptions v) (kdfPass v) (kdfSalt v) (B.length $ kdfResult v) - ) - - is :: [Int] - is = [1 ..] - -tests = - testGroup - "Argon2" - [ testGroup "KATs" kdfTests - ] diff --git a/tests/KAT_Blake2.hs b/tests/KAT_Blake2.hs deleted file mode 100644 --- a/tests/KAT_Blake2.hs +++ /dev/null @@ -1,203 +0,0 @@ -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RecordWildCards #-} - -module KAT_Blake2 (tests) where - -import Crypto.Hash (digestFromByteString) -import Crypto.Hash.Algorithms -import qualified Crypto.MAC.KeyedBlake2 as KB - -import qualified Data.ByteString as B - -import Imports - -data MACVector hash = MACVector - { macMessage :: ByteString - , macKey :: ByteString - , macResult :: KB.KeyedBlake2 hash - } - -instance Show (KB.KeyedBlake2 hash) where - show (KB.KeyedBlake2 d) = show d - -digest :: KB.HashBlake2 hash => ByteString -> KB.KeyedBlake2 hash -digest = maybe (error "cannot get digest") KB.KeyedBlake2 . digestFromByteString - --- From: https://github.com/BLAKE2/BLAKE2/blob/master/testvectors/ -vectorsBlake2bKAT :: [MACVector (Blake2b 512)] -vectorsBlake2bKAT = - [ MACVector - { macMessage = "" - , macKey = fixedKey - , macResult = - digest - "\x10\xeb\xb6\x77\x00\xb1\x86\x8e\xfb\x44\x17\x98\x7a\xcf\x46\x90\xae\x9d\x97\x2f\xb7\xa5\x90\xc2\xf0\x28\x71\x79\x9a\xaa\x47\x86\xb5\xe9\x96\xe8\xf0\xf4\xeb\x98\x1f\xc2\x14\xb0\x05\xf4\x2d\x2f\xf4\x23\x34\x99\x39\x16\x53\xdf\x7a\xef\xcb\xc1\x3f\xc5\x15\x68" - } - , MACVector - { macMessage = "\x00" - , macKey = fixedKey - , macResult = - digest - "\x96\x1f\x6d\xd1\xe4\xdd\x30\xf6\x39\x01\x69\x0c\x51\x2e\x78\xe4\xb4\x5e\x47\x42\xed\x19\x7c\x3c\x5e\x45\xc5\x49\xfd\x25\xf2\xe4\x18\x7b\x0b\xc9\xfe\x30\x49\x2b\x16\xb0\xd0\xbc\x4e\xf9\xb0\xf3\x4c\x70\x03\xfa\xc0\x9a\x5e\xf1\x53\x2e\x69\x43\x02\x34\xce\xbd" - } - , MACVector - { macMessage = B.pack [0x00 .. 0xfe] - , macKey = fixedKey - , macResult = - digest - "\x14\x27\x09\xd6\x2e\x28\xfc\xcc\xd0\xaf\x97\xfa\xd0\xf8\x46\x5b\x97\x1e\x82\x20\x1d\xc5\x10\x70\xfa\xa0\x37\x2a\xa4\x3e\x92\x48\x4b\xe1\xc1\xe7\x3b\xa1\x09\x06\xd5\xd1\x85\x3d\xb6\xa4\x10\x6e\x0a\x7b\xf9\x80\x0d\x37\x3d\x6d\xee\x2d\x46\xd6\x2e\xf2\xa4\x61" - } - ] - where - fixedKey = B.pack [0x00 .. 0x3f] - -vectorsBlake2bpKAT :: [MACVector (Blake2bp 512)] -vectorsBlake2bpKAT = - [ MACVector - { macMessage = "" - , macKey = fixedKey - , macResult = - digest - "\x9d\x94\x61\x07\x3e\x4e\xb6\x40\xa2\x55\x35\x7b\x83\x9f\x39\x4b\x83\x8c\x6f\xf5\x7c\x9b\x68\x6a\x3f\x76\x10\x7c\x10\x66\x72\x8f\x3c\x99\x56\xbd\x78\x5c\xbc\x3b\xf7\x9d\xc2\xab\x57\x8c\x5a\x0c\x06\x3b\x9d\x9c\x40\x58\x48\xde\x1d\xbe\x82\x1c\xd0\x5c\x94\x0a" - } - , MACVector - { macMessage = "\x00" - , macKey = fixedKey - , macResult = - digest - "\xff\x8e\x90\xa3\x7b\x94\x62\x39\x32\xc5\x9f\x75\x59\xf2\x60\x35\x02\x9c\x37\x67\x32\xcb\x14\xd4\x16\x02\x00\x1c\xbb\x73\xad\xb7\x92\x93\xa2\xdb\xda\x5f\x60\x70\x30\x25\x14\x4d\x15\x8e\x27\x35\x52\x95\x96\x25\x1c\x73\xc0\x34\x5c\xa6\xfc\xcb\x1f\xb1\xe9\x7e" - } - , MACVector - { macMessage = B.pack [0x00 .. 0xfe] - , macKey = fixedKey - , macResult = - digest - "\x96\xfb\xcb\xb6\x0b\xd3\x13\xb8\x84\x50\x33\xe5\xbc\x05\x8a\x38\x02\x74\x38\x57\x2d\x7e\x79\x57\xf3\x68\x4f\x62\x68\xaa\xdd\x3a\xd0\x8d\x21\x76\x7e\xd6\x87\x86\x85\x33\x1b\xa9\x85\x71\x48\x7e\x12\x47\x0a\xad\x66\x93\x26\x71\x6e\x46\x66\x7f\x69\xf8\xd7\xe8" - } - ] - where - fixedKey = B.pack [0x00 .. 0x3f] - -vectorsBlake2sKAT :: [MACVector (Blake2s 256)] -vectorsBlake2sKAT = - [ MACVector - { macMessage = "" - , macKey = fixedKey - , macResult = - digest - "\x48\xa8\x99\x7d\xa4\x07\x87\x6b\x3d\x79\xc0\xd9\x23\x25\xad\x3b\x89\xcb\xb7\x54\xd8\x6a\xb7\x1a\xee\x04\x7a\xd3\x45\xfd\x2c\x49" - } - , MACVector - { macMessage = "\x00" - , macKey = fixedKey - , macResult = - digest - "\x40\xd1\x5f\xee\x7c\x32\x88\x30\x16\x6a\xc3\xf9\x18\x65\x0f\x80\x7e\x7e\x01\xe1\x77\x25\x8c\xdc\x0a\x39\xb1\x1f\x59\x80\x66\xf1" - } - , MACVector - { macMessage = B.pack [0x00 .. 0xfe] - , macKey = fixedKey - , macResult = - digest - "\x3f\xb7\x35\x06\x1a\xbc\x51\x9d\xfe\x97\x9e\x54\xc1\xee\x5b\xfa\xd0\xa9\xd8\x58\xb3\x31\x5b\xad\x34\xbd\xe9\x99\xef\xd7\x24\xdd" - } - ] - where - fixedKey = B.pack [0x00 .. 0x1f] - -vectorsBlake2spKAT :: [MACVector (Blake2sp 256)] -vectorsBlake2spKAT = - [ MACVector - { macMessage = "" - , macKey = fixedKey - , macResult = - digest - "\x71\x5c\xb1\x38\x95\xae\xb6\x78\xf6\x12\x41\x60\xbf\xf2\x14\x65\xb3\x0f\x4f\x68\x74\x19\x3f\xc8\x51\xb4\x62\x10\x43\xf0\x9c\xc6" - } - , MACVector - { macMessage = "\x00" - , macKey = fixedKey - , macResult = - digest - "\x40\x57\x8f\xfa\x52\xbf\x51\xae\x18\x66\xf4\x28\x4d\x3a\x15\x7f\xc1\xbc\xd3\x6a\xc1\x3c\xbd\xcb\x03\x77\xe4\xd0\xcd\x0b\x66\x03" - } - , MACVector - { macMessage = B.pack [0x00 .. 0xfe] - , macKey = fixedKey - , macResult = - digest - "\x0c\x8a\x36\x59\x7d\x74\x61\xc6\x3a\x94\x73\x28\x21\xc9\x41\x85\x6c\x66\x83\x76\x60\x6c\x86\xa5\x2d\xe0\xee\x41\x04\xc6\x15\xdb" - } - ] - where - fixedKey = B.pack [0x00 .. 0x1f] - -macTests :: [TestTree] -macTests = - [ testGroup "Blake2b_512" (concatMap toMACTest $ zip is vectorsBlake2bKAT) - , testGroup "Blake2bp_512" (concatMap toMACTest $ zip is vectorsBlake2bpKAT) - , testGroup "Blake2s_512" (concatMap toMACTest $ zip is vectorsBlake2sKAT) - , testGroup "Blake2sp_512" (concatMap toMACTest $ zip is vectorsBlake2spKAT) - ] - where - toMACTest (i, MACVector{..}) = - [ testCase (show i) (macResult @=? KB.keyedBlake2 macKey macMessage) - , testCase - ("incr-" ++ show i) - ( macResult - @=? KB.finalize (KB.update (KB.initialize macKey) macMessage) - ) - ] - is :: [Int] - is = [1 ..] - -data MacIncremental a = MacIncremental ByteString ByteString (KB.KeyedBlake2 a) - deriving (Show, Eq) - -instance KB.HashBlake2 a => Arbitrary (MacIncremental a) where - arbitrary = do - key <- arbitraryBSof 32 64 - msg <- arbitraryBSof 1 99 - return $ MacIncremental key msg (KB.keyedBlake2 key msg) - -data MacIncrementalList a - = MacIncrementalList ByteString [ByteString] (KB.KeyedBlake2 a) - deriving (Show, Eq) - -instance KB.HashBlake2 a => Arbitrary (MacIncrementalList a) where - arbitrary = do - key <- arbitraryBSof 32 64 - msgs <- choose (1, 20) >>= \n -> replicateM n (arbitraryBSof 1 99) - return $ MacIncrementalList key msgs (KB.keyedBlake2 key (B.concat msgs)) - -macIncrementalTests :: [TestTree] -macIncrementalTests = - [ testIncrProperties "Blake2b_512" (Blake2b :: Blake2b 512) - , testIncrProperties "Blake2bp_512" (Blake2bp :: Blake2bp 512) - , testIncrProperties "Blake2s_256" (Blake2s :: Blake2s 256) - , testIncrProperties "Blake2sp_256" (Blake2sp :: Blake2sp 256) - ] - where - testIncrProperties :: KB.HashBlake2 a => TestName -> a -> TestTree - testIncrProperties name a = - testGroup - name - [ testProperty "list-one" (prop_inc0 a) - , testProperty "list-multi" (prop_inc1 a) - ] - - prop_inc0 :: KB.HashBlake2 a => a -> MacIncremental a -> Bool - prop_inc0 _ (MacIncremental secret msg result) = - result `assertEq` KB.finalize (KB.update (KB.initialize secret) msg) - - prop_inc1 :: KB.HashBlake2 a => a -> MacIncrementalList a -> Bool - prop_inc1 _ (MacIncrementalList secret msgs result) = - result `assertEq` KB.finalize (foldl' KB.update (KB.initialize secret) msgs) - -tests = - testGroup - "Blake2" - [ testGroup "KATs" macTests - , testGroup "properties" macIncrementalTests - ] diff --git a/tests/KAT_Blowfish.hs b/tests/KAT_Blowfish.hs deleted file mode 100644 --- a/tests/KAT_Blowfish.hs +++ /dev/null @@ -1,151 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Blowfish where - -import BlockCipher -import Crypto.Cipher.Blowfish -import Imports () - -vectors_ecb = - -- key plaintext cipher - [ KAT_ECB - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x4E\xF9\x97\x45\x61\x98\xDD\x78" - , KAT_ECB - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x51\x86\x6F\xD5\xB8\x5E\xCB\x8A" - , KAT_ECB - "\x30\x00\x00\x00\x00\x00\x00\x00" - "\x10\x00\x00\x00\x00\x00\x00\x01" - "\x7D\x85\x6F\x9A\x61\x30\x63\xF2" - , KAT_ECB - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x24\x66\xDD\x87\x8B\x96\x3C\x9D" - , KAT_ECB - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x61\xF9\xC3\x80\x22\x81\xB0\x96" - , KAT_ECB - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x7D\x0C\xC6\x30\xAF\xDA\x1E\xC7" - , KAT_ECB - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x4E\xF9\x97\x45\x61\x98\xDD\x78" - , KAT_ECB - "\xFE\xDC\xBA\x98\x76\x54\x32\x10" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x0A\xCE\xAB\x0F\xC6\xA0\xA2\x8D" - , KAT_ECB - "\x7C\xA1\x10\x45\x4A\x1A\x6E\x57" - "\x01\xA1\xD6\xD0\x39\x77\x67\x42" - "\x59\xC6\x82\x45\xEB\x05\x28\x2B" - , KAT_ECB - "\x01\x31\xD9\x61\x9D\xC1\x37\x6E" - "\x5C\xD5\x4C\xA8\x3D\xEF\x57\xDA" - "\xB1\xB8\xCC\x0B\x25\x0F\x09\xA0" - , KAT_ECB - "\x07\xA1\x13\x3E\x4A\x0B\x26\x86" - "\x02\x48\xD4\x38\x06\xF6\x71\x72" - "\x17\x30\xE5\x77\x8B\xEA\x1D\xA4" - , KAT_ECB - "\x38\x49\x67\x4C\x26\x02\x31\x9E" - "\x51\x45\x4B\x58\x2D\xDF\x44\x0A" - "\xA2\x5E\x78\x56\xCF\x26\x51\xEB" - , KAT_ECB - "\x04\xB9\x15\xBA\x43\xFE\xB5\xB6" - "\x42\xFD\x44\x30\x59\x57\x7F\xA2" - "\x35\x38\x82\xB1\x09\xCE\x8F\x1A" - , KAT_ECB - "\x01\x13\xB9\x70\xFD\x34\xF2\xCE" - "\x05\x9B\x5E\x08\x51\xCF\x14\x3A" - "\x48\xF4\xD0\x88\x4C\x37\x99\x18" - , KAT_ECB - "\x01\x70\xF1\x75\x46\x8F\xB5\xE6" - "\x07\x56\xD8\xE0\x77\x47\x61\xD2" - "\x43\x21\x93\xB7\x89\x51\xFC\x98" - , KAT_ECB - "\x43\x29\x7F\xAD\x38\xE3\x73\xFE" - "\x76\x25\x14\xB8\x29\xBF\x48\x6A" - "\x13\xF0\x41\x54\xD6\x9D\x1A\xE5" - , KAT_ECB - "\x07\xA7\x13\x70\x45\xDA\x2A\x16" - "\x3B\xDD\x11\x90\x49\x37\x28\x02" - "\x2E\xED\xDA\x93\xFF\xD3\x9C\x79" - , KAT_ECB - "\x04\x68\x91\x04\xC2\xFD\x3B\x2F" - "\x26\x95\x5F\x68\x35\xAF\x60\x9A" - "\xD8\x87\xE0\x39\x3C\x2D\xA6\xE3" - , KAT_ECB - "\x37\xD0\x6B\xB5\x16\xCB\x75\x46" - "\x16\x4D\x5E\x40\x4F\x27\x52\x32" - "\x5F\x99\xD0\x4F\x5B\x16\x39\x69" - , KAT_ECB - "\x1F\x08\x26\x0D\x1A\xC2\x46\x5E" - "\x6B\x05\x6E\x18\x75\x9F\x5C\xCA" - "\x4A\x05\x7A\x3B\x24\xD3\x97\x7B" - , KAT_ECB - "\x58\x40\x23\x64\x1A\xBA\x61\x76" - "\x00\x4B\xD6\xEF\x09\x17\x60\x62" - "\x45\x20\x31\xC1\xE4\xFA\xDA\x8E" - , KAT_ECB - "\x02\x58\x16\x16\x46\x29\xB0\x07" - "\x48\x0D\x39\x00\x6E\xE7\x62\xF2" - "\x75\x55\xAE\x39\xF5\x9B\x87\xBD" - , KAT_ECB - "\x49\x79\x3E\xBC\x79\xB3\x25\x8F" - "\x43\x75\x40\xC8\x69\x8F\x3C\xFA" - "\x53\xC5\x5F\x9C\xB4\x9F\xC0\x19" - , KAT_ECB - "\x4F\xB0\x5E\x15\x15\xAB\x73\xA7" - "\x07\x2D\x43\xA0\x77\x07\x52\x92" - "\x7A\x8E\x7B\xFA\x93\x7E\x89\xA3" - , KAT_ECB - "\x49\xE9\x5D\x6D\x4C\xA2\x29\xBF" - "\x02\xFE\x55\x77\x81\x17\xF1\x2A" - "\xCF\x9C\x5D\x7A\x49\x86\xAD\xB5" - , KAT_ECB - "\x01\x83\x10\xDC\x40\x9B\x26\xD6" - "\x1D\x9D\x5C\x50\x18\xF7\x28\xC2" - "\xD1\xAB\xB2\x90\x65\x8B\xC7\x78" - , KAT_ECB - "\x1C\x58\x7F\x1C\x13\x92\x4F\xEF" - "\x30\x55\x32\x28\x6D\x6F\x29\x5A" - "\x55\xCB\x37\x74\xD1\x3E\xF2\x01" - , KAT_ECB - "\x01\x01\x01\x01\x01\x01\x01\x01" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xFA\x34\xEC\x48\x47\xB2\x68\xB2" - , KAT_ECB - "\x1F\x1F\x1F\x1F\x0E\x0E\x0E\x0E" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xA7\x90\x79\x51\x08\xEA\x3C\xAE" - , KAT_ECB - "\xE0\xFE\xE0\xFE\xF1\xFE\xF1\xFE" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xC3\x9E\x07\x2D\x9F\xAC\x63\x1D" - , KAT_ECB - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x01\x49\x33\xE0\xCD\xAF\xF6\xE4" - , KAT_ECB - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xF2\x1E\x9A\x77\xB7\x1C\x49\xBC" - , KAT_ECB - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x24\x59\x46\x88\x57\x54\x36\x9A" - , KAT_ECB - "\xFE\xDC\xBA\x98\x76\x54\x32\x10" - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x6B\x5C\x5A\x9C\x5D\x9E\x0A\x5A" - ] - -kats = defaultKATs{kat_ECB = vectors_ecb} - -tests = testBlockCipher kats (undefined :: Blowfish64) diff --git a/tests/KAT_CAST5.hs b/tests/KAT_CAST5.hs deleted file mode 100644 --- a/tests/KAT_CAST5.hs +++ /dev/null @@ -1,26 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_CAST5 (tests) where - -import BlockCipher -import qualified Crypto.Cipher.CAST5 as CAST5 - -vectors_ecb = - -- key plaintext ciphertext - [ KAT_ECB - "\x01\x23\x45\x67\x12\x34\x56\x78\x23\x45\x67\x89\x34\x56\x78\x9A" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x23\x8B\x4F\xE5\x84\x7E\x44\xB2" - , KAT_ECB - "\x01\x23\x45\x67\x12\x34\x56\x78\x23\x45" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xEB\x6A\x71\x1A\x2C\x02\x27\x1B" - , KAT_ECB - "\x01\x23\x45\x67\x12" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x7A\xC8\x16\xD1\x6E\x9B\x30\x2E" - ] - -kats = defaultKATs{kat_ECB = vectors_ecb} - -tests = testBlockCipher kats (undefined :: CAST5.CAST5) diff --git a/tests/KAT_CMAC.hs b/tests/KAT_CMAC.hs deleted file mode 100644 --- a/tests/KAT_CMAC.hs +++ /dev/null @@ -1,217 +0,0 @@ -module KAT_CMAC (tests) where - -import Crypto.Cipher.AES (AES128, AES192, AES256) -import Crypto.Cipher.TripleDES (DES_EDE2, DES_EDE3) -import Crypto.Cipher.Types ( - BlockCipher, - Cipher, - blockSize, - cipherInit, - ecbEncrypt, - ) -import Crypto.Error (eitherCryptoError) -import qualified Crypto.MAC.CMAC as CMAC - -import Imports - -import qualified Data.ByteArray as B -import qualified Data.ByteString as BS -import Data.Char (digitToInt) - -hxs :: String -> ByteString -hxs = BS.pack . rec' - where - dtoW8 = fromIntegral . digitToInt - rec' (' ' : xs) = rec' xs - rec' (x : y : xs) = dtoW8 x * 16 + dtoW8 y : rec' xs - rec' [_] = error "hxs: invalid hex pattern." - rec' [] = [] - -unsafeCipher :: Cipher k => ByteString -> k -unsafeCipher = either (error . show) id . eitherCryptoError . cipherInit - -ecb0 :: BlockCipher k => k -> ByteString -ecb0 k = ecbEncrypt k $ BS.replicate (blockSize k) 0 - -{- Test vectors from NIST data-sheet - (AES128-CMAC, AES192-CMAC, AES256-CMAC, Three Key TDEA, Two Key TDEA) - http://csrc.nist.gov/publications/nistpubs/800-38B/Updated_CMAC_Examples.pdf - The data of AES128-CMAC is same as them in RFC4493. - -} - -msg512 :: ByteString -msg512 = - hxs $ - "6bc1bee2 2e409f96 e93d7e11 7393172a" - ++ "ae2d8a57 1e03ac9c 9eb76fac 45af8e51" - ++ "30c81c46 a35ce411 e5fbc119 1a0a52ef" - ++ "f69f2445 df4f9b17 ad2b417b e66c3710" - -msg320 :: ByteString -msg320 = BS.take 40 msg512 - -msg256 :: ByteString -msg256 = BS.take 32 msg512 - -msg160 :: ByteString -msg160 = BS.take 20 msg512 - -msg128 :: ByteString -msg128 = BS.take 16 msg512 - -msg64 :: ByteString -msg64 = BS.take 8 msg512 - -msg0 :: ByteString -msg0 = BS.empty - -bsCMAC :: BlockCipher k => k -> ByteString -> ByteString -bsCMAC k = B.convert . CMAC.cmac k - -gAES128 :: TestTree -gAES128 = - igroup - "aes128" - [ ecb0 aes128key @?= hxs "7df76b0c 1ab899b3 3e42f047 b91b546f" - , aes128k1 @?= hxs "fbeed618 35713366 7c85e08f 7236a8de" - , aes128k2 @?= hxs "f7ddac30 6ae266cc f90bc11e e46d513b" - , bsCMAC aes128key msg0 - @?= hxs "bb1d6929 e9593728 7fa37d12 9b756746" - , bsCMAC aes128key msg128 - @?= hxs "070a16b4 6b4d4144 f79bdd9d d04a287c" - , bsCMAC aes128key msg320 - @?= hxs "dfa66747 de9ae630 30ca3261 1497c827" - , bsCMAC aes128key msg512 - @?= hxs "51f0bebf 7e3b9d92 fc497417 79363cfe" - ] - where - aes128key :: AES128 - aes128key = - unsafeCipher $ - hxs - "2b7e1516 28aed2a6 abf71588 09cf4f3c" - - aes128k1, aes128k2 :: ByteString - (aes128k1, aes128k2) = CMAC.subKeys aes128key - -gAES192 :: TestTree -gAES192 = - igroup - "aes192" - [ ecb0 aes192key @?= hxs "22452d8e 49a8a593 9f7321ce ea6d514b" - , aes192k1 @?= hxs "448a5b1c 93514b27 3ee6439d d4daa296" - , aes192k2 @?= hxs "8914b639 26a2964e 7dcc873b a9b5452c" - , bsCMAC aes192key msg0 - @?= hxs "d17ddf46 adaacde5 31cac483 de7a9367" - , bsCMAC aes192key msg128 - @?= hxs "9e99a7bf 31e71090 0662f65e 617c5184" - , bsCMAC aes192key msg320 - @?= hxs "8a1de5be 2eb31aad 089a82e6 ee908b0e" - , bsCMAC aes192key msg512 - @?= hxs "a1d5df0e ed790f79 4d775896 59f39a11" - ] - where - aes192key :: AES192 - aes192key = - unsafeCipher . hxs $ - "8e73b0f7 da0e6452 c810f32b 809079e5" - ++ "62f8ead2 522c6b7b" - - aes192k1, aes192k2 :: ByteString - (aes192k1, aes192k2) = CMAC.subKeys aes192key - -gAES256 :: TestTree -gAES256 = - igroup - "aes256" - [ ecb0 aes256key @?= hxs "e568f681 94cf76d6 174d4cc0 4310a854" - , aes256k1 @?= hxs "cad1ed03 299eedac 2e9a9980 8621502f" - , aes256k2 @?= hxs "95a3da06 533ddb58 5d353301 0c42a0d9" - , bsCMAC aes256key msg0 - @?= hxs "028962f6 1b7bf89e fc6b551f 4667d983" - , bsCMAC aes256key msg128 - @?= hxs "28a7023f 452e8f82 bd4bf28d 8c37c35c" - , bsCMAC aes256key msg320 - @?= hxs "aaf3d8f1 de5640c2 32f5b169 b9c911e6" - , bsCMAC aes256key msg512 - @?= hxs "e1992190 549f6ed5 696a2c05 6c315410" - ] - where - aes256key :: AES256 - aes256key = - unsafeCipher . hxs $ - "603deb10 15ca71be 2b73aef0 857d7781" - ++ "1f352c07 3b6108d7 2d9810a3 0914dff4" - - aes256k1, aes256k2 :: ByteString - (aes256k1, aes256k2) = CMAC.subKeys aes256key - -gTDEA3 :: TestTree -gTDEA3 = - igroup - "Three Key TDEA" - [ ecb0 tdea3key @?= hxs "c8cc74e9 8a7329a2" - , tdea3k1 @?= hxs "9198e9d3 14e6535f" - , tdea3k2 @?= hxs "2331d3a6 29cca6a5" - , bsCMAC tdea3key msg0 - @?= hxs "b7a688e1 22ffaf95" - , bsCMAC tdea3key msg64 - @?= hxs "8e8f2931 36283797" - , bsCMAC tdea3key msg160 - @?= hxs "743ddbe0 ce2dc2ed" - , bsCMAC tdea3key msg256 - @?= hxs "33e6b109 2400eae5" - ] - where - tdea3key :: DES_EDE3 - tdea3key = - unsafeCipher . hxs $ - "8aa83bf8 cbda1062" - ++ "0bc1bf19 fbb6cd58" - ++ "bc313d4a 371ca8b5" - - tdea3k1, tdea3k2 :: ByteString - (tdea3k1, tdea3k2) = CMAC.subKeys tdea3key - -gTDEA2 :: TestTree -gTDEA2 = - igroup - "Two Key TDEA" - [ ecb0 tdea2key @?= hxs "c7679b9f 6b8d7d7a" - , tdea2k1 @?= hxs "8ecf373e d71afaef" - , tdea2k2 @?= hxs "1d9e6e7d ae35f5c5" - , bsCMAC tdea2key msg0 - @?= hxs "bd2ebf9a 3ba00361" - , bsCMAC tdea2key msg64 - @?= hxs "4ff2ab81 3c53ce83" - , bsCMAC tdea2key msg160 - @?= hxs "62dd1b47 1902bd4e" - , bsCMAC tdea2key msg256 - @?= hxs "31b1e431 dabc4eb8" - ] - where - tdea2key :: DES_EDE2 - tdea2key = - unsafeCipher . hxs $ - "4cf15134 a2850dd5" - ++ "8a3d10ba 80570d38" - - tdea2k1, tdea2k2 :: ByteString - (tdea2k1, tdea2k2) = CMAC.subKeys tdea2key - -igroup :: TestName -> [Assertion] -> TestTree -igroup nm = testGroup nm . zipWith (flip ($)) [1 ..] . map icase - where - icase c i = testCase (show (i :: Int)) c - -nistVectors :: TestTree -nistVectors = - testGroup - "KAT - NIST test vectors" - [gAES128, gAES192, gAES256, gTDEA3, gTDEA2] - -tests :: TestTree -tests = - testGroup - "CMAC" - [nistVectors] diff --git a/tests/KAT_Camellia.hs b/tests/KAT_Camellia.hs deleted file mode 100644 --- a/tests/KAT_Camellia.hs +++ /dev/null @@ -1,247 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE ViewPatterns #-} -{-# OPTIONS_GHC -fno-warn-unused-binds #-} -{-# OPTIONS_GHC -fno-warn-unused-matches #-} - -module KAT_Camellia (tests) where - -import BlockCipher -import Imports () - -import Crypto.Cipher.Camellia -import qualified Data.ByteString as B - -vectors_camellia128 = - [ KAT_ECB - (B.replicate 16 0) - (B.replicate 16 0) - ( B.pack - [ 0x3d - , 0x02 - , 0x80 - , 0x25 - , 0xb1 - , 0x56 - , 0x32 - , 0x7c - , 0x17 - , 0xf7 - , 0x62 - , 0xc1 - , 0xf2 - , 0xcb - , 0xca - , 0x71 - ] - ) - , KAT_ECB - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xab - , 0xcd - , 0xef - , 0xfe - , 0xdc - , 0xba - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - ] - ) - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xab - , 0xcd - , 0xef - , 0xfe - , 0xdc - , 0xba - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - ] - ) - ( B.pack - [ 0x67 - , 0x67 - , 0x31 - , 0x38 - , 0x54 - , 0x96 - , 0x69 - , 0x73 - , 0x08 - , 0x57 - , 0x06 - , 0x56 - , 0x48 - , 0xea - , 0xbe - , 0x43 - ] - ) - ] - -vectors_camellia192 = - [ KAT_ECB - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xab - , 0xcd - , 0xef - , 0xfe - , 0xdc - , 0xba - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - , 0x00 - , 0x11 - , 0x22 - , 0x33 - , 0x44 - , 0x55 - , 0x66 - , 0x77 - ] - ) - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xab - , 0xcd - , 0xef - , 0xfe - , 0xdc - , 0xba - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - ] - ) - ( B.pack - [ 0xb4 - , 0x99 - , 0x34 - , 0x01 - , 0xb3 - , 0xe9 - , 0x96 - , 0xf8 - , 0x4e - , 0xe5 - , 0xce - , 0xe7 - , 0xd7 - , 0x9b - , 0x09 - , 0xb9 - ] - ) - ] - -vectors_camellia256 = - [ KAT_ECB - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xab - , 0xcd - , 0xef - , 0xfe - , 0xdc - , 0xba - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - , 0x00 - , 0x11 - , 0x22 - , 0x33 - , 0x44 - , 0x55 - , 0x66 - , 0x77 - , 0x88 - , 0x99 - , 0xaa - , 0xbb - , 0xcc - , 0xdd - , 0xee - , 0xff - ] - ) - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xab - , 0xcd - , 0xef - , 0xfe - , 0xdc - , 0xba - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - ] - ) - ( B.pack - [ 0x9a - , 0xcc - , 0x23 - , 0x7d - , 0xff - , 0x16 - , 0xd7 - , 0x6c - , 0x20 - , 0xef - , 0x7c - , 0x91 - , 0x9e - , 0x3a - , 0x75 - , 0x09 - ] - ) - ] - -kats128 = defaultKATs{kat_ECB = vectors_camellia128} -kats192 = defaultKATs{kat_ECB = vectors_camellia192} -kats256 = defaultKATs{kat_ECB = vectors_camellia256} - -tests = testBlockCipher kats128 (undefined :: Camellia128) diff --git a/tests/KAT_Curve25519.hs b/tests/KAT_Curve25519.hs deleted file mode 100644 --- a/tests/KAT_Curve25519.hs +++ /dev/null @@ -1,50 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Curve25519 (tests) where - -import Crypto.Error -import qualified Crypto.PubKey.Curve25519 as Curve25519 -import Data.ByteArray as B -import Imports - -alicePrivate = - throwCryptoError $ - Curve25519.secretKey - ( "\x77\x07\x6d\x0a\x73\x18\xa5\x7d\x3c\x16\xc1\x72\x51\xb2\x66\x45\xdf\x4c\x2f\x87\xeb\xc0\x99\x2a\xb1\x77\xfb\xa5\x1d\xb9\x2c\x2a" - :: ByteString - ) -alicePublic = - throwCryptoError $ - Curve25519.publicKey - ( "\x85\x20\xf0\x09\x89\x30\xa7\x54\x74\x8b\x7d\xdc\xb4\x3e\xf7\x5a\x0d\xbf\x3a\x0d\x26\x38\x1a\xf4\xeb\xa4\xa9\x8e\xaa\x9b\x4e\x6a" - :: ByteString - ) -bobPrivate = - throwCryptoError $ - Curve25519.secretKey - ( "\x5d\xab\x08\x7e\x62\x4a\x8a\x4b\x79\xe1\x7f\x8b\x83\x80\x0e\xe6\x6f\x3b\xb1\x29\x26\x18\xb6\xfd\x1c\x2f\x8b\x27\xff\x88\xe0\xeb" - :: ByteString - ) -bobPublic = - throwCryptoError $ - Curve25519.publicKey - ( "\xde\x9e\xdb\x7d\x7b\x7d\xc1\xb4\xd3\x5b\x61\xc2\xec\xe4\x35\x37\x3f\x83\x43\xc8\x5b\x78\x67\x4d\xad\xfc\x7e\x14\x6f\x88\x2b\x4f" - :: ByteString - ) -aliceMultBob = - "\x4a\x5d\x9d\x5b\xa4\xce\x2d\xe1\x72\x8e\x3b\xf4\x80\x35\x0f\x25\xe0\x7e\x21\xc9\x47\xd1\x9e\x33\x76\xf0\x9b\x3c\x1e\x16\x17\x42" - :: ByteString - -katTests :: [TestTree] -katTests = - [ testCase "0" (aliceMultBob @=? B.convert (Curve25519.dh alicePublic bobPrivate)) - , testCase "1" (aliceMultBob @=? B.convert (Curve25519.dh bobPublic alicePrivate)) - , testCase "2" (alicePublic @=? Curve25519.toPublic alicePrivate) - , testCase "3" (bobPublic @=? Curve25519.toPublic bobPrivate) - ] - -tests = - testGroup - "Curve25519" - [ testGroup "KATs" katTests - ] diff --git a/tests/KAT_Curve448.hs b/tests/KAT_Curve448.hs deleted file mode 100644 --- a/tests/KAT_Curve448.hs +++ /dev/null @@ -1,50 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Curve448 (tests) where - -import Crypto.Error -import qualified Crypto.PubKey.Curve448 as Curve448 -import Data.ByteArray as B -import Imports - -alicePrivate = - throwCryptoError $ - Curve448.secretKey - ( "\x9a\x8f\x49\x25\xd1\x51\x9f\x57\x75\xcf\x46\xb0\x4b\x58\x00\xd4\xee\x9e\xe8\xba\xe8\xbc\x55\x65\xd4\x98\xc2\x8d\xd9\xc9\xba\xf5\x74\xa9\x41\x97\x44\x89\x73\x91\x00\x63\x82\xa6\xf1\x27\xab\x1d\x9a\xc2\xd8\xc0\xa5\x98\x72\x6b" - :: ByteString - ) -alicePublic = - throwCryptoError $ - Curve448.publicKey - ( "\x9b\x08\xf7\xcc\x31\xb7\xe3\xe6\x7d\x22\xd5\xae\xa1\x21\x07\x4a\x27\x3b\xd2\xb8\x3d\xe0\x9c\x63\xfa\xa7\x3d\x2c\x22\xc5\xd9\xbb\xc8\x36\x64\x72\x41\xd9\x53\xd4\x0c\x5b\x12\xda\x88\x12\x0d\x53\x17\x7f\x80\xe5\x32\xc4\x1f\xa0" - :: ByteString - ) -bobPrivate = - throwCryptoError $ - Curve448.secretKey - ( "\x1c\x30\x6a\x7a\xc2\xa0\xe2\xe0\x99\x0b\x29\x44\x70\xcb\xa3\x39\xe6\x45\x37\x72\xb0\x75\x81\x1d\x8f\xad\x0d\x1d\x69\x27\xc1\x20\xbb\x5e\xe8\x97\x2b\x0d\x3e\x21\x37\x4c\x9c\x92\x1b\x09\xd1\xb0\x36\x6f\x10\xb6\x51\x73\x99\x2d" - :: ByteString - ) -bobPublic = - throwCryptoError $ - Curve448.publicKey - ( "\x3e\xb7\xa8\x29\xb0\xcd\x20\xf5\xbc\xfc\x0b\x59\x9b\x6f\xec\xcf\x6d\xa4\x62\x71\x07\xbd\xb0\xd4\xf3\x45\xb4\x30\x27\xd8\xb9\x72\xfc\x3e\x34\xfb\x42\x32\xa1\x3c\xa7\x06\xdc\xb5\x7a\xec\x3d\xae\x07\xbd\xc1\xc6\x7b\xf3\x36\x09" - :: ByteString - ) -aliceMultBob = - "\x07\xff\xf4\x18\x1a\xc6\xcc\x95\xec\x1c\x16\xa9\x4a\x0f\x74\xd1\x2d\xa2\x32\xce\x40\xa7\x75\x52\x28\x1d\x28\x2b\xb6\x0c\x0b\x56\xfd\x24\x64\xc3\x35\x54\x39\x36\x52\x1c\x24\x40\x30\x85\xd5\x9a\x44\x9a\x50\x37\x51\x4a\x87\x9d" - :: ByteString - -katTests :: [TestTree] -katTests = - [ testCase "0" (aliceMultBob @=? B.convert (Curve448.dh alicePublic bobPrivate)) - , testCase "1" (aliceMultBob @=? B.convert (Curve448.dh bobPublic alicePrivate)) - , testCase "2" (alicePublic @=? Curve448.toPublic alicePrivate) - , testCase "3" (bobPublic @=? Curve448.toPublic bobPrivate) - ] - -tests = - testGroup - "Curve448" - [ testGroup "KATs" katTests - ] diff --git a/tests/KAT_DES.hs b/tests/KAT_DES.hs deleted file mode 100644 --- a/tests/KAT_DES.hs +++ /dev/null @@ -1,154 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE ViewPatterns #-} - -module KAT_DES (tests) where - -import BlockCipher -import qualified Crypto.Cipher.DES as DES -import Imports - -vectors_ecb = - -- key plaintext ciphertext - [ KAT_ECB - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x8C\xA6\x4D\xE9\xC1\xB1\x23\xA7" - , KAT_ECB - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x73\x59\xB2\x16\x3E\x4E\xDC\x58" - , KAT_ECB - "\x30\x00\x00\x00\x00\x00\x00\x00" - "\x10\x00\x00\x00\x00\x00\x00\x01" - "\x95\x8E\x6E\x62\x7A\x05\x55\x7B" - , KAT_ECB - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\xF4\x03\x79\xAB\x9E\x0E\xC5\x33" - , KAT_ECB - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x17\x66\x8D\xFC\x72\x92\x53\x2D" - , KAT_ECB - "\x11\x11\x11\x11\x11\x11\x11\x11" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x8A\x5A\xE1\xF8\x1A\xB8\xF2\xDD" - , KAT_ECB - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\x8C\xA6\x4D\xE9\xC1\xB1\x23\xA7" - , KAT_ECB - "\xFE\xDC\xBA\x98\x76\x54\x32\x10" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xED\x39\xD9\x50\xFA\x74\xBC\xC4" - , KAT_ECB - "\x7C\xA1\x10\x45\x4A\x1A\x6E\x57" - "\x01\xA1\xD6\xD0\x39\x77\x67\x42" - "\x69\x0F\x5B\x0D\x9A\x26\x93\x9B" - , KAT_ECB - "\x01\x31\xD9\x61\x9D\xC1\x37\x6E" - "\x5C\xD5\x4C\xA8\x3D\xEF\x57\xDA" - "\x7A\x38\x9D\x10\x35\x4B\xD2\x71" - , KAT_ECB - "\x07\xA1\x13\x3E\x4A\x0B\x26\x86" - "\x02\x48\xD4\x38\x06\xF6\x71\x72" - "\x86\x8E\xBB\x51\xCA\xB4\x59\x9A" - , KAT_ECB - "\x38\x49\x67\x4C\x26\x02\x31\x9E" - "\x51\x45\x4B\x58\x2D\xDF\x44\x0A" - "\x71\x78\x87\x6E\x01\xF1\x9B\x2A" - , KAT_ECB - "\x04\xB9\x15\xBA\x43\xFE\xB5\xB6" - "\x42\xFD\x44\x30\x59\x57\x7F\xA2" - "\xAF\x37\xFB\x42\x1F\x8C\x40\x95" - , KAT_ECB - "\x01\x13\xB9\x70\xFD\x34\xF2\xCE" - "\x05\x9B\x5E\x08\x51\xCF\x14\x3A" - "\x86\xA5\x60\xF1\x0E\xC6\xD8\x5B" - , KAT_ECB - "\x01\x70\xF1\x75\x46\x8F\xB5\xE6" - "\x07\x56\xD8\xE0\x77\x47\x61\xD2" - "\x0C\xD3\xDA\x02\x00\x21\xDC\x09" - , KAT_ECB - "\x43\x29\x7F\xAD\x38\xE3\x73\xFE" - "\x76\x25\x14\xB8\x29\xBF\x48\x6A" - "\xEA\x67\x6B\x2C\xB7\xDB\x2B\x7A" - , KAT_ECB - "\x07\xA7\x13\x70\x45\xDA\x2A\x16" - "\x3B\xDD\x11\x90\x49\x37\x28\x02" - "\xDF\xD6\x4A\x81\x5C\xAF\x1A\x0F" - , KAT_ECB - "\x04\x68\x91\x04\xC2\xFD\x3B\x2F" - "\x26\x95\x5F\x68\x35\xAF\x60\x9A" - "\x5C\x51\x3C\x9C\x48\x86\xC0\x88" - , KAT_ECB - "\x37\xD0\x6B\xB5\x16\xCB\x75\x46" - "\x16\x4D\x5E\x40\x4F\x27\x52\x32" - "\x0A\x2A\xEE\xAE\x3F\xF4\xAB\x77" - , KAT_ECB - "\x1F\x08\x26\x0D\x1A\xC2\x46\x5E" - "\x6B\x05\x6E\x18\x75\x9F\x5C\xCA" - "\xEF\x1B\xF0\x3E\x5D\xFA\x57\x5A" - , KAT_ECB - "\x58\x40\x23\x64\x1A\xBA\x61\x76" - "\x00\x4B\xD6\xEF\x09\x17\x60\x62" - "\x88\xBF\x0D\xB6\xD7\x0D\xEE\x56" - , KAT_ECB - "\x02\x58\x16\x16\x46\x29\xB0\x07" - "\x48\x0D\x39\x00\x6E\xE7\x62\xF2" - "\xA1\xF9\x91\x55\x41\x02\x0B\x56" - , KAT_ECB - "\x49\x79\x3E\xBC\x79\xB3\x25\x8F" - "\x43\x75\x40\xC8\x69\x8F\x3C\xFA" - "\x6F\xBF\x1C\xAF\xCF\xFD\x05\x56" - , KAT_ECB - "\x4F\xB0\x5E\x15\x15\xAB\x73\xA7" - "\x07\x2D\x43\xA0\x77\x07\x52\x92" - "\x2F\x22\xE4\x9B\xAB\x7C\xA1\xAC" - , KAT_ECB - "\x49\xE9\x5D\x6D\x4C\xA2\x29\xBF" - "\x02\xFE\x55\x77\x81\x17\xF1\x2A" - "\x5A\x6B\x61\x2C\xC2\x6C\xCE\x4A" - , KAT_ECB - "\x01\x83\x10\xDC\x40\x9B\x26\xD6" - "\x1D\x9D\x5C\x50\x18\xF7\x28\xC2" - "\x5F\x4C\x03\x8E\xD1\x2B\x2E\x41" - , KAT_ECB - "\x1C\x58\x7F\x1C\x13\x92\x4F\xEF" - "\x30\x55\x32\x28\x6D\x6F\x29\x5A" - "\x63\xFA\xC0\xD0\x34\xD9\xF7\x93" - , KAT_ECB - "\x01\x01\x01\x01\x01\x01\x01\x01" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x61\x7B\x3A\x0C\xE8\xF0\x71\x00" - , KAT_ECB - "\x1F\x1F\x1F\x1F\x0E\x0E\x0E\x0E" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xDB\x95\x86\x05\xF8\xC8\xC6\x06" - , KAT_ECB - "\xE0\xFE\xE0\xFE\xF1\xFE\xF1\xFE" - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\xED\xBF\xD1\xC6\x6C\x29\xCC\xC7" - , KAT_ECB - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x35\x55\x50\xB2\x15\x0E\x24\x51" - , KAT_ECB - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xCA\xAA\xAF\x4D\xEA\xF1\xDB\xAE" - , KAT_ECB - "\x01\x23\x45\x67\x89\xAB\xCD\xEF" - "\x00\x00\x00\x00\x00\x00\x00\x00" - "\xD5\xD4\x4F\xF7\x20\x68\x3D\x0D" - , KAT_ECB - "\xFE\xDC\xBA\x98\x76\x54\x32\x10" - "\xFF\xFF\xFF\xFF\xFF\xFF\xFF\xFF" - "\x2A\x2B\xB0\x08\xDF\x97\xC2\xF2" - ] - -kats = defaultKATs{kat_ECB = vectors_ecb} - -tests = - localOption (QuickCheckTests 5) $ - testBlockCipher kats (undefined :: DES.DES) diff --git a/tests/KAT_Ed25519.hs b/tests/KAT_Ed25519.hs deleted file mode 100644 --- a/tests/KAT_Ed25519.hs +++ /dev/null @@ -1,91 +0,0 @@ -{-# LANGUAGE BangPatterns #-} -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Ed25519 (tests) where - -import Crypto.Error -import qualified Crypto.PubKey.Ed25519 as Ed25519 -import Imports - -data Vec = Vec - { vecSec :: ByteString - , vecPub :: ByteString - , vecMsg :: ByteString - , vecSig :: ByteString - } - deriving (Show, Eq) - -vectors = - [ Vec - { vecSec = - "\x9d\x61\xb1\x9d\xef\xfd\x5a\x60\xba\x84\x4a\xf4\x92\xec\x2c\xc4\x44\x49\xc5\x69\x7b\x32\x69\x19\x70\x3b\xac\x03\x1c\xae\x7f\x60" - , vecPub = - "\xd7\x5a\x98\x01\x82\xb1\x0a\xb7\xd5\x4b\xfe\xd3\xc9\x64\x07\x3a\x0e\xe1\x72\xf3\xda\xa6\x23\x25\xaf\x02\x1a\x68\xf7\x07\x51\x1a" - , vecMsg = "" - , vecSig = - "\xe5\x56\x43\x00\xc3\x60\xac\x72\x90\x86\xe2\xcc\x80\x6e\x82\x8a\x84\x87\x7f\x1e\xb8\xe5\xd9\x74\xd8\x73\xe0\x65\x22\x49\x01\x55\x5f\xb8\x82\x15\x90\xa3\x3b\xac\xc6\x1e\x39\x70\x1c\xf9\xb4\x6b\xd2\x5b\xf5\xf0\x59\x5b\xbe\x24\x65\x51\x41\x43\x8e\x7a\x10\x0b" - } - , Vec - { vecSec = - "\x4c\xcd\x08\x9b\x28\xff\x96\xda\x9d\xb6\xc3\x46\xec\x11\x4e\x0f\x5b\x8a\x31\x9f\x35\xab\xa6\x24\xda\x8c\xf6\xed\x4f\xb8\xa6\xfb" - , vecPub = - "\x3d\x40\x17\xc3\xe8\x43\x89\x5a\x92\xb7\x0a\xa7\x4d\x1b\x7e\xbc\x9c\x98\x2c\xcf\x2e\xc4\x96\x8c\xc0\xcd\x55\xf1\x2a\xf4\x66\x0c" - , vecMsg = "\x72" - , vecSig = - "\x92\xa0\x09\xa9\xf0\xd4\xca\xb8\x72\x0e\x82\x0b\x5f\x64\x25\x40\xa2\xb2\x7b\x54\x16\x50\x3f\x8f\xb3\x76\x22\x23\xeb\xdb\x69\xda\x08\x5a\xc1\xe4\x3e\x15\x99\x6e\x45\x8f\x36\x13\xd0\xf1\x1d\x8c\x38\x7b\x2e\xae\xb4\x30\x2a\xee\xb0\x0d\x29\x16\x12\xbb\x0c\x00" - } - , Vec - { vecSec = - "\xc5\xaa\x8d\xf4\x3f\x9f\x83\x7b\xed\xb7\x44\x2f\x31\xdc\xb7\xb1\x66\xd3\x85\x35\x07\x6f\x09\x4b\x85\xce\x3a\x2e\x0b\x44\x58\xf7" - , vecPub = - "\xfc\x51\xcd\x8e\x62\x18\xa1\xa3\x8d\xa4\x7e\xd0\x02\x30\xf0\x58\x08\x16\xed\x13\xba\x33\x03\xac\x5d\xeb\x91\x15\x48\x90\x80\x25" - , vecMsg = "\xaf\x82" - , vecSig = - "\x62\x91\xd6\x57\xde\xec\x24\x02\x48\x27\xe6\x9c\x3a\xbe\x01\xa3\x0c\xe5\x48\xa2\x84\x74\x3a\x44\x5e\x36\x80\xd7\xdb\x5a\xc3\xac\x18\xff\x9b\x53\x8d\x16\xf2\x90\xae\x67\xf7\x60\x98\x4d\xc6\x59\x4a\x7c\x15\xe9\x71\x6e\xd2\x8d\xc0\x27\xbe\xce\xea\x1e\xc4\x0a" - } - , Vec - { vecSec = - "\xf5\xe5\x76\x7c\xf1\x53\x31\x95\x17\x63\x0f\x22\x68\x76\xb8\x6c\x81\x60\xcc\x58\x3b\xc0\x13\x74\x4c\x6b\xf2\x55\xf5\xcc\x0e\xe5" - , vecPub = - "\x27\x81\x17\xfc\x14\x4c\x72\x34\x0f\x67\xd0\xf2\x31\x6e\x83\x86\xce\xff\xbf\x2b\x24\x28\xc9\xc5\x1f\xef\x7c\x59\x7f\x1d\x42\x6e" - , vecMsg = - "\x08\xb8\xb2\xb7\x33\x42\x42\x43\x76\x0f\xe4\x26\xa4\xb5\x49\x08\x63\x21\x10\xa6\x6c\x2f\x65\x91\xea\xbd\x33\x45\xe3\xe4\xeb\x98\xfa\x6e\x26\x4b\xf0\x9e\xfe\x12\xee\x50\xf8\xf5\x4e\x9f\x77\xb1\xe3\x55\xf6\xc5\x05\x44\xe2\x3f\xb1\x43\x3d\xdf\x73\xbe\x84\xd8\x79\xde\x7c\x00\x46\xdc\x49\x96\xd9\xe7\x73\xf4\xbc\x9e\xfe\x57\x38\x82\x9a\xdb\x26\xc8\x1b\x37\xc9\x3a\x1b\x27\x0b\x20\x32\x9d\x65\x86\x75\xfc\x6e\xa5\x34\xe0\x81\x0a\x44\x32\x82\x6b\xf5\x8c\x94\x1e\xfb\x65\xd5\x7a\x33\x8b\xbd\x2e\x26\x64\x0f\x89\xff\xbc\x1a\x85\x8e\xfc\xb8\x55\x0e\xe3\xa5\xe1\x99\x8b\xd1\x77\xe9\x3a\x73\x63\xc3\x44\xfe\x6b\x19\x9e\xe5\xd0\x2e\x82\xd5\x22\xc4\xfe\xba\x15\x45\x2f\x80\x28\x8a\x82\x1a\x57\x91\x16\xec\x6d\xad\x2b\x3b\x31\x0d\xa9\x03\x40\x1a\xa6\x21\x00\xab\x5d\x1a\x36\x55\x3e\x06\x20\x3b\x33\x89\x0c\xc9\xb8\x32\xf7\x9e\xf8\x05\x60\xcc\xb9\xa3\x9c\xe7\x67\x96\x7e\xd6\x28\xc6\xad\x57\x3c\xb1\x16\xdb\xef\xef\xd7\x54\x99\xda\x96\xbd\x68\xa8\xa9\x7b\x92\x8a\x8b\xbc\x10\x3b\x66\x21\xfc\xde\x2b\xec\xa1\x23\x1d\x20\x6b\xe6\xcd\x9e\xc7\xaf\xf6\xf6\xc9\x4f\xcd\x72\x04\xed\x34\x55\xc6\x8c\x83\xf4\xa4\x1d\xa4\xaf\x2b\x74\xef\x5c\x53\xf1\xd8\xac\x70\xbd\xcb\x7e\xd1\x85\xce\x81\xbd\x84\x35\x9d\x44\x25\x4d\x95\x62\x9e\x98\x55\xa9\x4a\x7c\x19\x58\xd1\xf8\xad\xa5\xd0\x53\x2e\xd8\xa5\xaa\x3f\xb2\xd1\x7b\xa7\x0e\xb6\x24\x8e\x59\x4e\x1a\x22\x97\xac\xbb\xb3\x9d\x50\x2f\x1a\x8c\x6e\xb6\xf1\xce\x22\xb3\xde\x1a\x1f\x40\xcc\x24\x55\x41\x19\xa8\x31\xa9\xaa\xd6\x07\x9c\xad\x88\x42\x5d\xe6\xbd\xe1\xa9\x18\x7e\xbb\x60\x92\xcf\x67\xbf\x2b\x13\xfd\x65\xf2\x70\x88\xd7\x8b\x7e\x88\x3c\x87\x59\xd2\xc4\xf5\xc6\x5a\xdb\x75\x53\x87\x8a\xd5\x75\xf9\xfa\xd8\x78\xe8\x0a\x0c\x9b\xa6\x3b\xcb\xcc\x27\x32\xe6\x94\x85\xbb\xc9\xc9\x0b\xfb\xd6\x24\x81\xd9\x08\x9b\xec\xcf\x80\xcf\xe2\xdf\x16\xa2\xcf\x65\xbd\x92\xdd\x59\x7b\x07\x07\xe0\x91\x7a\xf4\x8b\xbb\x75\xfe\xd4\x13\xd2\x38\xf5\x55\x5a\x7a\x56\x9d\x80\xc3\x41\x4a\x8d\x08\x59\xdc\x65\xa4\x61\x28\xba\xb2\x7a\xf8\x7a\x71\x31\x4f\x31\x8c\x78\x2b\x23\xeb\xfe\x80\x8b\x82\xb0\xce\x26\x40\x1d\x2e\x22\xf0\x4d\x83\xd1\x25\x5d\xc5\x1a\xdd\xd3\xb7\x5a\x2b\x1a\xe0\x78\x45\x04\xdf\x54\x3a\xf8\x96\x9b\xe3\xea\x70\x82\xff\x7f\xc9\x88\x8c\x14\x4d\xa2\xaf\x58\x42\x9e\xc9\x60\x31\xdb\xca\xd3\xda\xd9\xaf\x0d\xcb\xaa\xaf\x26\x8c\xb8\xfc\xff\xea\xd9\x4f\x3c\x7c\xa4\x95\xe0\x56\xa9\xb4\x7a\xcd\xb7\x51\xfb\x73\xe6\x66\xc6\xc6\x55\xad\xe8\x29\x72\x97\xd0\x7a\xd1\xba\x5e\x43\xf1\xbc\xa3\x23\x01\x65\x13\x39\xe2\x29\x04\xcc\x8c\x42\xf5\x8c\x30\xc0\x4a\xaf\xdb\x03\x8d\xda\x08\x47\xdd\x98\x8d\xcd\xa6\xf3\xbf\xd1\x5c\x4b\x4c\x45\x25\x00\x4a\xa0\x6e\xef\xf8\xca\x61\x78\x3a\xac\xec\x57\xfb\x3d\x1f\x92\xb0\xfe\x2f\xd1\xa8\x5f\x67\x24\x51\x7b\x65\xe6\x14\xad\x68\x08\xd6\xf6\xee\x34\xdf\xf7\x31\x0f\xdc\x82\xae\xbf\xd9\x04\xb0\x1e\x1d\xc5\x4b\x29\x27\x09\x4b\x2d\xb6\x8d\x6f\x90\x3b\x68\x40\x1a\xde\xbf\x5a\x7e\x08\xd7\x8f\xf4\xef\x5d\x63\x65\x3a\x65\x04\x0c\xf9\xbf\xd4\xac\xa7\x98\x4a\x74\xd3\x71\x45\x98\x67\x80\xfc\x0b\x16\xac\x45\x16\x49\xde\x61\x88\xa7\xdb\xdf\x19\x1f\x64\xb5\xfc\x5e\x2a\xb4\x7b\x57\xf7\xf7\x27\x6c\xd4\x19\xc1\x7a\x3c\xa8\xe1\xb9\x39\xae\x49\xe4\x88\xac\xba\x6b\x96\x56\x10\xb5\x48\x01\x09\xc8\xb1\x7b\x80\xe1\xb7\xb7\x50\xdf\xc7\x59\x8d\x5d\x50\x11\xfd\x2d\xcc\x56\x00\xa3\x2e\xf5\xb5\x2a\x1e\xcc\x82\x0e\x30\x8a\xa3\x42\x72\x1a\xac\x09\x43\xbf\x66\x86\xb6\x4b\x25\x79\x37\x65\x04\xcc\xc4\x93\xd9\x7e\x6a\xed\x3f\xb0\xf9\xcd\x71\xa4\x3d\xd4\x97\xf0\x1f\x17\xc0\xe2\xcb\x37\x97\xaa\x2a\x2f\x25\x66\x56\x16\x8e\x6c\x49\x6a\xfc\x5f\xb9\x32\x46\xf6\xb1\x11\x63\x98\xa3\x46\xf1\xa6\x41\xf3\xb0\x41\xe9\x89\xf7\x91\x4f\x90\xcc\x2c\x7f\xff\x35\x78\x76\xe5\x06\xb5\x0d\x33\x4b\xa7\x7c\x22\x5b\xc3\x07\xba\x53\x71\x52\xf3\xf1\x61\x0e\x4e\xaf\xe5\x95\xf6\xd9\xd9\x0d\x11\xfa\xa9\x33\xa1\x5e\xf1\x36\x95\x46\x86\x8a\x7f\x3a\x45\xa9\x67\x68\xd4\x0f\xd9\xd0\x34\x12\xc0\x91\xc6\x31\x5c\xf4\xfd\xe7\xcb\x68\x60\x69\x37\x38\x0d\xb2\xea\xaa\x70\x7b\x4c\x41\x85\xc3\x2e\xdd\xcd\xd3\x06\x70\x5e\x4d\xc1\xff\xc8\x72\xee\xee\x47\x5a\x64\xdf\xac\x86\xab\xa4\x1c\x06\x18\x98\x3f\x87\x41\xc5\xef\x68\xd3\xa1\x01\xe8\xa3\xb8\xca\xc6\x0c\x90\x5c\x15\xfc\x91\x08\x40\xb9\x4c\x00\xa0\xb9\xd0" - , vecSig = - "\x0a\xab\x4c\x90\x05\x01\xb3\xe2\x4d\x7c\xdf\x46\x63\x32\x6a\x3a\x87\xdf\x5e\x48\x43\xb2\xcb\xdb\x67\xcb\xf6\xe4\x60\xfe\xc3\x50\xaa\x53\x71\xb1\x50\x8f\x9f\x45\x28\xec\xea\x23\xc4\x36\xd9\x4b\x5e\x8f\xcd\x4f\x68\x1e\x30\xa6\xac\x00\xa9\x70\x4a\x18\x8a\x03" - } - , Vec - { vecSec = - "\x83\x3f\xe6\x24\x09\x23\x7b\x9d\x62\xec\x77\x58\x75\x20\x91\x1e\x9a\x75\x9c\xec\x1d\x19\x75\x5b\x7d\xa9\x01\xb9\x6d\xca\x3d\x42" - , vecPub = - "\xec\x17\x2b\x93\xad\x5e\x56\x3b\xf4\x93\x2c\x70\xe1\x24\x50\x34\xc3\x54\x67\xef\x2e\xfd\x4d\x64\xeb\xf8\x19\x68\x34\x67\xe2\xbf" - , vecMsg = - "\xdd\xaf\x35\xa1\x93\x61\x7a\xba\xcc\x41\x73\x49\xae\x20\x41\x31\x12\xe6\xfa\x4e\x89\xa9\x7e\xa2\x0a\x9e\xee\xe6\x4b\x55\xd3\x9a\x21\x92\x99\x2a\x27\x4f\xc1\xa8\x36\xba\x3c\x23\xa3\xfe\xeb\xbd\x45\x4d\x44\x23\x64\x3c\xe8\x0e\x2a\x9a\xc9\x4f\xa5\x4c\xa4\x9f" - , vecSig = - "\xdc\x2a\x44\x59\xe7\x36\x96\x33\xa5\x2b\x1b\xf2\x77\x83\x9a\x00\x20\x10\x09\xa3\xef\xbf\x3e\xcb\x69\xbe\xa2\x18\x6c\x26\xb5\x89\x09\x35\x1f\xc9\xac\x90\xb3\xec\xfd\xfb\xc7\xc6\x64\x31\xe0\x30\x3d\xca\x17\x9c\x13\x8a\xc1\x7a\xd9\xbe\xf1\x17\x73\x31\xa7\x04" - } - ] - -doPublicKeyTest i vec = testCase (show i) (pub @=? Ed25519.toPublic sec) - where - !pub = throwCryptoError $ Ed25519.publicKey (vecPub vec) - !sec = throwCryptoError $ Ed25519.secretKey (vecSec vec) - -doSignatureTest i vec = testCase (show i) (sig @=? Ed25519.sign sec pub (vecMsg vec)) - where - !sig = throwCryptoError $ Ed25519.signature (vecSig vec) - !pub = throwCryptoError $ Ed25519.publicKey (vecPub vec) - !sec = throwCryptoError $ Ed25519.secretKey (vecSec vec) - -doVerifyTest i vec = testCase (show i) (True @=? Ed25519.verify pub (vecMsg vec) sig) - where - !sig = throwCryptoError $ Ed25519.signature (vecSig vec) - !pub = throwCryptoError $ Ed25519.publicKey (vecPub vec) - -tests = - testGroup - "Ed25519" - [ testCase "gen secretkey" (Ed25519.generateSecretKey *> pure ()) - , testGroup "gen publickey" $ zipWith doPublicKeyTest [katZero ..] vectors - , testGroup "gen signature" $ zipWith doSignatureTest [katZero ..] vectors - , testGroup "verify sig" $ zipWith doVerifyTest [katZero ..] vectors - ] diff --git a/tests/KAT_Ed448.hs b/tests/KAT_Ed448.hs deleted file mode 100644 --- a/tests/KAT_Ed448.hs +++ /dev/null @@ -1,119 +0,0 @@ -{-# LANGUAGE BangPatterns #-} -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Ed448 (tests) where - -import Crypto.Error -import qualified Crypto.PubKey.Ed448 as Ed448 -import Imports - -data Vec = Vec - { vecSec :: ByteString - , vecPub :: ByteString - , vecMsg :: ByteString - , vecSig :: ByteString - } - deriving (Show, Eq) - -vectors = - [ Vec - { vecSec = - "\x6c\x82\xa5\x62\xcb\x80\x8d\x10\xd6\x32\xbe\x89\xc8\x51\x3e\xbf\x6c\x92\x9f\x34\xdd\xfa\x8c\x9f\x63\xc9\x96\x0e\xf6\xe3\x48\xa3\x52\x8c\x8a\x3f\xcc\x2f\x04\x4e\x39\xa3\xfc\x5b\x94\x49\x2f\x8f\x03\x2e\x75\x49\xa2\x00\x98\xf9\x5b" - , vecPub = - "\x5f\xd7\x44\x9b\x59\xb4\x61\xfd\x2c\xe7\x87\xec\x61\x6a\xd4\x6a\x1d\xa1\x34\x24\x85\xa7\x0e\x1f\x8a\x0e\xa7\x5d\x80\xe9\x67\x78\xed\xf1\x24\x76\x9b\x46\xc7\x06\x1b\xd6\x78\x3d\xf1\xe5\x0f\x6c\xd1\xfa\x1a\xbe\xaf\xe8\x25\x61\x80" - , vecMsg = "" - , vecSig = - "\x53\x3a\x37\xf6\xbb\xe4\x57\x25\x1f\x02\x3c\x0d\x88\xf9\x76\xae\x2d\xfb\x50\x4a\x84\x3e\x34\xd2\x07\x4f\xd8\x23\xd4\x1a\x59\x1f\x2b\x23\x3f\x03\x4f\x62\x82\x81\xf2\xfd\x7a\x22\xdd\xd4\x7d\x78\x28\xc5\x9b\xd0\xa2\x1b\xfd\x39\x80\xff\x0d\x20\x28\xd4\xb1\x8a\x9d\xf6\x3e\x00\x6c\x5d\x1c\x2d\x34\x5b\x92\x5d\x8d\xc0\x0b\x41\x04\x85\x2d\xb9\x9a\xc5\xc7\xcd\xda\x85\x30\xa1\x13\xa0\xf4\xdb\xb6\x11\x49\xf0\x5a\x73\x63\x26\x8c\x71\xd9\x58\x08\xff\x2e\x65\x26\x00" - } - , Vec - { vecSec = - "\xc4\xea\xb0\x5d\x35\x70\x07\xc6\x32\xf3\xdb\xb4\x84\x89\x92\x4d\x55\x2b\x08\xfe\x0c\x35\x3a\x0d\x4a\x1f\x00\xac\xda\x2c\x46\x3a\xfb\xea\x67\xc5\xe8\xd2\x87\x7c\x5e\x3b\xc3\x97\xa6\x59\x94\x9e\xf8\x02\x1e\x95\x4e\x0a\x12\x27\x4e" - , vecPub = - "\x43\xba\x28\xf4\x30\xcd\xff\x45\x6a\xe5\x31\x54\x5f\x7e\xcd\x0a\xc8\x34\xa5\x5d\x93\x58\xc0\x37\x2b\xfa\x0c\x6c\x67\x98\xc0\x86\x6a\xea\x01\xeb\x00\x74\x28\x02\xb8\x43\x8e\xa4\xcb\x82\x16\x9c\x23\x51\x60\x62\x7b\x4c\x3a\x94\x80" - , vecMsg = "\x03" - , vecSig = - "\x26\xb8\xf9\x17\x27\xbd\x62\x89\x7a\xf1\x5e\x41\xeb\x43\xc3\x77\xef\xb9\xc6\x10\xd4\x8f\x23\x35\xcb\x0b\xd0\x08\x78\x10\xf4\x35\x25\x41\xb1\x43\xc4\xb9\x81\xb7\xe1\x8f\x62\xde\x8c\xcd\xf6\x33\xfc\x1b\xf0\x37\xab\x7c\xd7\x79\x80\x5e\x0d\xbc\xc0\xaa\xe1\xcb\xce\xe1\xaf\xb2\xe0\x27\xdf\x36\xbc\x04\xdc\xec\xbf\x15\x43\x36\xc1\x9f\x0a\xf7\xe0\xa6\x47\x29\x05\xe7\x99\xf1\x95\x3d\x2a\x0f\xf3\x34\x8a\xb2\x1a\xa4\xad\xaf\xd1\xd2\x34\x44\x1c\xf8\x07\xc0\x3a\x00" - } - , Vec - { vecSec = - "\xcd\x23\xd2\x4f\x71\x42\x74\xe7\x44\x34\x32\x37\xb9\x32\x90\xf5\x11\xf6\x42\x5f\x98\xe6\x44\x59\xff\x20\x3e\x89\x85\x08\x3f\xfd\xf6\x05\x00\x55\x3a\xbc\x0e\x05\xcd\x02\x18\x4b\xdb\x89\xc4\xcc\xd6\x7e\x18\x79\x51\x26\x7e\xb3\x28" - , vecPub = - "\xdc\xea\x9e\x78\xf3\x5a\x1b\xf3\x49\x9a\x83\x1b\x10\xb8\x6c\x90\xaa\xc0\x1c\xd8\x4b\x67\xa0\x10\x9b\x55\xa3\x6e\x93\x28\xb1\xe3\x65\xfc\xe1\x61\xd7\x1c\xe7\x13\x1a\x54\x3e\xa4\xcb\x5f\x7e\x9f\x1d\x8b\x00\x69\x64\x47\x00\x14\x00" - , vecMsg = "\x0c\x3e\x54\x40\x74\xec\x63\xb0\x26\x5e\x0c" - , vecSig = - "\x1f\x0a\x88\x88\xce\x25\xe8\xd4\x58\xa2\x11\x30\x87\x9b\x84\x0a\x90\x89\xd9\x99\xaa\xba\x03\x9e\xaf\x3e\x3a\xfa\x09\x0a\x09\xd3\x89\xdb\xa8\x2c\x4f\xf2\xae\x8a\xc5\xcd\xfb\x7c\x55\xe9\x4d\x5d\x96\x1a\x29\xfe\x01\x09\x94\x1e\x00\xb8\xdb\xde\xea\x6d\x3b\x05\x10\x68\xdf\x72\x54\xc0\xcd\xc1\x29\xcb\xe6\x2d\xb2\xdc\x95\x7d\xbb\x47\xb5\x1f\xd3\xf2\x13\xfb\x86\x98\xf0\x64\x77\x42\x50\xa5\x02\x89\x61\xc9\xbf\x8f\xfd\x97\x3f\xe5\xd5\xc2\x06\x49\x2b\x14\x0e\x00" - } - , Vec - { vecSec = - "\x25\x8c\xdd\x4a\xda\x32\xed\x9c\x9f\xf5\x4e\x63\x75\x6a\xe5\x82\xfb\x8f\xab\x2a\xc7\x21\xf2\xc8\xe6\x76\xa7\x27\x68\x51\x3d\x93\x9f\x63\xdd\xdb\x55\x60\x91\x33\xf2\x9a\xdf\x86\xec\x99\x29\xdc\xcb\x52\xc1\xc5\xfd\x2f\xf7\xe2\x1b" - , vecPub = - "\x3b\xa1\x6d\xa0\xc6\xf2\xcc\x1f\x30\x18\x77\x40\x75\x6f\x5e\x79\x8d\x6b\xc5\xfc\x01\x5d\x7c\x63\xcc\x95\x10\xee\x3f\xd4\x4a\xdc\x24\xd8\xe9\x68\xb6\xe4\x6e\x6f\x94\xd1\x9b\x94\x53\x61\x72\x6b\xd7\x5e\x14\x9e\xf0\x98\x17\xf5\x80" - , vecMsg = "\x64\xa6\x5f\x3c\xde\xdc\xdd\x66\x81\x1e\x29\x15" - , vecSig = - "\x7e\xee\xab\x7c\x4e\x50\xfb\x79\x9b\x41\x8e\xe5\xe3\x19\x7f\xf6\xbf\x15\xd4\x3a\x14\xc3\x43\x89\xb5\x9d\xd1\xa7\xb1\xb8\x5b\x4a\xe9\x04\x38\xac\xa6\x34\xbe\xa4\x5e\x3a\x26\x95\xf1\x27\x0f\x07\xfd\xcd\xf7\xc6\x2b\x8e\xfe\xaf\x00\xb4\x5c\x2c\x96\xba\x45\x7e\xb1\xa8\xbf\x07\x5a\x3d\xb2\x8e\x5c\x24\xf6\xb9\x23\xed\x4a\xd7\x47\xc3\xc9\xe0\x3c\x70\x79\xef\xb8\x7c\xb1\x10\xd3\xa9\x98\x61\xe7\x20\x03\xcb\xae\x6d\x6b\x8b\x82\x7e\x4e\x6c\x14\x30\x64\xff\x3c\x00" - } - , Vec - { vecSec = - "\x7e\xf4\xe8\x45\x44\x23\x67\x52\xfb\xb5\x6b\x8f\x31\xa2\x3a\x10\xe4\x28\x14\xf5\xf5\x5c\xa0\x37\xcd\xcc\x11\xc6\x4c\x9a\x3b\x29\x49\xc1\xbb\x60\x70\x03\x14\x61\x17\x32\xa6\xc2\xfe\xa9\x8e\xeb\xc0\x26\x6a\x11\xa9\x39\x70\x10\x0e" - , vecPub = - "\xb3\xda\x07\x9b\x0a\xa4\x93\xa5\x77\x20\x29\xf0\x46\x7b\xae\xbe\xe5\xa8\x11\x2d\x9d\x3a\x22\x53\x23\x61\xda\x29\x4f\x7b\xb3\x81\x5c\x5d\xc5\x9e\x17\x6b\x4d\x9f\x38\x1c\xa0\x93\x8e\x13\xc6\xc0\x7b\x17\x4b\xe6\x5d\xfa\x57\x8e\x80" - , vecMsg = "\x64\xa6\x5f\x3c\xde\xdc\xdd\x66\x81\x1e\x29\x15\xe7" - , vecSig = - "\x6a\x12\x06\x6f\x55\x33\x1b\x6c\x22\xac\xd5\xd5\xbf\xc5\xd7\x12\x28\xfb\xda\x80\xae\x8d\xec\x26\xbd\xd3\x06\x74\x3c\x50\x27\xcb\x48\x90\x81\x0c\x16\x2c\x02\x74\x68\x67\x5e\xcf\x64\x5a\x83\x17\x6c\x0d\x73\x23\xa2\xcc\xde\x2d\x80\xef\xe5\xa1\x26\x8e\x8a\xca\x1d\x6f\xbc\x19\x4d\x3f\x77\xc4\x49\x86\xeb\x4a\xb4\x17\x79\x19\xad\x8b\xec\x33\xeb\x47\xbb\xb5\xfc\x6e\x28\x19\x6f\xd1\xca\xf5\x6b\x4e\x7e\x0b\xa5\x51\x92\x34\xd0\x47\x15\x5a\xc7\x27\xa1\x05\x31\x00" - } - , Vec - { vecSec = - "\xd6\x5d\xf3\x41\xad\x13\xe0\x08\x56\x76\x88\xba\xed\xda\x8e\x9d\xcd\xc1\x7d\xc0\x24\x97\x4e\xa5\xb4\x22\x7b\x65\x30\xe3\x39\xbf\xf2\x1f\x99\xe6\x8c\xa6\x96\x8f\x3c\xca\x6d\xfe\x0f\xb9\xf4\xfa\xb4\xfa\x13\x5d\x55\x42\xea\x3f\x01" - , vecPub = - "\xdf\x97\x05\xf5\x8e\xdb\xab\x80\x2c\x7f\x83\x63\xcf\xe5\x56\x0a\xb1\xc6\x13\x2c\x20\xa9\xf1\xdd\x16\x34\x83\xa2\x6f\x8a\xc5\x3a\x39\xd6\x80\x8b\xf4\xa1\xdf\xbd\x26\x1b\x09\x9b\xb0\x3b\x3f\xb5\x09\x06\xcb\x28\xbd\x8a\x08\x1f\x00" - , vecMsg = - "\xbd\x0f\x6a\x37\x47\xcd\x56\x1b\xdd\xdf\x46\x40\xa3\x32\x46\x1a\x4a\x30\xa1\x2a\x43\x4c\xd0\xbf\x40\xd7\x66\xd9\xc6\xd4\x58\xe5\x51\x22\x04\xa3\x0c\x17\xd1\xf5\x0b\x50\x79\x63\x1f\x64\xeb\x31\x12\x18\x2d\xa3\x00\x58\x35\x46\x11\x13\x71\x8d\x1a\x5e\xf9\x44" - , vecSig = - "\x55\x4b\xc2\x48\x08\x60\xb4\x9e\xab\x85\x32\xd2\xa5\x33\xb7\xd5\x78\xef\x47\x3e\xeb\x58\xc9\x8b\xb2\xd0\xe1\xce\x48\x8a\x98\xb1\x8d\xfd\xe9\xb9\xb9\x07\x75\xe6\x7f\x47\xd4\xa1\xc3\x48\x20\x58\xef\xc9\xf4\x0d\x2c\xa0\x33\xa0\x80\x1b\x63\xd4\x5b\x3b\x72\x2e\xf5\x52\xba\xd3\xb4\xcc\xb6\x67\xda\x35\x01\x92\xb6\x1c\x50\x8c\xf7\xb6\xb5\xad\xad\xc2\xc8\xd9\xa4\x46\xef\x00\x3f\xb0\x5c\xba\x5f\x30\xe8\x8e\x36\xec\x27\x03\xb3\x49\xca\x22\x9c\x26\x70\x83\x39\x00" - } - , Vec - { vecSec = - "\x2e\xc5\xfe\x3c\x17\x04\x5a\xbd\xb1\x36\xa5\xe6\xa9\x13\xe3\x2a\xb7\x5a\xe6\x8b\x53\xd2\xfc\x14\x9b\x77\xe5\x04\x13\x2d\x37\x56\x9b\x7e\x76\x6b\xa7\x4a\x19\xbd\x61\x62\x34\x3a\x21\xc8\x59\x0a\xa9\xce\xbc\xa9\x01\x4c\x63\x6d\xf5" - , vecPub = - "\x79\x75\x6f\x01\x4d\xcf\xe2\x07\x9f\x5d\xd9\xe7\x18\xbe\x41\x71\xe2\xef\x24\x86\xa0\x8f\x25\x18\x6f\x6b\xff\x43\xa9\x93\x6b\x9b\xfe\x12\x40\x2b\x08\xae\x65\x79\x8a\x3d\x81\xe2\x2e\x9e\xc8\x0e\x76\x90\x86\x2e\xf3\xd4\xed\x3a\x00" - , vecMsg = - "\x15\x77\x75\x32\xb0\xbd\xd0\xd1\x38\x9f\x63\x6c\x5f\x6b\x9b\xa7\x34\xc9\x0a\xf5\x72\x87\x7e\x2d\x27\x2d\xd0\x78\xaa\x1e\x56\x7c\xfa\x80\xe1\x29\x28\xbb\x54\x23\x30\xe8\x40\x9f\x31\x74\x50\x41\x07\xec\xd5\xef\xac\x61\xae\x75\x04\xda\xbe\x2a\x60\x2e\xde\x89\xe5\xcc\xa6\x25\x7a\x7c\x77\xe2\x7a\x70\x2b\x3a\xe3\x9f\xc7\x69\xfc\x54\xf2\x39\x5a\xe6\xa1\x17\x8c\xab\x47\x38\xe5\x43\x07\x2f\xc1\xc1\x77\xfe\x71\xe9\x2e\x25\xbf\x03\xe4\xec\xb7\x2f\x47\xb6\x4d\x04\x65\xaa\xea\x4c\x7f\xad\x37\x25\x36\xc8\xba\x51\x6a\x60\x39\xc3\xc2\xa3\x9f\x0e\x4d\x83\x2b\xe4\x32\xdf\xa9\xa7\x06\xa6\xe5\xc7\xe1\x9f\x39\x79\x64\xca\x42\x58\x00\x2f\x7c\x05\x41\xb5\x90\x31\x6d\xbc\x56\x22\xb6\xb2\xa6\xfe\x7a\x4a\xbf\xfd\x96\x10\x5e\xca\x76\xea\x7b\x98\x81\x6a\xf0\x74\x8c\x10\xdf\x04\x8c\xe0\x12\xd9\x01\x01\x5a\x51\xf1\x89\xf3\x88\x81\x45\xc0\x36\x50\xaa\x23\xce\x89\x4c\x3b\xd8\x89\xe0\x30\xd5\x65\x07\x1c\x59\xf4\x09\xa9\x98\x1b\x51\x87\x8f\xd6\xfc\x11\x06\x24\xdc\xbc\xde\x0b\xf7\xa6\x9c\xcc\xe3\x8f\xab\xdf\x86\xf3\xbe\xf6\x04\x48\x19\xde\x11" - , vecSig = - "\xc6\x50\xdd\xbb\x06\x01\xc1\x9c\xa1\x14\x39\xe1\x64\x0d\xd9\x31\xf4\x3c\x51\x8e\xa5\xbe\xa7\x0d\x3d\xcd\xe5\xf4\x19\x1f\xe5\x3f\x00\xcf\x96\x65\x46\xb7\x2b\xcc\x7d\x58\xbe\x2b\x9b\xad\xef\x28\x74\x39\x54\xe3\xa4\x4a\x23\xf8\x80\xe8\xd4\xf1\xcf\xce\x2d\x7a\x61\x45\x2d\x26\xda\x05\x89\x6f\x0a\x50\xda\x66\xa2\x39\xa8\xa1\x88\xb6\xd8\x25\xb3\x30\x5a\xd7\x7b\x73\xfb\xac\x08\x36\xec\xc6\x09\x87\xfd\x08\x52\x7c\x1a\x8e\x80\xd5\x82\x3e\x65\xca\xfe\x2a\x3d\x00" - } - , Vec - { vecSec = - "\x87\x2d\x09\x37\x80\xf5\xd3\x73\x0d\xf7\xc2\x12\x66\x4b\x37\xb8\xa0\xf2\x4f\x56\x81\x0d\xaa\x83\x82\xcd\x4f\xa3\xf7\x76\x34\xec\x44\xdc\x54\xf1\xc2\xed\x9b\xea\x86\xfa\xfb\x76\x32\xd8\xbe\x19\x9e\xa1\x65\xf5\xad\x55\xdd\x9c\xe8" - , vecPub = - "\xa8\x1b\x2e\x8a\x70\xa5\xac\x94\xff\xdb\xcc\x9b\xad\xfc\x3f\xeb\x08\x01\xf2\x58\x57\x8b\xb1\x14\xad\x44\xec\xe1\xec\x0e\x79\x9d\xa0\x8e\xff\xb8\x1c\x5d\x68\x5c\x0c\x56\xf6\x4e\xec\xae\xf8\xcd\xf1\x1c\xc3\x87\x37\x83\x8c\xf4\x00" - , vecMsg = - "\x6d\xdf\x80\x2e\x1a\xae\x49\x86\x93\x5f\x7f\x98\x1b\xa3\xf0\x35\x1d\x62\x73\xc0\xa0\xc2\x2c\x9c\x0e\x83\x39\x16\x8e\x67\x54\x12\xa3\xde\xbf\xaf\x43\x5e\xd6\x51\x55\x80\x07\xdb\x43\x84\xb6\x50\xfc\xc0\x7e\x3b\x58\x6a\x27\xa4\xf7\xa0\x0a\xc8\xa6\xfe\xc2\xcd\x86\xae\x4b\xf1\x57\x0c\x41\xe6\xa4\x0c\x93\x1d\xb2\x7b\x2f\xaa\x15\xa8\xce\xdd\x52\xcf\xf7\x36\x2c\x4e\x6e\x23\xda\xec\x0f\xbc\x3a\x79\xb6\x80\x6e\x31\x6e\xfc\xc7\xb6\x81\x19\xbf\x46\xbc\x76\xa2\x60\x67\xa5\x3f\x29\x6d\xaf\xdb\xdc\x11\xc7\x7f\x77\x77\xe9\x72\x66\x0c\xf4\xb6\xa9\xb3\x69\xa6\x66\x5f\x02\xe0\xcc\x9b\x6e\xdf\xad\x13\x6b\x4f\xab\xe7\x23\xd2\x81\x3d\xb3\x13\x6c\xfd\xe9\xb6\xd0\x44\x32\x2f\xee\x29\x47\x95\x2e\x03\x1b\x73\xab\x5c\x60\x33\x49\xb3\x07\xbd\xc2\x7b\xc6\xcb\x8b\x8b\xbd\x7b\xd3\x23\x21\x9b\x80\x33\xa5\x81\xb5\x9e\xad\xeb\xb0\x9b\x3c\x4f\x3d\x22\x77\xd4\xf0\x34\x36\x24\xac\xc8\x17\x80\x47\x28\xb2\x5a\xb7\x97\x17\x2b\x4c\x5c\x21\xa2\x2f\x9c\x78\x39\xd6\x43\x00\x23\x2e\xb6\x6e\x53\xf3\x1c\x72\x3f\xa3\x7f\xe3\x87\xc7\xd3\xe5\x0b\xdf\x98\x13\xa3\x0e\x5b\xb1\x2c\xf4\xcd\x93\x0c\x40\xcf\xb4\xe1\xfc\x62\x25\x92\xa4\x95\x88\x79\x44\x94\xd5\x6d\x24\xea\x4b\x40\xc8\x9f\xc0\x59\x6c\xc9\xeb\xb9\x61\xc8\xcb\x10\xad\xde\x97\x6a\x5d\x60\x2b\x1c\x3f\x85\xb9\xb9\xa0\x01\xed\x3c\x6a\x4d\x3b\x14\x37\xf5\x20\x96\xcd\x19\x56\xd0\x42\xa5\x97\xd5\x61\xa5\x96\xec\xd3\xd1\x73\x5a\x8d\x57\x0e\xa0\xec\x27\x22\x5a\x2c\x4a\xaf\xf2\x63\x06\xd1\x52\x6c\x1a\xf3\xca\x6d\x9c\xf5\xa2\xc9\x8f\x47\xe1\xc4\x6d\xb9\xa3\x32\x34\xcf\xd4\xd8\x1f\x2c\x98\x53\x8a\x09\xeb\xe7\x69\x98\xd0\xd8\xfd\x25\x99\x7c\x7d\x25\x5c\x6d\x66\xec\xe6\xfa\x56\xf1\x11\x44\x95\x0f\x02\x77\x95\xe6\x53\x00\x8f\x4b\xd7\xca\x2d\xee\x85\xd8\xe9\x0f\x3d\xc3\x15\x13\x0c\xe2\xa0\x03\x75\xa3\x18\xc7\xc3\xd9\x7b\xe2\xc8\xce\x5b\x6d\xb4\x1a\x62\x54\xff\x26\x4f\xa6\x15\x5b\xae\xe3\xb0\x77\x3c\x0f\x49\x7c\x57\x3f\x19\xbb\x4f\x42\x40\x28\x1f\x0b\x1f\x4f\x7b\xe8\x57\xa4\xe5\x9d\x41\x6c\x06\xb4\xc5\x0f\xa0\x9e\x18\x10\xdd\xc6\xb1\x46\x7b\xae\xac\x5a\x36\x68\xd1\x1b\x6e\xca\xa9\x01\x44\x00\x16\xf3\x89\xf8\x0a\xcc\x4d\xb9\x77\x02\x5e\x7f\x59\x24\x38\x8c\x7e\x34\x0a\x73\x2e\x55\x44\x40\xe7\x65\x70\xf8\xdd\x71\xb7\xd6\x40\xb3\x45\x0d\x1f\xd5\xf0\x41\x0a\x18\xf9\xa3\x49\x4f\x70\x7c\x71\x7b\x79\xb4\xbf\x75\xc9\x84\x00\xb0\x96\xb2\x16\x53\xb5\xd2\x17\xcf\x35\x65\xc9\x59\x74\x56\xf7\x07\x03\x49\x7a\x07\x87\x63\x82\x9b\xc0\x1b\xb1\xcb\xc8\xfa\x04\xea\xdc\x9a\x6e\x3f\x66\x99\x58\x7a\x9e\x75\xc9\x4e\x5b\xab\x00\x36\xe0\xb2\xe7\x11\x39\x2c\xff\x00\x47\xd0\xd6\xb0\x5b\xd2\xa5\x88\xbc\x10\x97\x18\x95\x42\x59\xf1\xd8\x66\x78\xa5\x79\xa3\x12\x0f\x19\xcf\xb2\x96\x3f\x17\x7a\xeb\x70\xf2\xd4\x84\x48\x26\x26\x2e\x51\xb8\x02\x71\x27\x20\x68\xef\x5b\x38\x56\xfa\x85\x35\xaa\x2a\x88\xb2\xd4\x1f\x2a\x0e\x2f\xda\x76\x24\xc2\x85\x02\x72\xac\x4a\x2f\x56\x1f\x8f\x2f\x7a\x31\x8b\xfd\x5c\xaf\x96\x96\x14\x9e\x4a\xc8\x24\xad\x34\x60\x53\x8f\xdc\x25\x42\x1b\xee\xc2\xcc\x68\x18\x16\x2d\x06\xbb\xed\x0c\x40\xa3\x87\x19\x23\x49\xdb\x67\xa1\x18\xba\xda\x6c\xd5\xab\x01\x40\xee\x27\x32\x04\xf6\x28\xaa\xd1\xc1\x35\xf7\x70\x27\x9a\x65\x1e\x24\xd8\xc1\x4d\x75\xa6\x05\x9d\x76\xb9\x6a\x6f\xd8\x57\xde\xf5\xe0\xb3\x54\xb2\x7a\xb9\x37\xa5\x81\x5d\x16\xb5\xfa\xe4\x07\xff\x18\x22\x2c\x6d\x1e\xd2\x63\xbe\x68\xc9\x5f\x32\xd9\x08\xbd\x89\x5c\xd7\x62\x07\xae\x72\x64\x87\x56\x7f\x9a\x67\xda\xd7\x9a\xbe\xc3\x16\xf6\x83\xb1\x7f\x2d\x02\xbf\x07\xe0\xac\x8b\x5b\xc6\x16\x2c\xf9\x46\x97\xb3\xc2\x7c\xd1\xfe\xa4\x9b\x27\xf2\x3b\xa2\x90\x18\x71\x96\x25\x06\x52\x0c\x39\x2d\xa8\xb6\xad\x0d\x99\xf7\x01\x3f\xbc\x06\xc2\xc1\x7a\x56\x95\x00\xc8\xa7\x69\x64\x81\xc1\xcd\x33\xe9\xb1\x4e\x40\xb8\x2e\x79\xa5\xf5\xdb\x82\x57\x1b\xa9\x7b\xae\x3a\xd3\xe0\x47\x95\x15\xbb\x0e\x2b\x0f\x3b\xfc\xd1\xfd\x33\x03\x4e\xfc\x62\x45\xed\xdd\x7e\xe2\x08\x6d\xda\xe2\x60\x0d\x8c\xa7\x3e\x21\x4e\x8c\x2b\x0b\xdb\x2b\x04\x7c\x6a\x46\x4a\x56\x2e\xd7\x7b\x73\xd2\xd8\x41\xc4\xb3\x49\x73\x55\x12\x57\x71\x3b\x75\x36\x32\xef\xba\x34\x81\x69\xab\xc9\x0a\x68\xf4\x26\x11\xa4\x01\x26\xd7\xcb\x21\xb5\x86\x95\x56\x81\x86\xf7\xe5\x69\xd2\xff\x0f\x9e\x74\x5d\x04\x87\xdd\x2e\xb9\x97\xca\xfc\x5a\xbf\x9d\xd1\x02\xe6\x2f\xf6\x6c\xba\x87" - , vecSig = - "\xe3\x01\x34\x5a\x41\xa3\x9a\x4d\x72\xff\xf8\xdf\x69\xc9\x80\x75\xa0\xcc\x08\x2b\x80\x2f\xc9\xb2\xb6\xbc\x50\x3f\x92\x6b\x65\xbd\xdf\x7f\x4c\x8f\x1c\xb4\x9f\x63\x96\xaf\xc8\xa7\x0a\xbe\x6d\x8a\xef\x0d\xb4\x78\xd4\xc6\xb2\x97\x00\x76\xc6\xa0\x48\x4f\xe7\x6d\x76\xb3\xa9\x76\x25\xd7\x9f\x1c\xe2\x40\xe7\xc5\x76\x75\x0d\x29\x55\x28\x28\x6f\x71\x9b\x41\x3d\xe9\xad\xa3\xe8\xeb\x78\xed\x57\x36\x03\xce\x30\xd8\xbb\x76\x17\x85\xdc\x30\xdb\xc3\x20\x86\x9e\x1a\x00" - } - ] - -doPublicKeyTest i vec = testCase (show i) (pub @=? Ed448.toPublic sec) - where - !pub = throwCryptoError $ Ed448.publicKey (vecPub vec) - !sec = throwCryptoError $ Ed448.secretKey (vecSec vec) - -doSignatureTest i vec = testCase (show i) (sig @=? Ed448.sign sec pub (vecMsg vec)) - where - !sig = throwCryptoError $ Ed448.signature (vecSig vec) - !pub = throwCryptoError $ Ed448.publicKey (vecPub vec) - !sec = throwCryptoError $ Ed448.secretKey (vecSec vec) - -doVerifyTest i vec = testCase (show i) (True @=? Ed448.verify pub (vecMsg vec) sig) - where - !sig = throwCryptoError $ Ed448.signature (vecSig vec) - !pub = throwCryptoError $ Ed448.publicKey (vecPub vec) - -tests = - testGroup - "Ed448" - [ testCase "gen secretkey" (Ed448.generateSecretKey *> pure ()) - , testGroup "gen publickey" $ zipWith doPublicKeyTest [katZero ..] vectors - , testGroup "gen signature" $ zipWith doSignatureTest [katZero ..] vectors - , testGroup "verify sig" $ zipWith doVerifyTest [katZero ..] vectors - ] diff --git a/tests/KAT_EdDSA.hs b/tests/KAT_EdDSA.hs deleted file mode 100644 --- a/tests/KAT_EdDSA.hs +++ /dev/null @@ -1,165 +0,0 @@ -{-# LANGUAGE BangPatterns #-} -{-# LANGUAGE ExistentialQuantification #-} -{-# LANGUAGE GADTs #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RecordWildCards #-} -{-# LANGUAGE TypeOperators #-} - -module KAT_EdDSA (tests) where - -import Crypto.ECC -import Crypto.Error -import Crypto.Hash.Algorithms -import Crypto.Hash.IO -import qualified Crypto.PubKey.EdDSA as EdDSA -import Imports - -data Vec - = forall curve hash. - ( EdDSA.EllipticCurveEdDSA curve - , HashAlgorithm hash - , HashDigestSize hash ~ EdDSA.CurveDigestSize curve - ) => - Vec - { vecPrx :: Maybe curve - , vecAlg :: hash - , vecSec :: ByteString - , vecPub :: ByteString - , vecMsg :: ByteString - , vecSig :: ByteString - } - -vectors = - [ Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = SHA512 - , vecSec = - "\x9d\x61\xb1\x9d\xef\xfd\x5a\x60\xba\x84\x4a\xf4\x92\xec\x2c\xc4\x44\x49\xc5\x69\x7b\x32\x69\x19\x70\x3b\xac\x03\x1c\xae\x7f\x60" - , vecPub = - "\xd7\x5a\x98\x01\x82\xb1\x0a\xb7\xd5\x4b\xfe\xd3\xc9\x64\x07\x3a\x0e\xe1\x72\xf3\xda\xa6\x23\x25\xaf\x02\x1a\x68\xf7\x07\x51\x1a" - , vecMsg = "" - , vecSig = - "\xe5\x56\x43\x00\xc3\x60\xac\x72\x90\x86\xe2\xcc\x80\x6e\x82\x8a\x84\x87\x7f\x1e\xb8\xe5\xd9\x74\xd8\x73\xe0\x65\x22\x49\x01\x55\x5f\xb8\x82\x15\x90\xa3\x3b\xac\xc6\x1e\x39\x70\x1c\xf9\xb4\x6b\xd2\x5b\xf5\xf0\x59\x5b\xbe\x24\x65\x51\x41\x43\x8e\x7a\x10\x0b" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = SHA512 - , vecSec = - "\x4c\xcd\x08\x9b\x28\xff\x96\xda\x9d\xb6\xc3\x46\xec\x11\x4e\x0f\x5b\x8a\x31\x9f\x35\xab\xa6\x24\xda\x8c\xf6\xed\x4f\xb8\xa6\xfb" - , vecPub = - "\x3d\x40\x17\xc3\xe8\x43\x89\x5a\x92\xb7\x0a\xa7\x4d\x1b\x7e\xbc\x9c\x98\x2c\xcf\x2e\xc4\x96\x8c\xc0\xcd\x55\xf1\x2a\xf4\x66\x0c" - , vecMsg = "\x72" - , vecSig = - "\x92\xa0\x09\xa9\xf0\xd4\xca\xb8\x72\x0e\x82\x0b\x5f\x64\x25\x40\xa2\xb2\x7b\x54\x16\x50\x3f\x8f\xb3\x76\x22\x23\xeb\xdb\x69\xda\x08\x5a\xc1\xe4\x3e\x15\x99\x6e\x45\x8f\x36\x13\xd0\xf1\x1d\x8c\x38\x7b\x2e\xae\xb4\x30\x2a\xee\xb0\x0d\x29\x16\x12\xbb\x0c\x00" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = SHA512 - , vecSec = - "\xc5\xaa\x8d\xf4\x3f\x9f\x83\x7b\xed\xb7\x44\x2f\x31\xdc\xb7\xb1\x66\xd3\x85\x35\x07\x6f\x09\x4b\x85\xce\x3a\x2e\x0b\x44\x58\xf7" - , vecPub = - "\xfc\x51\xcd\x8e\x62\x18\xa1\xa3\x8d\xa4\x7e\xd0\x02\x30\xf0\x58\x08\x16\xed\x13\xba\x33\x03\xac\x5d\xeb\x91\x15\x48\x90\x80\x25" - , vecMsg = "\xaf\x82" - , vecSig = - "\x62\x91\xd6\x57\xde\xec\x24\x02\x48\x27\xe6\x9c\x3a\xbe\x01\xa3\x0c\xe5\x48\xa2\x84\x74\x3a\x44\x5e\x36\x80\xd7\xdb\x5a\xc3\xac\x18\xff\x9b\x53\x8d\x16\xf2\x90\xae\x67\xf7\x60\x98\x4d\xc6\x59\x4a\x7c\x15\xe9\x71\x6e\xd2\x8d\xc0\x27\xbe\xce\xea\x1e\xc4\x0a" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = SHA512 - , vecSec = - "\xf5\xe5\x76\x7c\xf1\x53\x31\x95\x17\x63\x0f\x22\x68\x76\xb8\x6c\x81\x60\xcc\x58\x3b\xc0\x13\x74\x4c\x6b\xf2\x55\xf5\xcc\x0e\xe5" - , vecPub = - "\x27\x81\x17\xfc\x14\x4c\x72\x34\x0f\x67\xd0\xf2\x31\x6e\x83\x86\xce\xff\xbf\x2b\x24\x28\xc9\xc5\x1f\xef\x7c\x59\x7f\x1d\x42\x6e" - , vecMsg = - "\x08\xb8\xb2\xb7\x33\x42\x42\x43\x76\x0f\xe4\x26\xa4\xb5\x49\x08\x63\x21\x10\xa6\x6c\x2f\x65\x91\xea\xbd\x33\x45\xe3\xe4\xeb\x98\xfa\x6e\x26\x4b\xf0\x9e\xfe\x12\xee\x50\xf8\xf5\x4e\x9f\x77\xb1\xe3\x55\xf6\xc5\x05\x44\xe2\x3f\xb1\x43\x3d\xdf\x73\xbe\x84\xd8\x79\xde\x7c\x00\x46\xdc\x49\x96\xd9\xe7\x73\xf4\xbc\x9e\xfe\x57\x38\x82\x9a\xdb\x26\xc8\x1b\x37\xc9\x3a\x1b\x27\x0b\x20\x32\x9d\x65\x86\x75\xfc\x6e\xa5\x34\xe0\x81\x0a\x44\x32\x82\x6b\xf5\x8c\x94\x1e\xfb\x65\xd5\x7a\x33\x8b\xbd\x2e\x26\x64\x0f\x89\xff\xbc\x1a\x85\x8e\xfc\xb8\x55\x0e\xe3\xa5\xe1\x99\x8b\xd1\x77\xe9\x3a\x73\x63\xc3\x44\xfe\x6b\x19\x9e\xe5\xd0\x2e\x82\xd5\x22\xc4\xfe\xba\x15\x45\x2f\x80\x28\x8a\x82\x1a\x57\x91\x16\xec\x6d\xad\x2b\x3b\x31\x0d\xa9\x03\x40\x1a\xa6\x21\x00\xab\x5d\x1a\x36\x55\x3e\x06\x20\x3b\x33\x89\x0c\xc9\xb8\x32\xf7\x9e\xf8\x05\x60\xcc\xb9\xa3\x9c\xe7\x67\x96\x7e\xd6\x28\xc6\xad\x57\x3c\xb1\x16\xdb\xef\xef\xd7\x54\x99\xda\x96\xbd\x68\xa8\xa9\x7b\x92\x8a\x8b\xbc\x10\x3b\x66\x21\xfc\xde\x2b\xec\xa1\x23\x1d\x20\x6b\xe6\xcd\x9e\xc7\xaf\xf6\xf6\xc9\x4f\xcd\x72\x04\xed\x34\x55\xc6\x8c\x83\xf4\xa4\x1d\xa4\xaf\x2b\x74\xef\x5c\x53\xf1\xd8\xac\x70\xbd\xcb\x7e\xd1\x85\xce\x81\xbd\x84\x35\x9d\x44\x25\x4d\x95\x62\x9e\x98\x55\xa9\x4a\x7c\x19\x58\xd1\xf8\xad\xa5\xd0\x53\x2e\xd8\xa5\xaa\x3f\xb2\xd1\x7b\xa7\x0e\xb6\x24\x8e\x59\x4e\x1a\x22\x97\xac\xbb\xb3\x9d\x50\x2f\x1a\x8c\x6e\xb6\xf1\xce\x22\xb3\xde\x1a\x1f\x40\xcc\x24\x55\x41\x19\xa8\x31\xa9\xaa\xd6\x07\x9c\xad\x88\x42\x5d\xe6\xbd\xe1\xa9\x18\x7e\xbb\x60\x92\xcf\x67\xbf\x2b\x13\xfd\x65\xf2\x70\x88\xd7\x8b\x7e\x88\x3c\x87\x59\xd2\xc4\xf5\xc6\x5a\xdb\x75\x53\x87\x8a\xd5\x75\xf9\xfa\xd8\x78\xe8\x0a\x0c\x9b\xa6\x3b\xcb\xcc\x27\x32\xe6\x94\x85\xbb\xc9\xc9\x0b\xfb\xd6\x24\x81\xd9\x08\x9b\xec\xcf\x80\xcf\xe2\xdf\x16\xa2\xcf\x65\xbd\x92\xdd\x59\x7b\x07\x07\xe0\x91\x7a\xf4\x8b\xbb\x75\xfe\xd4\x13\xd2\x38\xf5\x55\x5a\x7a\x56\x9d\x80\xc3\x41\x4a\x8d\x08\x59\xdc\x65\xa4\x61\x28\xba\xb2\x7a\xf8\x7a\x71\x31\x4f\x31\x8c\x78\x2b\x23\xeb\xfe\x80\x8b\x82\xb0\xce\x26\x40\x1d\x2e\x22\xf0\x4d\x83\xd1\x25\x5d\xc5\x1a\xdd\xd3\xb7\x5a\x2b\x1a\xe0\x78\x45\x04\xdf\x54\x3a\xf8\x96\x9b\xe3\xea\x70\x82\xff\x7f\xc9\x88\x8c\x14\x4d\xa2\xaf\x58\x42\x9e\xc9\x60\x31\xdb\xca\xd3\xda\xd9\xaf\x0d\xcb\xaa\xaf\x26\x8c\xb8\xfc\xff\xea\xd9\x4f\x3c\x7c\xa4\x95\xe0\x56\xa9\xb4\x7a\xcd\xb7\x51\xfb\x73\xe6\x66\xc6\xc6\x55\xad\xe8\x29\x72\x97\xd0\x7a\xd1\xba\x5e\x43\xf1\xbc\xa3\x23\x01\x65\x13\x39\xe2\x29\x04\xcc\x8c\x42\xf5\x8c\x30\xc0\x4a\xaf\xdb\x03\x8d\xda\x08\x47\xdd\x98\x8d\xcd\xa6\xf3\xbf\xd1\x5c\x4b\x4c\x45\x25\x00\x4a\xa0\x6e\xef\xf8\xca\x61\x78\x3a\xac\xec\x57\xfb\x3d\x1f\x92\xb0\xfe\x2f\xd1\xa8\x5f\x67\x24\x51\x7b\x65\xe6\x14\xad\x68\x08\xd6\xf6\xee\x34\xdf\xf7\x31\x0f\xdc\x82\xae\xbf\xd9\x04\xb0\x1e\x1d\xc5\x4b\x29\x27\x09\x4b\x2d\xb6\x8d\x6f\x90\x3b\x68\x40\x1a\xde\xbf\x5a\x7e\x08\xd7\x8f\xf4\xef\x5d\x63\x65\x3a\x65\x04\x0c\xf9\xbf\xd4\xac\xa7\x98\x4a\x74\xd3\x71\x45\x98\x67\x80\xfc\x0b\x16\xac\x45\x16\x49\xde\x61\x88\xa7\xdb\xdf\x19\x1f\x64\xb5\xfc\x5e\x2a\xb4\x7b\x57\xf7\xf7\x27\x6c\xd4\x19\xc1\x7a\x3c\xa8\xe1\xb9\x39\xae\x49\xe4\x88\xac\xba\x6b\x96\x56\x10\xb5\x48\x01\x09\xc8\xb1\x7b\x80\xe1\xb7\xb7\x50\xdf\xc7\x59\x8d\x5d\x50\x11\xfd\x2d\xcc\x56\x00\xa3\x2e\xf5\xb5\x2a\x1e\xcc\x82\x0e\x30\x8a\xa3\x42\x72\x1a\xac\x09\x43\xbf\x66\x86\xb6\x4b\x25\x79\x37\x65\x04\xcc\xc4\x93\xd9\x7e\x6a\xed\x3f\xb0\xf9\xcd\x71\xa4\x3d\xd4\x97\xf0\x1f\x17\xc0\xe2\xcb\x37\x97\xaa\x2a\x2f\x25\x66\x56\x16\x8e\x6c\x49\x6a\xfc\x5f\xb9\x32\x46\xf6\xb1\x11\x63\x98\xa3\x46\xf1\xa6\x41\xf3\xb0\x41\xe9\x89\xf7\x91\x4f\x90\xcc\x2c\x7f\xff\x35\x78\x76\xe5\x06\xb5\x0d\x33\x4b\xa7\x7c\x22\x5b\xc3\x07\xba\x53\x71\x52\xf3\xf1\x61\x0e\x4e\xaf\xe5\x95\xf6\xd9\xd9\x0d\x11\xfa\xa9\x33\xa1\x5e\xf1\x36\x95\x46\x86\x8a\x7f\x3a\x45\xa9\x67\x68\xd4\x0f\xd9\xd0\x34\x12\xc0\x91\xc6\x31\x5c\xf4\xfd\xe7\xcb\x68\x60\x69\x37\x38\x0d\xb2\xea\xaa\x70\x7b\x4c\x41\x85\xc3\x2e\xdd\xcd\xd3\x06\x70\x5e\x4d\xc1\xff\xc8\x72\xee\xee\x47\x5a\x64\xdf\xac\x86\xab\xa4\x1c\x06\x18\x98\x3f\x87\x41\xc5\xef\x68\xd3\xa1\x01\xe8\xa3\xb8\xca\xc6\x0c\x90\x5c\x15\xfc\x91\x08\x40\xb9\x4c\x00\xa0\xb9\xd0" - , vecSig = - "\x0a\xab\x4c\x90\x05\x01\xb3\xe2\x4d\x7c\xdf\x46\x63\x32\x6a\x3a\x87\xdf\x5e\x48\x43\xb2\xcb\xdb\x67\xcb\xf6\xe4\x60\xfe\xc3\x50\xaa\x53\x71\xb1\x50\x8f\x9f\x45\x28\xec\xea\x23\xc4\x36\xd9\x4b\x5e\x8f\xcd\x4f\x68\x1e\x30\xa6\xac\x00\xa9\x70\x4a\x18\x8a\x03" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = SHA512 - , vecSec = - "\x83\x3f\xe6\x24\x09\x23\x7b\x9d\x62\xec\x77\x58\x75\x20\x91\x1e\x9a\x75\x9c\xec\x1d\x19\x75\x5b\x7d\xa9\x01\xb9\x6d\xca\x3d\x42" - , vecPub = - "\xec\x17\x2b\x93\xad\x5e\x56\x3b\xf4\x93\x2c\x70\xe1\x24\x50\x34\xc3\x54\x67\xef\x2e\xfd\x4d\x64\xeb\xf8\x19\x68\x34\x67\xe2\xbf" - , vecMsg = - "\xdd\xaf\x35\xa1\x93\x61\x7a\xba\xcc\x41\x73\x49\xae\x20\x41\x31\x12\xe6\xfa\x4e\x89\xa9\x7e\xa2\x0a\x9e\xee\xe6\x4b\x55\xd3\x9a\x21\x92\x99\x2a\x27\x4f\xc1\xa8\x36\xba\x3c\x23\xa3\xfe\xeb\xbd\x45\x4d\x44\x23\x64\x3c\xe8\x0e\x2a\x9a\xc9\x4f\xa5\x4c\xa4\x9f" - , vecSig = - "\xdc\x2a\x44\x59\xe7\x36\x96\x33\xa5\x2b\x1b\xf2\x77\x83\x9a\x00\x20\x10\x09\xa3\xef\xbf\x3e\xcb\x69\xbe\xa2\x18\x6c\x26\xb5\x89\x09\x35\x1f\xc9\xac\x90\xb3\xec\xfd\xfb\xc7\xc6\x64\x31\xe0\x30\x3d\xca\x17\x9c\x13\x8a\xc1\x7a\xd9\xbe\xf1\x17\x73\x31\xa7\x04" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = Blake2b_512 - , vecSec = - "\x9d\x61\xb1\x9d\xef\xfd\x5a\x60\xba\x84\x4a\xf4\x92\xec\x2c\xc4\x44\x49\xc5\x69\x7b\x32\x69\x19\x70\x3b\xac\x03\x1c\xae\x7f\x60" - , vecPub = - "\x78\xe6\x5b\xf3\x0f\x89\x3d\x32\xfc\x57\xef\x05\x1c\x34\x1b\xde\xde\x24\x25\x44\xfc\x2a\x21\x12\xf0\xfa\x2c\x7a\xfd\xeb\xc0\x2f" - , vecMsg = "" - , vecSig = - "\x99\xa5\x23\xbd\x46\x16\xc8\x16\x11\x44\xd6\xa9\x9d\x3c\x32\x40\x0c\xb4\xa3\x26\xf4\xd7\x9e\x30\x73\x40\xf6\xaf\xa1\x17\x50\xa0\x08\x5d\x7d\x84\x62\x6b\xc9\xe4\xb1\x53\xfc\x0e\x39\x6d\x15\xce\x44\xc3\x9b\xae\x45\x33\x80\x4d\xb1\xfe\x5b\x52\xf2\xb1\xb8\x05" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = Blake2b_512 - , vecSec = - "\x4c\xcd\x08\x9b\x28\xff\x96\xda\x9d\xb6\xc3\x46\xec\x11\x4e\x0f\x5b\x8a\x31\x9f\x35\xab\xa6\x24\xda\x8c\xf6\xed\x4f\xb8\xa6\xfb" - , vecPub = - "\x5e\x71\x39\x2d\x91\xe6\xa5\x8f\xed\xeb\x08\x50\x36\x4f\x56\xcd\x15\x8a\x60\x44\x75\x57\xd7\x89\x03\x89\xc9\xb3\xd4\x57\x6d\x4d" - , vecMsg = "\x72" - , vecSig = - "\x6d\xa7\x5e\x15\xb5\x70\x7f\x4d\xe5\xa1\x53\xc4\x8a\x5d\x83\x9f\xb8\x50\x74\xc3\x8a\xeb\x62\x85\x97\x7f\x03\xa1\x39\x77\x59\x7f\x97\x60\x69\xfd\xb9\x03\xf1\x83\x47\x4a\xaa\x5e\xd0\xcf\xe8\x78\xba\x8e\xf8\x68\xc5\xe4\x7c\xa3\xf9\x6c\xcf\xb3\xa8\x9b\x2a\x06" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = Blake2b_512 - , vecSec = - "\xc5\xaa\x8d\xf4\x3f\x9f\x83\x7b\xed\xb7\x44\x2f\x31\xdc\xb7\xb1\x66\xd3\x85\x35\x07\x6f\x09\x4b\x85\xce\x3a\x2e\x0b\x44\x58\xf7" - , vecPub = - "\x8d\x53\xca\x70\xf0\xea\xb2\x3b\x91\x78\x34\x57\x85\xfc\xdb\x69\xed\x67\x23\xf8\x14\x8f\x7e\x33\x9e\x88\x65\x37\x00\xb7\x18\xda" - , vecMsg = "\xaf\x82" - , vecSig = - "\x7c\xc3\xc1\x38\x52\xbd\x12\xab\xf3\xce\x4c\xa8\xca\x28\x36\xcb\xf8\x6d\xa9\x6c\x46\x34\xc5\x0d\xf3\xfb\x80\xdc\x80\x9e\x29\xdb\x0e\x10\x9c\x36\x13\x53\x40\x7c\x12\x36\xa9\x04\xf6\x36\x86\x8a\xa3\x39\x77\xa9\x9d\x3f\x84\x45\x98\xdb\x15\x38\xb4\x29\x52\x03" - } - , Vec - { vecPrx = Just Curve_Edwards25519 - , vecAlg = Blake2b_512 - , vecSec = - "\xf5\xe5\x76\x7c\xf1\x53\x31\x95\x17\x63\x0f\x22\x68\x76\xb8\x6c\x81\x60\xcc\x58\x3b\xc0\x13\x74\x4c\x6b\xf2\x55\xf5\xcc\x0e\xe5" - , vecPub = - "\x9e\x3c\xa4\x9b\xb2\xd9\xe3\x6b\x8f\x0c\x94\x4a\x7b\x1c\x29\x26\x45\xda\x87\xce\x6f\xa6\xb4\x28\x86\xe5\xd7\xc8\x68\x33\xa7\x14" - , vecMsg = - "\x08\xb8\xb2\xb7\x33\x42\x42\x43\x76\x0f\xe4\x26\xa4\xb5\x49\x08\x63\x21\x10\xa6\x6c\x2f\x65\x91\xea\xbd\x33\x45\xe3\xe4\xeb\x98\xfa\x6e\x26\x4b\xf0\x9e\xfe\x12\xee\x50\xf8\xf5\x4e\x9f\x77\xb1\xe3\x55\xf6\xc5\x05\x44\xe2\x3f\xb1\x43\x3d\xdf\x73\xbe\x84\xd8\x79\xde\x7c\x00\x46\xdc\x49\x96\xd9\xe7\x73\xf4\xbc\x9e\xfe\x57\x38\x82\x9a\xdb\x26\xc8\x1b\x37\xc9\x3a\x1b\x27\x0b\x20\x32\x9d\x65\x86\x75\xfc\x6e\xa5\x34\xe0\x81\x0a\x44\x32\x82\x6b\xf5\x8c\x94\x1e\xfb\x65\xd5\x7a\x33\x8b\xbd\x2e\x26\x64\x0f\x89\xff\xbc\x1a\x85\x8e\xfc\xb8\x55\x0e\xe3\xa5\xe1\x99\x8b\xd1\x77\xe9\x3a\x73\x63\xc3\x44\xfe\x6b\x19\x9e\xe5\xd0\x2e\x82\xd5\x22\xc4\xfe\xba\x15\x45\x2f\x80\x28\x8a\x82\x1a\x57\x91\x16\xec\x6d\xad\x2b\x3b\x31\x0d\xa9\x03\x40\x1a\xa6\x21\x00\xab\x5d\x1a\x36\x55\x3e\x06\x20\x3b\x33\x89\x0c\xc9\xb8\x32\xf7\x9e\xf8\x05\x60\xcc\xb9\xa3\x9c\xe7\x67\x96\x7e\xd6\x28\xc6\xad\x57\x3c\xb1\x16\xdb\xef\xef\xd7\x54\x99\xda\x96\xbd\x68\xa8\xa9\x7b\x92\x8a\x8b\xbc\x10\x3b\x66\x21\xfc\xde\x2b\xec\xa1\x23\x1d\x20\x6b\xe6\xcd\x9e\xc7\xaf\xf6\xf6\xc9\x4f\xcd\x72\x04\xed\x34\x55\xc6\x8c\x83\xf4\xa4\x1d\xa4\xaf\x2b\x74\xef\x5c\x53\xf1\xd8\xac\x70\xbd\xcb\x7e\xd1\x85\xce\x81\xbd\x84\x35\x9d\x44\x25\x4d\x95\x62\x9e\x98\x55\xa9\x4a\x7c\x19\x58\xd1\xf8\xad\xa5\xd0\x53\x2e\xd8\xa5\xaa\x3f\xb2\xd1\x7b\xa7\x0e\xb6\x24\x8e\x59\x4e\x1a\x22\x97\xac\xbb\xb3\x9d\x50\x2f\x1a\x8c\x6e\xb6\xf1\xce\x22\xb3\xde\x1a\x1f\x40\xcc\x24\x55\x41\x19\xa8\x31\xa9\xaa\xd6\x07\x9c\xad\x88\x42\x5d\xe6\xbd\xe1\xa9\x18\x7e\xbb\x60\x92\xcf\x67\xbf\x2b\x13\xfd\x65\xf2\x70\x88\xd7\x8b\x7e\x88\x3c\x87\x59\xd2\xc4\xf5\xc6\x5a\xdb\x75\x53\x87\x8a\xd5\x75\xf9\xfa\xd8\x78\xe8\x0a\x0c\x9b\xa6\x3b\xcb\xcc\x27\x32\xe6\x94\x85\xbb\xc9\xc9\x0b\xfb\xd6\x24\x81\xd9\x08\x9b\xec\xcf\x80\xcf\xe2\xdf\x16\xa2\xcf\x65\xbd\x92\xdd\x59\x7b\x07\x07\xe0\x91\x7a\xf4\x8b\xbb\x75\xfe\xd4\x13\xd2\x38\xf5\x55\x5a\x7a\x56\x9d\x80\xc3\x41\x4a\x8d\x08\x59\xdc\x65\xa4\x61\x28\xba\xb2\x7a\xf8\x7a\x71\x31\x4f\x31\x8c\x78\x2b\x23\xeb\xfe\x80\x8b\x82\xb0\xce\x26\x40\x1d\x2e\x22\xf0\x4d\x83\xd1\x25\x5d\xc5\x1a\xdd\xd3\xb7\x5a\x2b\x1a\xe0\x78\x45\x04\xdf\x54\x3a\xf8\x96\x9b\xe3\xea\x70\x82\xff\x7f\xc9\x88\x8c\x14\x4d\xa2\xaf\x58\x42\x9e\xc9\x60\x31\xdb\xca\xd3\xda\xd9\xaf\x0d\xcb\xaa\xaf\x26\x8c\xb8\xfc\xff\xea\xd9\x4f\x3c\x7c\xa4\x95\xe0\x56\xa9\xb4\x7a\xcd\xb7\x51\xfb\x73\xe6\x66\xc6\xc6\x55\xad\xe8\x29\x72\x97\xd0\x7a\xd1\xba\x5e\x43\xf1\xbc\xa3\x23\x01\x65\x13\x39\xe2\x29\x04\xcc\x8c\x42\xf5\x8c\x30\xc0\x4a\xaf\xdb\x03\x8d\xda\x08\x47\xdd\x98\x8d\xcd\xa6\xf3\xbf\xd1\x5c\x4b\x4c\x45\x25\x00\x4a\xa0\x6e\xef\xf8\xca\x61\x78\x3a\xac\xec\x57\xfb\x3d\x1f\x92\xb0\xfe\x2f\xd1\xa8\x5f\x67\x24\x51\x7b\x65\xe6\x14\xad\x68\x08\xd6\xf6\xee\x34\xdf\xf7\x31\x0f\xdc\x82\xae\xbf\xd9\x04\xb0\x1e\x1d\xc5\x4b\x29\x27\x09\x4b\x2d\xb6\x8d\x6f\x90\x3b\x68\x40\x1a\xde\xbf\x5a\x7e\x08\xd7\x8f\xf4\xef\x5d\x63\x65\x3a\x65\x04\x0c\xf9\xbf\xd4\xac\xa7\x98\x4a\x74\xd3\x71\x45\x98\x67\x80\xfc\x0b\x16\xac\x45\x16\x49\xde\x61\x88\xa7\xdb\xdf\x19\x1f\x64\xb5\xfc\x5e\x2a\xb4\x7b\x57\xf7\xf7\x27\x6c\xd4\x19\xc1\x7a\x3c\xa8\xe1\xb9\x39\xae\x49\xe4\x88\xac\xba\x6b\x96\x56\x10\xb5\x48\x01\x09\xc8\xb1\x7b\x80\xe1\xb7\xb7\x50\xdf\xc7\x59\x8d\x5d\x50\x11\xfd\x2d\xcc\x56\x00\xa3\x2e\xf5\xb5\x2a\x1e\xcc\x82\x0e\x30\x8a\xa3\x42\x72\x1a\xac\x09\x43\xbf\x66\x86\xb6\x4b\x25\x79\x37\x65\x04\xcc\xc4\x93\xd9\x7e\x6a\xed\x3f\xb0\xf9\xcd\x71\xa4\x3d\xd4\x97\xf0\x1f\x17\xc0\xe2\xcb\x37\x97\xaa\x2a\x2f\x25\x66\x56\x16\x8e\x6c\x49\x6a\xfc\x5f\xb9\x32\x46\xf6\xb1\x11\x63\x98\xa3\x46\xf1\xa6\x41\xf3\xb0\x41\xe9\x89\xf7\x91\x4f\x90\xcc\x2c\x7f\xff\x35\x78\x76\xe5\x06\xb5\x0d\x33\x4b\xa7\x7c\x22\x5b\xc3\x07\xba\x53\x71\x52\xf3\xf1\x61\x0e\x4e\xaf\xe5\x95\xf6\xd9\xd9\x0d\x11\xfa\xa9\x33\xa1\x5e\xf1\x36\x95\x46\x86\x8a\x7f\x3a\x45\xa9\x67\x68\xd4\x0f\xd9\xd0\x34\x12\xc0\x91\xc6\x31\x5c\xf4\xfd\xe7\xcb\x68\x60\x69\x37\x38\x0d\xb2\xea\xaa\x70\x7b\x4c\x41\x85\xc3\x2e\xdd\xcd\xd3\x06\x70\x5e\x4d\xc1\xff\xc8\x72\xee\xee\x47\x5a\x64\xdf\xac\x86\xab\xa4\x1c\x06\x18\x98\x3f\x87\x41\xc5\xef\x68\xd3\xa1\x01\xe8\xa3\xb8\xca\xc6\x0c\x90\x5c\x15\xfc\x91\x08\x40\xb9\x4c\x00\xa0\xb9\xd0" - , vecSig = - "\xd0\x39\x65\xac\x31\x6a\x20\xf5\xa4\x7a\xb2\xd6\x18\x5e\xb3\xf0\xae\xea\x9c\x2e\xb8\xab\xe9\x22\xe9\x6d\x31\x7b\x3b\xd0\xef\x02\xe8\xd4\x7f\xd9\x23\x84\xe2\x86\x15\xeb\x33\x14\xad\xbc\x71\xc4\x67\x59\x96\x09\x9e\x48\x4c\xeb\x16\x28\x47\xc4\x0c\x32\x44\x0e" - } - ] - -doPublicKeyTest :: Int -> Vec -> TestTree -doPublicKeyTest i Vec{..} = - testCase (show i) (pub @=? EdDSA.toPublic vecPrx vecAlg sec) - where - !pub = throwCryptoError $ EdDSA.publicKey vecPrx vecAlg vecPub - !sec = throwCryptoError $ EdDSA.secretKey vecPrx vecSec - -doSignatureTest :: Int -> Vec -> TestTree -doSignatureTest i Vec{..} = - testCase (show i) (sig @=? EdDSA.sign vecPrx sec pub vecMsg) - where - !sig = throwCryptoError $ EdDSA.signature vecPrx vecAlg vecSig - !pub = throwCryptoError $ EdDSA.publicKey vecPrx vecAlg vecPub - !sec = throwCryptoError $ EdDSA.secretKey vecPrx vecSec - -doVerifyTest :: Int -> Vec -> TestTree -doVerifyTest i Vec{..} = - testCase (show i) (True @=? EdDSA.verify vecPrx pub vecMsg sig) - where - !sig = throwCryptoError $ EdDSA.signature vecPrx vecAlg vecSig - !pub = throwCryptoError $ EdDSA.publicKey vecPrx vecAlg vecPub - -tests = - testGroup - "EdDSA" - [ testGroup "gen publickey" $ zipWith doPublicKeyTest [katZero ..] vectors - , testGroup "gen signature" $ zipWith doSignatureTest [katZero ..] vectors - , testGroup "verify sig" $ zipWith doVerifyTest [katZero ..] vectors - ] diff --git a/tests/KAT_HKDF.hs b/tests/KAT_HKDF.hs deleted file mode 100644 --- a/tests/KAT_HKDF.hs +++ /dev/null @@ -1,387 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_HKDF (tests) where - -import Crypto.Hash (HashAlgorithm, SHA256 (..)) -import qualified Crypto.KDF.HKDF as HKDF -import qualified Data.ByteString as B - -import Imports - -data KDFVector hash = KDFVector - { kdfIKM :: ByteString - , kdfSalt :: ByteString - , kdfInfo :: ByteString - , kdfResult :: ByteString - } - -sha256KDFVectors :: [KDFVector SHA256] -sha256KDFVectors = - [ KDFVector - "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" - ( B.pack - [0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c] - ) - "\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9" - "\x3c\xb2\x5f\x25\xfa\xac\xd5\x7a\x90\x43\x4f\x64\xd0\x36\x2f\x2a\x2d\x2d\x0a\x90\xcf\x1a\x5a\x4c\x5d\xb0\x2d\x56\xec\xc4\xc5\xbf\x34\x00\x72\x08\xd5\xb8\x87\x18\x58\x65" - , KDFVector - ( B.pack - [ 0x00 - , 0x01 - , 0x02 - , 0x03 - , 0x04 - , 0x05 - , 0x06 - , 0x07 - , 0x08 - , 0x09 - , 0x0a - , 0x0b - , 0x0c - , 0x0d - , 0x0e - , 0x0f - , 0x10 - , 0x11 - , 0x12 - , 0x13 - , 0x14 - , 0x15 - , 0x16 - , 0x17 - , 0x18 - , 0x19 - , 0x1a - , 0x1b - , 0x1c - , 0x1d - , 0x1e - , 0x1f - , 0x20 - , 0x21 - , 0x22 - , 0x23 - , 0x24 - , 0x25 - , 0x26 - , 0x27 - , 0x28 - , 0x29 - , 0x2a - , 0x2b - , 0x2c - , 0x2d - , 0x2e - , 0x2f - , 0x30 - , 0x31 - , 0x32 - , 0x33 - , 0x34 - , 0x35 - , 0x36 - , 0x37 - , 0x38 - , 0x39 - , 0x3a - , 0x3b - , 0x3c - , 0x3d - , 0x3e - , 0x3f - , 0x40 - , 0x41 - , 0x42 - , 0x43 - , 0x44 - , 0x45 - , 0x46 - , 0x47 - , 0x48 - , 0x49 - , 0x4a - , 0x4b - , 0x4c - , 0x4d - , 0x4e - , 0x4f - ] - ) - ( B.pack - [ 0x60 - , 0x61 - , 0x62 - , 0x63 - , 0x64 - , 0x65 - , 0x66 - , 0x67 - , 0x68 - , 0x69 - , 0x6a - , 0x6b - , 0x6c - , 0x6d - , 0x6e - , 0x6f - , 0x70 - , 0x71 - , 0x72 - , 0x73 - , 0x74 - , 0x75 - , 0x76 - , 0x77 - , 0x78 - , 0x79 - , 0x7a - , 0x7b - , 0x7c - , 0x7d - , 0x7e - , 0x7f - , 0x80 - , 0x81 - , 0x82 - , 0x83 - , 0x84 - , 0x85 - , 0x86 - , 0x87 - , 0x88 - , 0x89 - , 0x8a - , 0x8b - , 0x8c - , 0x8d - , 0x8e - , 0x8f - , 0x90 - , 0x91 - , 0x92 - , 0x93 - , 0x94 - , 0x95 - , 0x96 - , 0x97 - , 0x98 - , 0x99 - , 0x9a - , 0x9b - , 0x9c - , 0x9d - , 0x9e - , 0x9f - , 0xa0 - , 0xa1 - , 0xa2 - , 0xa3 - , 0xa4 - , 0xa5 - , 0xa6 - , 0xa7 - , 0xa8 - , 0xa9 - , 0xaa - , 0xab - , 0xac - , 0xad - , 0xae - , 0xaf - ] - ) - ( B.pack - [ 0xb0 - , 0xb1 - , 0xb2 - , 0xb3 - , 0xb4 - , 0xb5 - , 0xb6 - , 0xb7 - , 0xb8 - , 0xb9 - , 0xba - , 0xbb - , 0xbc - , 0xbd - , 0xbe - , 0xbf - , 0xc0 - , 0xc1 - , 0xc2 - , 0xc3 - , 0xc4 - , 0xc5 - , 0xc6 - , 0xc7 - , 0xc8 - , 0xc9 - , 0xca - , 0xcb - , 0xcc - , 0xcd - , 0xce - , 0xcf - , 0xd0 - , 0xd1 - , 0xd2 - , 0xd3 - , 0xd4 - , 0xd5 - , 0xd6 - , 0xd7 - , 0xd8 - , 0xd9 - , 0xda - , 0xdb - , 0xdc - , 0xdd - , 0xde - , 0xdf - , 0xe0 - , 0xe1 - , 0xe2 - , 0xe3 - , 0xe4 - , 0xe5 - , 0xe6 - , 0xe7 - , 0xe8 - , 0xe9 - , 0xea - , 0xeb - , 0xec - , 0xed - , 0xee - , 0xef - , 0xf0 - , 0xf1 - , 0xf2 - , 0xf3 - , 0xf4 - , 0xf5 - , 0xf6 - , 0xf7 - , 0xf8 - , 0xf9 - , 0xfa - , 0xfb - , 0xfc - , 0xfd - , 0xfe - , 0xff - ] - ) - ( B.pack - [ 0xb1 - , 0x1e - , 0x39 - , 0x8d - , 0xc8 - , 0x03 - , 0x27 - , 0xa1 - , 0xc8 - , 0xe7 - , 0xf7 - , 0x8c - , 0x59 - , 0x6a - , 0x49 - , 0x34 - , 0x4f - , 0x01 - , 0x2e - , 0xda - , 0x2d - , 0x4e - , 0xfa - , 0xd8 - , 0xa0 - , 0x50 - , 0xcc - , 0x4c - , 0x19 - , 0xaf - , 0xa9 - , 0x7c - , 0x59 - , 0x04 - , 0x5a - , 0x99 - , 0xca - , 0xc7 - , 0x82 - , 0x72 - , 0x71 - , 0xcb - , 0x41 - , 0xc6 - , 0x5e - , 0x59 - , 0x0e - , 0x09 - , 0xda - , 0x32 - , 0x75 - , 0x60 - , 0x0c - , 0x2f - , 0x09 - , 0xb8 - , 0x36 - , 0x77 - , 0x93 - , 0xa9 - , 0xac - , 0xa3 - , 0xdb - , 0x71 - , 0xcc - , 0x30 - , 0xc5 - , 0x81 - , 0x79 - , 0xec - , 0x3e - , 0x87 - , 0xc1 - , 0x4c - , 0x01 - , 0xd5 - , 0xc1 - , 0xf3 - , 0x43 - , 0x4f - , 0x1d - , 0x87 - ] - ) - ] - -kdfTests :: [TestTree] -kdfTests = - [ testGroup "sha256" $ concatMap toKDFTest $ zip is sha256KDFVectors - ] - where - toKDFTest (i, kdfVector) = - [ testCase (show i) (t HKDF.extract kdfVector) - ] - - t - :: HashAlgorithm a - => (ByteString -> ByteString -> HKDF.PRK a) -> KDFVector a -> Assertion - t ext v = - let prk = ext (kdfSalt v) (kdfIKM v) - in kdfResult v @=? HKDF.expand prk (kdfInfo v) (B.length $ kdfResult v) - - is :: [Int] - is = [1 ..] - -tests = - testGroup - "HKDF" - [ testGroup "KATs" kdfTests - ] diff --git a/tests/KAT_HMAC.hs b/tests/KAT_HMAC.hs deleted file mode 100644 --- a/tests/KAT_HMAC.hs +++ /dev/null @@ -1,212 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_HMAC (tests) where - -import Crypto.Hash ( - HashAlgorithm, - Keccak_224 (..), - Keccak_256 (..), - Keccak_384 (..), - Keccak_512 (..), - MD5 (..), - SHA1 (..), - SHA256 (..), - SHA3_224 (..), - SHA3_256 (..), - SHA3_384 (..), - SHA3_512 (..), - digestFromByteString, - ) -import qualified Crypto.MAC.HMAC as HMAC -import qualified Data.ByteString as B - -import Imports - -data MACVector hash = MACVector - { macKey :: ByteString - , macSecret :: ByteString - , macResult :: HMAC.HMAC hash - } - -instance Show (HMAC.HMAC a) where - show (HMAC.HMAC d) = show d - -digest :: HashAlgorithm hash => ByteString -> HMAC.HMAC hash -digest = maybe (error "cannot get digest") HMAC.HMAC . digestFromByteString - -v1 :: ByteString -v1 = "The quick brown fox jumps over the lazy dog" - -md5MACVectors :: [MACVector MD5] -md5MACVectors = - [ MACVector B.empty B.empty $ - digest "\x74\xe6\xf7\x29\x8a\x9c\x2d\x16\x89\x35\xf5\x8c\x00\x1b\xad\x88" - , MACVector "key" v1 $ - digest "\x80\x07\x07\x13\x46\x3e\x77\x49\xb9\x0c\x2d\xc2\x49\x11\xe2\x75" - ] - -sha1MACVectors :: [MACVector SHA1] -sha1MACVectors = - [ MACVector B.empty B.empty $ - digest - "\xfb\xdb\x1d\x1b\x18\xaa\x6c\x08\x32\x4b\x7d\x64\xb7\x1f\xb7\x63\x70\x69\x0e\x1d" - , MACVector "key" v1 $ - digest - "\xde\x7c\x9b\x85\xb8\xb7\x8a\xa6\xbc\x8a\x7a\x36\xf7\x0a\x90\x70\x1c\x9d\xb4\xd9" - ] - -sha256MACVectors :: [MACVector SHA256] -sha256MACVectors = - [ MACVector B.empty B.empty $ - digest - "\xb6\x13\x67\x9a\x08\x14\xd9\xec\x77\x2f\x95\xd7\x78\xc3\x5f\xc5\xff\x16\x97\xc4\x93\x71\x56\x53\xc6\xc7\x12\x14\x42\x92\xc5\xad" - , MACVector "key" v1 $ - digest - "\xf7\xbc\x83\xf4\x30\x53\x84\x24\xb1\x32\x98\xe6\xaa\x6f\xb1\x43\xef\x4d\x59\xa1\x49\x46\x17\x59\x97\x47\x9d\xbc\x2d\x1a\x3c\xd8" - ] - -keccak_key1 = "\x4a\x65\x66\x65" -keccak_data1 = - "\x77\x68\x61\x74\x20\x64\x6f\x20\x79\x61\x20\x77\x61\x6e\x74\x20\x66\x6f\x72\x20\x6e\x6f\x74\x68\x69\x6e\x67\x3f" - -keccak_224_MAC_Vectors :: [MACVector Keccak_224] -keccak_224_MAC_Vectors = - [ MACVector keccak_key1 keccak_data1 $ - digest - "\xe8\x24\xfe\xc9\x6c\x07\x4f\x22\xf9\x92\x35\xbb\x94\x2d\xa1\x98\x26\x64\xab\x69\x2c\xa8\x50\x10\x53\xcb\xd4\x14" - ] - -keccak_256_MAC_Vectors :: [MACVector Keccak_256] -keccak_256_MAC_Vectors = - [ MACVector keccak_key1 keccak_data1 $ - digest - "\xaa\x9a\xed\x44\x8c\x7a\xbc\x8b\x5e\x32\x6f\xfa\x6a\x01\xcd\xed\xf7\xb4\xb8\x31\x88\x14\x68\xc0\x44\xba\x8d\xd4\x56\x63\x69\xa1" - ] - -keccak_384_MAC_Vectors :: [MACVector Keccak_384] -keccak_384_MAC_Vectors = - [ MACVector keccak_key1 keccak_data1 $ - digest - "\x5a\xf5\xc9\xa7\x7a\x23\xa6\xa9\x3d\x80\x64\x9e\x56\x2a\xb7\x7f\x4f\x35\x52\xe3\xc5\xca\xff\xd9\x3b\xdf\x8b\x3c\xfc\x69\x20\xe3\x02\x3f\xc2\x67\x75\xd9\xdf\x1f\x3c\x94\x61\x31\x46\xad\x2c\x9d" - ] - -keccak_512_MAC_Vectors :: [MACVector Keccak_512] -keccak_512_MAC_Vectors = - [ MACVector keccak_key1 keccak_data1 $ - digest - "\xc2\x96\x2e\x5b\xbe\x12\x38\x00\x78\x52\xf7\x9d\x81\x4d\xbb\xec\xd4\x68\x2e\x6f\x09\x7d\x37\xa3\x63\x58\x7c\x03\xbf\xa2\xeb\x08\x59\xd8\xd9\xc7\x01\xe0\x4c\xec\xec\xfd\x3d\xd7\xbf\xd4\x38\xf2\x0b\x8b\x64\x8e\x01\xbf\x8c\x11\xd2\x68\x24\xb9\x6c\xeb\xbd\xcb" - ] - -sha3_key1 = "\x4a\x65\x66\x65" -sha3_data1 = - "\x77\x68\x61\x74\x20\x64\x6f\x20\x79\x61\x20\x77\x61\x6e\x74\x20\x66\x6f\x72\x20\x6e\x6f\x74\x68\x69\x6e\x67\x3f" - -sha3_224_MAC_Vectors :: [MACVector SHA3_224] -sha3_224_MAC_Vectors = - [ MACVector sha3_key1 sha3_data1 $ - digest - "\x7f\xdb\x8d\xd8\x8b\xd2\xf6\x0d\x1b\x79\x86\x34\xad\x38\x68\x11\xc2\xcf\xc8\x5b\xfa\xf5\xd5\x2b\xba\xce\x5e\x66" - ] - -sha3_256_MAC_Vectors :: [MACVector SHA3_256] -sha3_256_MAC_Vectors = - [ MACVector sha3_key1 sha3_data1 $ - digest - "\xc7\xd4\x07\x2e\x78\x88\x77\xae\x35\x96\xbb\xb0\xda\x73\xb8\x87\xc9\x17\x1f\x93\x09\x5b\x29\x4a\xe8\x57\xfb\xe2\x64\x5e\x1b\xa5" - ] - -sha3_384_MAC_Vectors :: [MACVector SHA3_384] -sha3_384_MAC_Vectors = - [ MACVector sha3_key1 sha3_data1 $ - digest - "\xf1\x10\x1f\x8c\xbf\x97\x66\xfd\x67\x64\xd2\xed\x61\x90\x3f\x21\xca\x9b\x18\xf5\x7c\xf3\xe1\xa2\x3c\xa1\x35\x08\xa9\x32\x43\xce\x48\xc0\x45\xdc\x00\x7f\x26\xa2\x1b\x3f\x5e\x0e\x9d\xf4\xc2\x0a" - ] - -sha3_512_MAC_Vectors :: [MACVector SHA3_512] -sha3_512_MAC_Vectors = - [ MACVector sha3_key1 sha3_data1 $ - digest - "\x5a\x4b\xfe\xab\x61\x66\x42\x7c\x7a\x36\x47\xb7\x47\x29\x2b\x83\x84\x53\x7c\xdb\x89\xaf\xb3\xbf\x56\x65\xe4\xc5\xe7\x09\x35\x0b\x28\x7b\xae\xc9\x21\xfd\x7c\xa0\xee\x7a\x0c\x31\xd0\x22\xa9\x5e\x1f\xc9\x2b\xa9\xd7\x7d\xf8\x83\x96\x02\x75\xbe\xb4\xe6\x20\x24" - ] - -macTests :: [TestTree] -macTests = - [ testGroup "md5" $ concatMap toMACTest $ zip is md5MACVectors - , testGroup "sha1" $ concatMap toMACTest $ zip is sha1MACVectors - , testGroup "sha256" $ concatMap toMACTest $ zip is sha256MACVectors - , testGroup "keccak-224" $ concatMap toMACTest $ zip is keccak_224_MAC_Vectors - , testGroup "keccak-256" $ concatMap toMACTest $ zip is keccak_256_MAC_Vectors - , testGroup "keccak-384" $ concatMap toMACTest $ zip is keccak_384_MAC_Vectors - , testGroup "keccak-512" $ concatMap toMACTest $ zip is keccak_512_MAC_Vectors - , testGroup "sha3-224" $ concatMap toMACTest $ zip is sha3_224_MAC_Vectors - , testGroup "sha3-256" $ concatMap toMACTest $ zip is sha3_256_MAC_Vectors - , testGroup "sha3-384" $ concatMap toMACTest $ zip is sha3_384_MAC_Vectors - , testGroup "sha3-512" $ concatMap toMACTest $ zip is sha3_512_MAC_Vectors - ] - where - toMACTest (i, macVector) = - [ testCase - (show i) - (macResult macVector @=? HMAC.hmac (macKey macVector) (macSecret macVector)) - , testCase - ("incr-" ++ show i) - ( macResult macVector - @=? HMAC.finalize - (HMAC.update (HMAC.initialize (macKey macVector)) (macSecret macVector)) - ) - ] - is :: [Int] - is = [1 ..] - -data MacIncremental a = MacIncremental ByteString ByteString (HMAC.HMAC a) - deriving (Show, Eq) - -instance HashAlgorithm a => Arbitrary (MacIncremental a) where - arbitrary = do - key <- arbitraryBSof 1 89 - msg <- arbitraryBSof 1 99 - return $ MacIncremental key msg (HMAC.hmac key msg) - -data MacIncrementalList a = MacIncrementalList ByteString [ByteString] (HMAC.HMAC a) - deriving (Show, Eq) - -instance HashAlgorithm a => Arbitrary (MacIncrementalList a) where - arbitrary = do - key <- arbitraryBSof 1 89 - msgs <- choose (1, 20) >>= \n -> replicateM n (arbitraryBSof 1 99) - return $ MacIncrementalList key msgs (HMAC.hmac key (B.concat msgs)) - -macIncrementalTests :: [TestTree] -macIncrementalTests = - [ testIncrProperties MD5 - , testIncrProperties SHA1 - , testIncrProperties SHA256 - , testIncrProperties SHA3_224 - , testIncrProperties SHA3_256 - , testIncrProperties SHA3_384 - , testIncrProperties SHA3_512 - ] - where - -- testIncrProperties :: HashAlgorithm a => a -> [Property] - testIncrProperties a = - testGroup - (show a) - [ testProperty "list-one" (prop_inc0 a) - , testProperty "list-multi" (prop_inc1 a) - ] - - prop_inc0 :: HashAlgorithm a => a -> MacIncremental a -> Bool - prop_inc0 _ (MacIncremental secret msg result) = - result `assertEq` HMAC.finalize (HMAC.update (HMAC.initialize secret) msg) - - prop_inc1 :: HashAlgorithm a => a -> MacIncrementalList a -> Bool - prop_inc1 _ (MacIncrementalList secret msgs result) = - result - `assertEq` HMAC.finalize (foldl' HMAC.update (HMAC.initialize secret) msgs) - -tests = - testGroup - "HMAC" - [ testGroup "KATs" macTests - , testGroup "properties" macIncrementalTests - ] diff --git a/tests/KAT_KMAC.hs b/tests/KAT_KMAC.hs deleted file mode 100644 --- a/tests/KAT_KMAC.hs +++ /dev/null @@ -1,156 +0,0 @@ -{-# LANGUAGE DataKinds #-} -{-# LANGUAGE FlexibleContexts #-} -{-# LANGUAGE FlexibleInstances #-} -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE RecordWildCards #-} - -module KAT_KMAC (tests) where - -import Crypto.Hash ( - HashAlgorithm, - SHAKE128 (..), - SHAKE256 (..), - digestFromByteString, - ) -import qualified Crypto.MAC.KMAC as KMAC - -import qualified Data.ByteString as B - -import Imports - -data MACVector hash = MACVector - { macString :: ByteString - , macKey :: ByteString - , macSecret :: ByteString - , macResult :: KMAC.KMAC hash - } - -instance Show (KMAC.KMAC a) where - show (KMAC.KMAC d) = show d - -digest :: HashAlgorithm hash => ByteString -> KMAC.KMAC hash -digest = maybe (error "cannot get digest") KMAC.KMAC . digestFromByteString - -vectors128 :: [MACVector (SHAKE128 256)] -vectors128 = - [ MACVector - { macString = "" - , macKey = B.pack [0x40 .. 0x5f] - , macSecret = B.pack [0x00 .. 0x03] - , macResult = - digest - "\xe5\x78\x0b\x0d\x3e\xa6\xf7\xd3\xa4\x29\xc5\x70\x6a\xa4\x3a\x00\xfa\xdb\xd7\xd4\x96\x28\x83\x9e\x31\x87\x24\x3f\x45\x6e\xe1\x4e" - } - , MACVector - { macString = "My Tagged Application" - , macKey = B.pack [0x40 .. 0x5f] - , macSecret = B.pack [0x00 .. 0x03] - , macResult = - digest - "\x3b\x1f\xba\x96\x3c\xd8\xb0\xb5\x9e\x8c\x1a\x6d\x71\x88\x8b\x71\x43\x65\x1a\xf8\xba\x0a\x70\x70\xc0\x97\x9e\x28\x11\x32\x4a\xa5" - } - , MACVector - { macString = "My Tagged Application" - , macKey = B.pack [0x40 .. 0x5f] - , macSecret = B.pack [0x00 .. 0xc7] - , macResult = - digest - "\x1f\x5b\x4e\x6c\xca\x02\x20\x9e\x0d\xcb\x5c\xa6\x35\xb8\x9a\x15\xe2\x71\xec\xc7\x60\x07\x1d\xfd\x80\x5f\xaa\x38\xf9\x72\x92\x30" - } - ] - -vectors256 :: [MACVector (SHAKE256 512)] -vectors256 = - [ MACVector - { macString = "My Tagged Application" - , macKey = B.pack [0x40 .. 0x5f] - , macSecret = B.pack [0x00 .. 0x03] - , macResult = - digest - "\x20\xc5\x70\xc3\x13\x46\xf7\x03\xc9\xac\x36\xc6\x1c\x03\xcb\x64\xc3\x97\x0d\x0c\xfc\x78\x7e\x9b\x79\x59\x9d\x27\x3a\x68\xd2\xf7\xf6\x9d\x4c\xc3\xde\x9d\x10\x4a\x35\x16\x89\xf2\x7c\xf6\xf5\x95\x1f\x01\x03\xf3\x3f\x4f\x24\x87\x10\x24\xd9\xc2\x77\x73\xa8\xdd" - } - , MACVector - { macString = "" - , macKey = B.pack [0x40 .. 0x5f] - , macSecret = B.pack [0x00 .. 0xc7] - , macResult = - digest - "\x75\x35\x8c\xf3\x9e\x41\x49\x4e\x94\x97\x07\x92\x7c\xee\x0a\xf2\x0a\x3f\xf5\x53\x90\x4c\x86\xb0\x8f\x21\xcc\x41\x4b\xcf\xd6\x91\x58\x9d\x27\xcf\x5e\x15\x36\x9c\xbb\xff\x8b\x9a\x4c\x2e\xb1\x78\x00\x85\x5d\x02\x35\xff\x63\x5d\xa8\x25\x33\xec\x6b\x75\x9b\x69" - } - , MACVector - { macString = "My Tagged Application" - , macKey = B.pack [0x40 .. 0x5f] - , macSecret = B.pack [0x00 .. 0xc7] - , macResult = - digest - "\xb5\x86\x18\xf7\x1f\x92\xe1\xd5\x6c\x1b\x8c\x55\xdd\xd7\xcd\x18\x8b\x97\xb4\xca\x4d\x99\x83\x1e\xb2\x69\x9a\x83\x7d\xa2\xe4\xd9\x70\xfb\xac\xfd\xe5\x00\x33\xae\xa5\x85\xf1\xa2\x70\x85\x10\xc3\x2d\x07\x88\x08\x01\xbd\x18\x28\x98\xfe\x47\x68\x76\xfc\x89\x65" - } - ] - -macTests :: [TestTree] -macTests = - [ testGroup "SHAKE128" (concatMap toMACTest $ zip is vectors128) - , testGroup "SHAKE256" (concatMap toMACTest $ zip is vectors256) - ] - where - toMACTest (i, MACVector{..}) = - [ testCase (show i) (macResult @=? KMAC.kmac macString macKey macSecret) - , testCase - ("incr-" ++ show i) - ( macResult - @=? KMAC.finalize (KMAC.update (KMAC.initialize macString macKey) macSecret) - ) - ] - is :: [Int] - is = [1 ..] - -data MacIncremental a = MacIncremental ByteString ByteString ByteString (KMAC.KMAC a) - deriving (Show, Eq) - -instance KMAC.HashSHAKE a => Arbitrary (MacIncremental a) where - arbitrary = do - str <- arbitraryBSof 0 49 - key <- arbitraryBSof 1 89 - msg <- arbitraryBSof 1 99 - return $ MacIncremental str key msg (KMAC.kmac str key msg) - -data MacIncrementalList a - = MacIncrementalList ByteString ByteString [ByteString] (KMAC.KMAC a) - deriving (Show, Eq) - -instance KMAC.HashSHAKE a => Arbitrary (MacIncrementalList a) where - arbitrary = do - str <- arbitraryBSof 0 49 - key <- arbitraryBSof 1 89 - msgs <- choose (1, 20) >>= \n -> replicateM n (arbitraryBSof 1 99) - return $ MacIncrementalList str key msgs (KMAC.kmac str key (B.concat msgs)) - -macIncrementalTests :: [TestTree] -macIncrementalTests = - [ testIncrProperties "SHAKE128_256" (SHAKE128 :: SHAKE128 256) - , testIncrProperties "SHAKE256_512" (SHAKE256 :: SHAKE256 512) - ] - where - testIncrProperties :: KMAC.HashSHAKE a => TestName -> a -> TestTree - testIncrProperties name a = - testGroup - name - [ testProperty "list-one" (prop_inc0 a) - , testProperty "list-multi" (prop_inc1 a) - ] - - prop_inc0 :: KMAC.HashSHAKE a => a -> MacIncremental a -> Bool - prop_inc0 _ (MacIncremental str secret msg result) = - result `assertEq` KMAC.finalize (KMAC.update (KMAC.initialize str secret) msg) - - prop_inc1 :: KMAC.HashSHAKE a => a -> MacIncrementalList a -> Bool - prop_inc1 _ (MacIncrementalList str secret msgs result) = - result - `assertEq` KMAC.finalize (foldl' KMAC.update (KMAC.initialize str secret) msgs) - -tests = - testGroup - "KMAC" - [ testGroup "KATs" macTests - , testGroup "properties" macIncrementalTests - ] diff --git a/tests/KAT_MiyaguchiPreneel.hs b/tests/KAT_MiyaguchiPreneel.hs deleted file mode 100644 --- a/tests/KAT_MiyaguchiPreneel.hs +++ /dev/null @@ -1,55 +0,0 @@ -module KAT_MiyaguchiPreneel (tests) where - -import Crypto.Cipher.AES (AES128) -import Crypto.ConstructHash.MiyaguchiPreneel as MiyaguchiPreneel - -import Imports - -import qualified Data.ByteArray as B -import Data.ByteArray.Encoding (Base (Base16), convertFromBase) -import qualified Data.ByteString.Char8 as B8 - -runMP128 :: ByteString -> ByteString -runMP128 s = B.convert (MiyaguchiPreneel.compute s :: MiyaguchiPreneel AES128) - -hxs :: String -> ByteString -hxs = - either (error . ("hxs:" ++)) id - . convertFromBase Base16 - . B8.pack - . filter (/= ' ') - -gAES128 :: TestTree -gAES128 = - igroup - "aes128" - [ runMP128 B8.empty - @?= hxs "66e94bd4 ef8a2c3b 884cfa59 ca342b2e" - , runMP128 (hxs "01000000 00000000 00000000 00000000") - @?= hxs "46711816 e91d6ff0 59bbbf2b f58e0fd3" - , runMP128 (hxs "00000000 00000000 00000000 00000001") - @?= hxs "58e2fcce fa7e3061 367f1d57 a4e7455b" - , runMP128 - ( hxs $ - "00000000 00000000 00000000 00000000" - ++ "01" - ) - @?= hxs "a5ff35ae 097adf5d 646abf5e bf4c16f4" - ] - -igroup :: TestName -> [Assertion] -> TestTree -igroup nm = testGroup nm . zipWith (flip ($)) [1 ..] . map icase - where - icase c i = testCase (show (i :: Int)) c - -vectors :: TestTree -vectors = - testGroup - "KATs" - [gAES128] - -tests :: TestTree -tests = - testGroup - "MiyaguchiPreneel" - [vectors] diff --git a/tests/KAT_OTP.hs b/tests/KAT_OTP.hs deleted file mode 100644 --- a/tests/KAT_OTP.hs +++ /dev/null @@ -1,130 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_OTP ( - tests, -) -where - -import Crypto.Hash.Algorithms (SHA1 (..), SHA256 (..), SHA512 (..)) -import Crypto.OTP -import Imports - --- | Test values from Appendix D of http://tools.ietf.org/html/rfc4226 -hotpExpected :: [(Word64, Word32)] -hotpExpected = - [ (0, 755224) - , (1, 287082) - , (3, 969429) - , (4, 338314) - , (5, 254676) - , (6, 287922) - , (7, 162583) - , (8, 399871) - , (9, 520489) - ] - --- | Test data from Appendix B of http://tools.ietf.org/html/rfc6238 --- Note that the shared keys for the non SHA-1 values are actually --- different (see the errata, or the Java example code). -totpSHA1Expected :: [(Word64, Word32)] -totpSHA1Expected = - [ (59, 94287082) - , (1111111109, 07081804) - , (1111111111, 14050471) - , (1234567890, 89005924) - , (2000000000, 69279037) - , (20000000000, 65353130) - ] - -totpSHA256Expected :: [(Word64, Word32)] -totpSHA256Expected = - [ (59, 46119246) - , (1111111109, 68084774) - , (1111111111, 67062674) - , (1234567890, 91819424) - , (2000000000, 90698825) - , (20000000000, 77737706) - ] - -totpSHA512Expected :: [(Word64, Word32)] -totpSHA512Expected = - [ (59, 90693936) - , (1111111109, 25091201) - , (1111111111, 99943326) - , (1234567890, 93441116) - , (2000000000, 38618901) - , (20000000000, 47863826) - ] - -otpKey :: ByteString -otpKey = "12345678901234567890" - -totpSHA256Key :: ByteString -totpSHA256Key = "12345678901234567890123456789012" - -totpSHA512Key :: ByteString -totpSHA512Key = - "1234567890123456789012345678901234567890123456789012345678901234" - -makeKATs :: (Eq a, Show a) => (t -> a) -> [(t, a)] -> [TestTree] -makeKATs otp expected = concatMap (makeTest otp) (zip3 is counts otps) - where - is :: [Int] - is = [1 ..] - - counts = map fst expected - otps = map snd expected - -makeTest :: (Eq a1, Show a2, Show a1) => (t -> a1) -> (a2, t, a1) -> [TestTree] -makeTest otp (i, count, password) = - [ testCase (show i) (assertEqual "" password (otp count)) - ] - -totpSHA1Params :: TOTPParams SHA1 -totpSHA1Params = case mkTOTPParams SHA1 0 30 OTP8 TwoSteps of - Right x -> x - _ -> error "totpSHA1Params" - -totpSHA256Params :: TOTPParams SHA256 -totpSHA256Params = case mkTOTPParams SHA256 0 30 OTP8 TwoSteps of - Right x -> x - _ -> error "totpSHA256Params" - -totpSHA512Params :: TOTPParams SHA512 -totpSHA512Params = case mkTOTPParams SHA512 0 30 OTP8 TwoSteps of - Right x -> x - _ -> error "totpSHA512Params" - --- resynching with the expected value should just return the current counter + 1 -prop_resyncExpected :: Word64 -> Word16 -> Bool -prop_resyncExpected ctr window = resynchronize SHA1 OTP6 window key ctr (otp, []) == Just (ctr + 1) - where - key = "1234" :: ByteString - otp = hotp SHA1 OTP6 key ctr - -tests :: TestTree -tests = - testGroup - "OTP" - [ testGroup - "HOTP" - [ testGroup "KATs" (makeKATs (hotp SHA1 OTP6 otpKey) hotpExpected) - , testGroup - "properties" - [ testProperty "resync-expected" prop_resyncExpected - ] - ] - , testGroup - "TOTP" - [ testGroup - "KATs" - [ testGroup "SHA1" (makeKATs (totp totpSHA1Params otpKey) totpSHA1Expected) - , testGroup - "SHA256" - (makeKATs (totp totpSHA256Params totpSHA256Key) totpSHA256Expected) - , testGroup - "SHA512" - (makeKATs (totp totpSHA512Params totpSHA512Key) totpSHA512Expected) - ] - ] - ] diff --git a/tests/KAT_PBKDF2.hs b/tests/KAT_PBKDF2.hs deleted file mode 100644 --- a/tests/KAT_PBKDF2.hs +++ /dev/null @@ -1,114 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - --- from -module KAT_PBKDF2 (tests) where - -import Crypto.Hash (SHA1 (..), SHA256 (..), SHA512 (..)) -import qualified Crypto.KDF.PBKDF2 as PBKDF2 - -import Data.ByteString (ByteString) -import Data.ByteString.Char8 () - -import Test.Tasty -import Test.Tasty.HUnit - -type VectParams = (ByteString, ByteString, Int, Int) - -vectors_hmac_sha1 :: [(VectParams, ByteString)] -vectors_hmac_sha1 = - [ - ( ("password", "salt", 2, 20) - , "\xea\x6c\x01\x4d\xc7\x2d\x6f\x8c\xcd\x1e\xd9\x2a\xce\x1d\x41\xf0\xd8\xde\x89\x57" - ) - , - ( ("password", "salt", 4096, 20) - , "\x4b\x00\x79\x01\xb7\x65\x48\x9a\xbe\xad\x49\xd9\x26\xf7\x21\xd0\x65\xa4\x29\xc1" - ) - , - ( ("passwordPASSWORDpassword", "saltSALTsaltSALTsaltSALTsaltSALTsalt", 4096, 25) - , "\x3d\x2e\xec\x4f\xe4\x1c\x84\x9b\x80\xc8\xd8\x36\x62\xc0\xe4\x4a\x8b\x29\x1a\x96\x4c\xf2\xf0\x70\x38" - ) - , - ( ("pass\0word", "sa\0lt", 4096, 16) - , "\x56\xfa\x6a\xa7\x55\x48\x09\x9d\xcc\x37\xd7\xf0\x34\x25\xe0\xc3" - ) - ] - -vectors_hmac_sha256 :: [(VectParams, ByteString)] -vectors_hmac_sha256 = - [ - ( ("password", "salt", 2, 32) - , "\xae\x4d\x0c\x95\xaf\x6b\x46\xd3\x2d\x0a\xdf\xf9\x28\xf0\x6d\xd0\x2a\x30\x3f\x8e\xf3\xc2\x51\xdf\xd6\xe2\xd8\x5a\x95\x47\x4c\x43" - ) - , - ( ("passwordPASSWORDpassword", "saltSALTsaltSALTsaltSALTsaltSALTsalt", 4096, 40) - , "\x34\x8c\x89\xdb\xcb\xd3\x2b\x2f\x32\xd8\x14\xb8\x11\x6e\x84\xcf\x2b\x17\x34\x7e\xbc\x18\x00\x18\x1c\x4e\x2a\x1f\xb8\xdd\x53\xe1\xc6\x35\x51\x8c\x7d\xac\x47\xe9" - ) - ] - -vectors_hmac_sha512 :: [(VectParams, ByteString)] -vectors_hmac_sha512 = - [ - ( ("password", "salt", 1, 32) - , "\x86\x7f\x70\xcf\x1a\xde\x02\xcf\xf3\x75\x25\x99\xa3\xa5\x3d\xc4\xaf\x34\xc7\xa6\x69\x81\x5a\xe5\xd5\x13\x55\x4e\x1c\x8c\xf2\x52" - ) - , - ( ("password", "salt", 2, 32) - , "\xe1\xd9\xc1\x6a\xa6\x81\x70\x8a\x45\xf5\xc7\xc4\xe2\x15\xce\xb6\x6e\x01\x1a\x2e\x9f\x00\x40\x71\x3f\x18\xae\xfd\xb8\x66\xd5\x3c" - ) - , - ( ("password", "salt", 4096, 32) - , "\xd1\x97\xb1\xb3\x3d\xb0\x14\x3e\x01\x8b\x12\xf3\xd1\xd1\x47\x9e\x6c\xde\xbd\xcc\x97\xc5\xc0\xf8\x7f\x69\x02\xe0\x72\xf4\x57\xb5" - ) - , - ( ("passwordPASSWORDpassword", "saltSALTsaltSALTsaltSALTsaltSALTsalt", 1, 72) - , "n\x23\xf2\x76\x38\x08\x4b\x0f\x7e\xa1\x73\x4e\x0d\x98\x41\xf5\x5d\xd2\x9e\xa6\x0a\x83\x44\x66\xf3\x39\x6b\xac\x80\x1f\xac\x1e\xeb\x63\x80\x2f\x03\xa0\xb4\xac\xd7\x60\x3e\x36\x99\xc8\xb7\x44\x37\xbe\x83\xff\x01\xad\x7f\x55\xda\xc1\xef\x60\xf4\xd5\x64\x80\xc3\x5e\xe6\x8f\xd5\x2c\x69\x36" - ) - ] - -tests = - testGroup - "PBKDF2" - [ testGroup "KATs-HMAC-SHA1" (katTests (PBKDF2.prfHMAC SHA1) vectors_hmac_sha1) - , testGroup "KATs-HMAC-SHA1 (fast)" (katTestFastPBKDF2_SHA1 vectors_hmac_sha1) - , testGroup - "KATs-HMAC-SHA256" - (katTests (PBKDF2.prfHMAC SHA256) vectors_hmac_sha256) - , testGroup - "KATs-HMAC-SHA256 (fast)" - (katTestFastPBKDF2_SHA256 vectors_hmac_sha256) - , testGroup - "KATs-HMAC-SHA512" - (katTests (PBKDF2.prfHMAC SHA512) vectors_hmac_sha512) - , testGroup - "KATs-HMAC-SHA512 (fast)" - (katTestFastPBKDF2_SHA512 vectors_hmac_sha512) - ] - where - katTests prf = zipWith (toKatTest prf) is - - toKatTest prf i ((pass, salt, iter, dkLen), output) = - testCase - (show i) - (output @=? PBKDF2.generate prf (PBKDF2.Parameters iter dkLen) pass salt) - - katTestFastPBKDF2_SHA1 = zipWith toKatTestFastPBKDF2_SHA1 is - toKatTestFastPBKDF2_SHA1 i ((pass, salt, iter, dkLen), output) = - testCase - (show i) - (output @=? PBKDF2.fastPBKDF2_SHA1 (PBKDF2.Parameters iter dkLen) pass salt) - - katTestFastPBKDF2_SHA256 = zipWith toKatTestFastPBKDF2_SHA256 is - toKatTestFastPBKDF2_SHA256 i ((pass, salt, iter, dkLen), output) = - testCase - (show i) - (output @=? PBKDF2.fastPBKDF2_SHA256 (PBKDF2.Parameters iter dkLen) pass salt) - - katTestFastPBKDF2_SHA512 = zipWith toKatTestFastPBKDF2_SHA512 is - toKatTestFastPBKDF2_SHA512 i ((pass, salt, iter, dkLen), output) = - testCase - (show i) - (output @=? PBKDF2.fastPBKDF2_SHA512 (PBKDF2.Parameters iter dkLen) pass salt) - - is :: [Int] - is = [1 ..] diff --git a/tests/KAT_PubKey.hs b/tests/KAT_PubKey.hs deleted file mode 100644 --- a/tests/KAT_PubKey.hs +++ /dev/null @@ -1,57 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey (tests) where - -import Test.Tasty -import Test.Tasty.HUnit - -import Data.ByteString (ByteString) -import qualified Data.ByteString as B -import Data.ByteString.Char8 () - -import Crypto.Hash -import Crypto.PubKey.MaskGenFunction - -import KAT_PubKey.DSA -import KAT_PubKey.ECC -import KAT_PubKey.ECDSA -import KAT_PubKey.OAEP -import qualified KAT_PubKey.P256 as P256 -import KAT_PubKey.PSS -import KAT_PubKey.RSA -import KAT_PubKey.Rabin -import Utils - -data VectorMgf = VectorMgf - { seed :: ByteString - , dbMask :: ByteString - } - -doMGFTest i vmgf = testCase (show i) (dbMask vmgf @=? actual) - where - actual = mgf1 SHA1 (seed vmgf) (B.length $ dbMask vmgf) - -vectorsMGF = - [ VectorMgf - { seed = - "\xdf\x1a\x89\x6f\x9d\x8b\xc8\x16\xd9\x7c\xd7\xa2\xc4\x3b\xad\x54\x6f\xbe\x8c\xfe" - , dbMask = - "\x66\xe4\x67\x2e\x83\x6a\xd1\x21\xba\x24\x4b\xed\x65\x76\xb8\x67\xd9\xa4\x47\xc2\x8a\x6e\x66\xa5\xb8\x7d\xee\x7f\xbc\x7e\x65\xaf\x50\x57\xf8\x6f\xae\x89\x84\xd9\xba\x7f\x96\x9a\xd6\xfe\x02\xa4\xd7\x5f\x74\x45\xfe\xfd\xd8\x5b\x6d\x3a\x47\x7c\x28\xd2\x4b\xa1\xe3\x75\x6f\x79\x2d\xd1\xdc\xe8\xca\x94\x44\x0e\xcb\x52\x79\xec\xd3\x18\x3a\x31\x1f\xc8\x97\x39\xa9\x66\x43\x13\x6e\x8b\x0f\x46\x5e\x87\xa4\x53\x5c\xd4\xc5\x9b\x10\x02\x8d" - } - ] - -tests = - testGroup - "PubKey" - [ testGroup "MGF1" $ zipWith doMGFTest [katZero ..] vectorsMGF - , rsaTests - , pssTests - , oaepTests - , dsaTests - , eccTests - , ecdsaTests - , P256.tests - , rabinTests - ] - --- newKats = [ eccKatTests ] diff --git a/tests/KAT_PubKey/DSA.hs b/tests/KAT_PubKey/DSA.hs deleted file mode 100644 --- a/tests/KAT_PubKey/DSA.hs +++ /dev/null @@ -1,434 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.DSA (dsaTests) where - -import Crypto.Hash -import qualified Crypto.PubKey.DSA as DSA - -import Imports - -data VectorDSA = VectorDSA - { pgq :: DSA.Params - , msg :: ByteString - , x :: Integer - , y :: Integer - , k :: Integer - , r :: Integer - , s :: Integer - } - -vectorsSHA1 = - [ VectorDSA - { msg = - "\x3b\x46\x73\x6d\x55\x9b\xd4\xe0\xc2\xc1\xb2\x55\x3a\x33\xad\x3c\x6c\xf2\x3c\xac\x99\x8d\x3d\x0c\x0e\x8f\xa4\xb1\x9b\xca\x06\xf2\xf3\x86\xdb\x2d\xcf\xf9\xdc\xa4\xf4\x0a\xd8\xf5\x61\xff\xc3\x08\xb4\x6c\x5f\x31\xa7\x73\x5b\x5f\xa7\xe0\xf9\xe6\xcb\x51\x2e\x63\xd7\xee\xa0\x55\x38\xd6\x6a\x75\xcd\x0d\x42\x34\xb5\xcc\xf6\xc1\x71\x5c\xca\xaf\x9c\xdc\x0a\x22\x28\x13\x5f\x71\x6e\xe9\xbd\xee\x7f\xc1\x3e\xc2\x7a\x03\xa6\xd1\x1c\x5c\x5b\x36\x85\xf5\x19\x00\xb1\x33\x71\x53\xbc\x6c\x4e\x8f\x52\x92\x0c\x33\xfa\x37\xf4\xe7" - , x = 0xc53eae6d45323164c7d07af5715703744a63fc3a - , y = - 0x313fd9ebca91574e1c2eebe1517c57e0c21b0209872140c5328761bbb2450b33f1b18b409ce9ab7c4cd8fda3391e8e34868357c199e16a6b2eba06d6749def791d79e95d3a4d09b24c392ad89dbf100995ae19c01062056bb14bce005e8731efde175f95b975089bdcdaea562b32786d96f5a31aedf75364008ad4fffebb970b - , k = 0x98cbcc4969d845e2461b5f66383dd503712bbcfa - , r = 0x50ed0e810e3f1c7cb6ac62332058448bd8b284c0 - , s = 0xc6aded17216b46b7e4b6f2a97c1ad7cc3da83fde - , pgq = dsaParams - } - , VectorDSA - { msg = - "\xd2\xbc\xb5\x3b\x04\x4b\x3e\x2e\x4b\x61\xba\x2f\x91\xc0\x99\x5f\xb8\x3a\x6a\x97\x52\x5e\x66\x44\x1a\x3b\x48\x9d\x95\x94\x23\x8b\xc7\x40\xbd\xee\xa0\xf7\x18\xa7\x69\xc9\x77\xe2\xde\x00\x38\x77\xb5\xd7\xdc\x25\xb1\x82\xae\x53\x3d\xb3\x3e\x78\xf2\xc3\xff\x06\x45\xf2\x13\x7a\xbc\x13\x7d\x4e\x7d\x93\xcc\xf2\x4f\x60\xb1\x8a\x82\x0b\xc0\x7c\x7b\x4b\x5f\xe0\x8b\x4f\x9e\x7d\x21\xb2\x56\xc1\x8f\x3b\x9d\x49\xac\xc4\xf9\x3e\x2c\xe6\xf3\x75\x4c\x78\x07\x75\x7d\x2e\x11\x76\x04\x26\x12\xcb\x32\xfc\x3f\x4f\x70\x70\x0e\x25" - , x = 0xe65131d73470f6ad2e5878bdc9bef536faf78831 - , y = - 0x29bdd759aaa62d4bf16b4861c81cf42eac2e1637b9ecba512bdbc13ac12a80ae8de2526b899ae5e4a231aef884197c944c732693a634d7659abc6975a773f8d3cd5a361fe2492386a3c09aaef12e4a7e73ad7dfc3637f7b093f2c40d6223a195c136adf2ea3fbf8704a675aa7817aa7ec7f9adfb2854d4e05c3ce7f76560313b - , k = 0x87256a64e98cf5be1034ecfa766f9d25d1ac7ceb - , r = 0xa26c00b5750a2d27fe7435b93476b35438b4d8ab - , s = 0x61c9bfcb2938755afa7dad1d1e07c6288617bf70 - , pgq = dsaParams - } - , VectorDSA - { msg = - "\xd5\x43\x1e\x6b\x16\xfd\xae\x31\x48\x17\x42\xbd\x39\x47\x58\xbe\xb8\xe2\x4f\x31\x94\x7e\x19\xb7\xea\x7b\x45\x85\x21\x88\x22\x70\xc1\xf4\x31\x92\xaa\x05\x0f\x44\x85\x14\x5a\xf8\xf3\xf9\xc5\x14\x2d\x68\xb8\x50\x18\xd2\xec\x9c\xb7\xa3\x7b\xa1\x2e\xd2\x3e\x73\xb9\x5f\xd6\x80\xfb\xa3\xc6\x12\x65\xe9\xf5\xa0\xa0\x27\xd7\x0f\xad\x0c\x8a\xa0\x8a\x3c\xbf\xbe\x99\x01\x8d\x00\x45\x38\x61\x73\xe5\xfa\xe2\x25\xfa\xeb\xe0\xce\xf5\xdd\x45\x91\x0f\x40\x0a\x86\xc2\xbe\x4e\x15\x25\x2a\x16\xde\x41\x20\xa2\x67\xbe\x2b\x59\x4d" - , x = 0x20bcabc6d9347a6e79b8e498c60c44a19c73258c - , y = - 0x23b4f404aa3c575e550bb320fdb1a085cd396a10e5ebc6771da62f037cab19eacd67d8222b6344038c4f7af45f5e62b55480cbe2111154ca9697ca76d87b56944138084e74c6f90a05cf43660dff8b8b3fabfcab3f0e4416775fdf40055864be102b4587392e77752ed2aeb182ee4f70be4a291dbe77b84a44ee34007957b1e0 - , k = 0x7d9bcfc9225432de9860f605a38d389e291ca750 - , r = 0x3f0a4ad32f0816821b8affb518e9b599f35d57c2 - , s = 0xea06638f2b2fc9d1dfe99c2a492806b497e2b0ea - , pgq = dsaParams - } - , VectorDSA - { msg = - "\x85\x66\x2b\x69\x75\x50\xe4\x91\x5c\x29\xe3\x38\xb6\x24\xb9\x12\x84\x5d\x6d\x1a\x92\x0d\x9e\x4c\x16\x04\xdd\x47\xd6\x92\xbc\x7c\x0f\xfb\x95\xae\x61\x4e\x85\x2b\xeb\xaf\x15\x73\x75\x8a\xd0\x1c\x71\x3c\xac\x0b\x47\x6e\x2f\x12\x17\x45\xa3\xcf\xee\xff\xb2\x44\x1f\xf6\xab\xfb\x9b\xbe\xb9\x8a\xa6\x34\xca\x6f\xf5\x41\x94\x7d\xcc\x99\x27\x65\x9d\x44\xf9\x5c\x5f\xf9\x17\x0f\xdc\x3c\x86\x47\x3c\xb6\x01\xba\x31\xb4\x87\xfe\x59\x36\xba\xc5\xd9\xc6\x32\xcb\xcc\x3d\xb0\x62\x46\xba\x01\xc5\x5a\x03\x8d\x79\x7f\xe3\xf6\xc3" - , x = 0x52d1fbe687aa0702a51a5bf9566bd51bd569424c - , y = - 0x6bc36cb3fa61cecc157be08639a7ca9e3de073b8a0ff23574ce5ab0a867dfd60669a56e60d1c989b3af8c8a43f5695d503e3098963990e12b63566784171058eace85c728cd4c08224c7a6efea75dca20df461013c75f40acbc23799ebee7f3361336dadc4a56f305708667bfe602b8ea75a491a5cf0c06ebd6fdc7161e10497 - , k = 0x960c211891c090d05454646ebac1bfe1f381e82b - , r = 0x3bc29dee96957050ba438d1b3e17b02c1725d229 - , s = 0x0af879cf846c434e08fb6c63782f4d03e0d88865 - , pgq = dsaParams - } - , VectorDSA - { msg = - "\x87\xb6\xe7\x5b\x9f\x8e\x99\xc4\xdd\x62\xad\xb6\x93\xdd\x58\x90\xed\xff\x1b\xd0\x02\x8f\x4e\xf8\x49\xdf\x0f\x1d\x2c\xe6\xb1\x81\xfc\x3a\x55\xae\xa6\xd0\xa1\xf0\xae\xca\xb8\xed\x9e\x24\x8a\x00\xe9\x6b\xe7\x94\xa7\xcf\xba\x12\x46\xef\xb7\x10\xef\x4b\x37\x47\x1c\xef\x0a\x1b\xcf\x55\xce\xbc\x8d\x5a\xd0\x71\x61\x2b\xd2\x37\xef\xed\xd5\x10\x23\x62\xdb\x07\xa1\xe2\xc7\xa6\xf1\x5e\x09\xfe\x64\xba\x42\xb6\x0a\x26\x28\xd8\x69\xae\x05\xef\x61\x1f\xe3\x8d\x9c\xe1\x5e\xee\xc9\xbb\x3d\xec\xc8\xdc\x17\x80\x9f\x3b\x6e\x95" - , x = 0xc86a54ec5c4ec63d7332cf43ddb082a34ed6d5f5 - , y = - 0x014ac746d3605efcb8a2c7dae1f54682a262e27662b252c09478ce87d0aaa522d7c200043406016c0c42896d21750b15dbd57f9707ec37dcea5651781b67ad8d01f5099fe7584b353b641bb159cc717d8ceb18b66705e656f336f1214b34f0357e577ab83641969e311bf40bdcb3ffd5e0bb59419f229508d2f432cc2859ff75 - , k = 0x6c445cee68042553fbe63be61be4ddb99d8134af - , r = 0x637e07a5770f3dc65e4506c68c770e5ef6b8ced3 - , s = 0x7dfc6f83e24f09745e01d3f7ae0ed1474e811d47 - , pgq = dsaParams - } - , VectorDSA - { msg = - "\x22\x59\xee\xad\x2d\x6b\xbc\x76\xd4\x92\x13\xea\x0d\xc8\xb7\x35\x0a\x97\x69\x9f\x22\x34\x10\x44\xc3\x94\x07\x82\x36\x4a\xc9\xea\x68\x31\x79\xa4\x38\xa5\xea\x45\x99\x8d\xf9\x7c\x29\x72\xda\xe0\x38\x51\xf5\xbe\x23\xfa\x9f\x04\x18\x2e\x79\xdd\xb2\xb5\x6d\xc8\x65\x23\x93\xec\xb2\x7f\x3f\x3b\x7c\x8a\x8d\x76\x1a\x86\xb3\xb8\xf4\xd4\x1a\x07\xb4\xbe\x7d\x02\xfd\xde\xfc\x42\xb9\x28\x12\x4a\x5a\x45\xb9\xf4\x60\x90\x42\x20\x9b\x3a\x7f\x58\x5b\xd5\x14\xcc\x39\xc0\x0e\xff\xcc\x42\xc7\xfe\x70\xfa\x83\xed\xf8\xa3\x2b\xf4" - , x = 0xaee6f213b9903c8069387e64729a08999e5baf65 - , y = - 0x0fe74045d7b0d472411202831d4932396f242a9765e92be387fd81bbe38d845054528b348c03984179b8e505674cb79d88cc0d8d3e8d7392f9aa773b29c29e54a9e326406075d755c291fcedbcc577934c824af988250f64ed5685fce726cff65e92d708ae11cbfaa958ab8d8b15340a29a137b5b4357f7ed1c7a5190cbf98a4 - , k = 0xe1704bae025942e2e63c6d76bab88da79640073a - , r = 0x83366ba3fed93dfb38d541203ecbf81c363998e2 - , s = 0x1fe299c36a1332f23bf2e10a6c6a4e0d3cdd2bf4 - , pgq = dsaParams - } - , VectorDSA - { msg = - "\x21\x9e\x8d\xf5\xbf\x88\x15\x90\x43\x0e\xce\x60\x82\x50\xf7\x67\x0d\xc5\x65\x37\x24\x93\x02\x42\x9e\x28\xec\xfe\xb9\xce\xaa\xa5\x49\x10\xa6\x94\x90\xf7\x65\xf3\xdf\x82\xe8\xb0\x1c\xd7\xd7\x6e\x56\x1d\x0f\x6c\xe2\x26\xef\x3c\xf7\x52\xca\xda\x6f\xeb\xdc\x5b\xf0\x0d\x67\x94\x7f\x92\xd4\x20\x51\x6b\x9e\x37\xc9\x6c\x8f\x1f\x2d\xa0\xb0\x75\x09\x7c\x3b\xda\x75\x8a\x8d\x91\xbd\x2e\xbe\x9c\x75\xcf\x14\x7f\x25\x4c\x25\x69\x63\xb3\x3b\x67\xd0\x2b\x6a\xa0\x9e\x7d\x74\x65\xd0\x38\xe5\x01\x95\xec\xe4\x18\x9b\x41\xe7\x68" - , x = 0x699f1c07aa458c6786e770b40197235fe49cf21a - , y = - 0x3a41b0678ff3c4dde20fa39772bac31a2f18bae4bedec9e12ee8e02e30e556b1a136013bef96b0d30b568233dcecc71e485ed75c922afb4d0654e709bee84993792130220e3005fdb06ebdfc0e2df163b5ec424e836465acd6d92e243c86f2b94b26b8d73bd9cf722c757e0b80b0af16f185de70e8ca850b1402d126ea60f309 - , k = 0x5bbb795bfa5fa72191fed3434a08741410367491 - , r = 0x579761039ae0ddb81106bf4968e320083bbcb947 - , s = 0x503ea15dbac9dedeba917fa8e9f386b93aa30353 - , pgq = dsaParams - } - , VectorDSA - { msg = - "\x2d\xa7\x9d\x06\x78\x85\xeb\x3c\xcf\x5e\x29\x3a\xe3\xb1\xd8\x22\x53\x22\x20\x3a\xbb\x5a\xdf\xde\x3b\x0f\x53\xbb\xe2\x4c\x4f\xe0\x01\x54\x1e\x11\x83\xd8\x70\xa9\x97\xf1\xf9\x46\x01\x00\xb5\xd7\x11\x92\x31\x80\x15\x43\x45\x28\x7a\x02\x14\xcf\x1c\xac\x37\xb7\xa4\x7d\xfb\xb2\xa0\xe8\xce\x49\x16\xf9\x4e\xbd\x6f\xa5\x4e\x31\x5b\x7a\x8e\xb5\xb6\x3c\xd9\x54\xc5\xba\x05\xc1\xbf\x7e\x33\xa4\xe8\xa1\x51\xf3\x2d\x28\x77\xb0\x17\x29\xc1\xad\x0e\x7c\x01\xbb\x8a\xe7\x23\xc9\x95\x18\x38\x03\xe4\x56\x36\x52\x0e\xa3\x8c\xa1" - , x = 0xd6e08c20c82949ddba93ea81eb2fea8c595894dc - , y = - 0x56f7272210f316c51af8bfc45a421fd4e9b1043853271b7e79f40936f0adcf262a86097aa86e19e6cb5307685d863dba761342db6c973b3849b1e060aca926f41fe07323601062515ae85f3172b8f34899c621d59fa21f73d5ae97a3deb5e840b25a18fd580862fd7b1cf416c7ae9fc5842a0197fdb0c5173ff4a4f102a8cf89 - , k = 0x6d72c30d4430959800740f2770651095d0c181c2 - , r = 0x5dd90d69add67a5fae138eec1aaff0229aa4afc4 - , s = 0x47f39c4db2387f10762f45b80dfd027906d7ef04 - , pgq = dsaParams - } - , VectorDSA - { msg = - "\xba\x30\xd8\x5b\xe3\x57\xe7\xfb\x29\xf8\xa0\x7e\x1f\x12\x7b\xaa\xa2\x4b\x2e\xe0\x27\xf6\x4c\xb5\xef\xee\xc6\xaa\xea\xbc\xc7\x34\x5c\x5d\x55\x6e\xbf\x4b\xdc\x7a\x61\xc7\x7c\x7b\x7e\xa4\x3c\x73\xba\xbc\x18\xf7\xb4\x80\x77\x22\xda\x23\x9e\x45\xdd\xf2\x49\x84\x9c\xbb\xfe\x35\x07\x11\x2e\xbf\x87\xd7\xef\x56\x0c\x2e\x7d\x39\x1e\xd8\x42\x4f\x87\x10\xce\xa4\x16\x85\x14\x3e\x30\x06\xf8\x1b\x68\xfb\xb4\xd5\xf9\x64\x4c\x7c\xd1\x0f\x70\x92\xef\x24\x39\xb8\xd1\x8c\x0d\xf6\x55\xe0\x02\x89\x37\x2a\x41\x66\x38\x5d\x64\x0c" - , x = 0x50018482864c1864e9db1f04bde8dbfd3875c76d - , y = - 0x0942a5b7a72ab116ead29308cf658dfe3d55d5d61afed9e3836e64237f9d6884fdd827d2d5890c9a41ae88e7a69fc9f345ade9c480c6f08cff067c183214c227236cedb6dd1283ca2a602574e8327510221d4c27b162143b7002d8c726916826265937b87be9d5ec6d7bd28fb015f84e0ab730da7a4eaf4ef3174bf0a22a6392 - , k = 0xdf3a9348f37b5d2d4c9176db266ae388f1fa7e0f - , r = 0x448434b214eee38bde080f8ec433e8d19b3ddf0d - , s = 0x0c02e881b777923fe0ea674f2621298e00199d5f - , pgq = dsaParams - } - , VectorDSA - { msg = - "\x83\x49\x9e\xfb\x06\xbb\x7f\xf0\x2f\xfb\x46\xc2\x78\xa5\xe9\x26\x30\xac\x5b\xc3\xf9\xe5\x3d\xd2\xe7\x8f\xf1\x5e\x36\x8c\x7e\x31\xaa\xd7\x7c\xf7\x71\xf3\x5f\xa0\x2d\x0b\x5f\x13\x52\x08\xa4\xaf\xdd\x86\x7b\xb2\xec\x26\xea\x2e\x7d\xd6\x4c\xde\xf2\x37\x50\x8a\x38\xb2\x7f\x39\xd8\xb2\x2d\x45\xca\xc5\xa6\x8a\x90\xb6\xea\x76\x05\x86\x45\xf6\x35\x6a\x93\x44\xd3\x6f\x00\xec\x66\x52\xea\xa4\xe9\xba\xe7\xb6\x94\xf9\xf1\xfc\x8c\x6c\x5e\x86\xfa\xdc\x7b\x27\xa2\x19\xb5\xc1\xb2\xae\x80\xa7\x25\xe5\xf6\x11\x65\xfe\x2e\xdc" - , x = 0xae56f66b0a9405b9cca54c60ec4a3bb5f8be7c3f - , y = - 0xa01542c3da410dd57930ca724f0f507c4df43d553c7f69459939685941ceb95c7dcc3f175a403b359621c0d4328e98f15f330a63865baf3e7eb1604a0715e16eed64fd14b35d3a534259a6a7ddf888c4dbb5f51bbc6ed339e5bb2a239d5cfe2100ac8e2f9c16e536f25119ab435843af27dc33414a9e4602f96d7c94d6021cec - , k = 0x8857ff301ad0169d164fa269977a116e070bac17 - , r = 0x8c2fab489c34672140415d41a65cef1e70192e23 - , s = 0x3df86a9e2efe944a1c7ea9c30cac331d00599a0e - , pgq = dsaParams - } - , VectorDSA -- 1024-bit example from RFC 6979 with SHA-1 - { msg = "sample" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x7BDB6B0FF756E1BB5D53583EF979082F9AD5BD5B - , r = 0x2E1A0C2562B2912CAAF89186FB0F42001585DA55 - , s = 0x29EFB6B0AFF2D7A68EB70CA313022253B9A88DF5 - , pgq = rfc6979Params1024 - } - , VectorDSA -- 1024-bit example from RFC 6979 with SHA-1 - { msg = "test" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x5C842DF4F9E344EE09F056838B42C7A17F4A6433 - , r = 0x42AB2052FD43E123F0607F115052A67DCD9C5C77 - , s = 0x183916B0230D45B9931491D4C6B0BD2FB4AAF088 - , pgq = rfc6979Params1024 - } - , VectorDSA -- 2048-bit example from RFC 6979 with SHA-1 - { msg = "sample" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x888FA6F7738A41BDC9846466ABDB8174C0338250AE50CE955CA16230F9CBD53E - , r = 0x3A1B2DBD7489D6ED7E608FD036C83AF396E290DBD602408E8677DAABD6E7445A - , s = 0xD26FCBA19FA3E3058FFC02CA1596CDBB6E0D20CB37B06054F7E36DED0CDBBCCF - , pgq = rfc6979Params2048 - } - , VectorDSA -- 2048-bit example from RFC 6979 with SHA-1 - { msg = "test" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x6EEA486F9D41A037B2C640BC5645694FF8FF4B98D066A25F76BE641CCB24BA4F - , r = 0xC18270A93CFC6063F57A4DFA86024F700D980E4CF4E2CB65A504397273D98EA0 - , s = 0x414F22E5F31A8B6D33295C7539C1C1BA3A6160D7D68D50AC0D3A5BEAC2884FAA - , pgq = rfc6979Params2048 - } - ] - where - -- (p,g,q) - dsaParams = - DSA.Params - { DSA.params_p = - 0xa8f9cd201e5e35d892f85f80e4db2599a5676a3b1d4f190330ed3256b26d0e80a0e49a8fffaaad2a24f472d2573241d4d6d6c7480c80b4c67bb4479c15ada7ea8424d2502fa01472e760241713dab025ae1b02e1703a1435f62ddf4ee4c1b664066eb22f2e3bf28bb70a2a76e4fd5ebe2d1229681b5b06439ac9c7e9d8bde283 - , DSA.params_g = - 0x2b3152ff6c62f14622b8f48e59f8af46883b38e79b8c74deeae9df131f8b856e3ad6c8455dab87cc0da8ac973417ce4f7878557d6cdf40b35b4a0ca3eb310c6a95d68ce284ad4e25ea28591611ee08b8444bd64b25f3f7c572410ddfb39cc728b9c936f85f419129869929cdb909a6a3a99bbe089216368171bd0ba81de4fe33 - , DSA.params_q = 0xf85f0f83ac4df7ea0cdf8f469bfeeaea14156495 - } - -vectorsSHA224 = - [ VectorDSA - { msg = "sample" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x562097C06782D60C3037BA7BE104774344687649 - , r = 0x4BC3B686AEA70145856814A6F1BB53346F02101E - , s = 0x410697B92295D994D21EDD2F4ADA85566F6F94C1 - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "test" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x4598B8EFC1A53BC8AECD58D1ABBB0C0C71E67297 - , r = 0x6868E9964E36C1689F6037F91F28D5F2C30610F2 - , s = 0x49CEC3ACDC83018C5BD2674ECAAD35B8CD22940F - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "sample" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0xBC372967702082E1AA4FCE892209F71AE4AD25A6DFD869334E6F153BD0C4D806 - , r = 0xDC9F4DEADA8D8FF588E98FED0AB690FFCE858DC8C79376450EB6B76C24537E2C - , s = 0xA65A9C3BC7BABE286B195D5DA68616DA8D47FA0097F36DD19F517327DC848CEC - , pgq = rfc6979Params2048 - } - , VectorDSA - { msg = "test" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x06BD4C05ED74719106223BE33F2D95DA6B3B541DAD7BFBD7AC508213B6DA6670 - , r = 0x272ABA31572F6CC55E30BF616B7A265312018DD325BE031BE0CC82AA17870EA3 - , s = 0xE9CC286A52CCE201586722D36D1E917EB96A4EBDB47932F9576AC645B3A60806 - , pgq = rfc6979Params2048 - } - ] - -vectorsSHA256 = - [ VectorDSA - { msg = "sample" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x519BA0546D0C39202A7D34D7DFA5E760B318BCFB - , r = 0x81F2F5850BE5BC123C43F71A3033E9384611C545 - , s = 0x4CDD914B65EB6C66A8AAAD27299BEE6B035F5E89 - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "test" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x5A67592E8128E03A417B0484410FB72C0B630E1A - , r = 0x22518C127299B0F6FDC9872B282B9E70D0790812 - , s = 0x6837EC18F150D55DE95B5E29BE7AF5D01E4FE160 - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "sample" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x8926A27C40484216F052F4427CFD5647338B7B3939BC6573AF4333569D597C52 - , r = 0xEACE8BDBBE353C432A795D9EC556C6D021F7A03F42C36E9BC87E4AC7932CC809 - , s = 0x7081E175455F9247B812B74583E9E94F9EA79BD640DC962533B0680793A38D53 - , pgq = rfc6979Params2048 - } - , VectorDSA - { msg = "test" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x1D6CE6DDA1C5D37307839CD03AB0A5CBB18E60D800937D67DFB4479AAC8DEAD7 - , r = 0x8190012A1969F9957D56FCCAAD223186F423398D58EF5B3CEFD5A4146A4476F0 - , s = 0x7452A53F7075D417B4B013B278D1BB8BBD21863F5E7B1CEE679CF2188E1AB19E - , pgq = rfc6979Params2048 - } - ] - -vectorsSHA384 = - [ VectorDSA - { msg = "sample" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x95897CD7BBB944AA932DBC579C1C09EB6FCFC595 - , r = 0x07F2108557EE0E3921BC1774F1CA9B410B4CE65A - , s = 0x54DF70456C86FAC10FAB47C1949AB83F2C6F7595 - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "test" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x220156B761F6CA5E6C9F1B9CF9C24BE25F98CD89 - , r = 0x854CF929B58D73C3CBFDC421E8D5430CD6DB5E66 - , s = 0x91D0E0F53E22F898D158380676A871A157CDA622 - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "sample" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0xC345D5AB3DA0A5BCB7EC8F8FB7A7E96069E03B206371EF7D83E39068EC564920 - , r = 0xB2DA945E91858834FD9BF616EBAC151EDBC4B45D27D0DD4A7F6A22739F45C00B - , s = 0x19048B63D9FD6BCA1D9BAE3664E1BCB97F7276C306130969F63F38FA8319021B - , pgq = rfc6979Params2048 - } - , VectorDSA - { msg = "test" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x206E61F73DBE1B2DC8BE736B22B079E9DACD974DB00EEBBC5B64CAD39CF9F91C - , r = 0x239E66DDBE8F8C230A3D071D601B6FFBDFB5901F94D444C6AF56F732BEB954BE - , s = 0x6BD737513D5E72FE85D1C750E0F73921FE299B945AAD1C802F15C26A43D34961 - , pgq = rfc6979Params2048 - } - ] - -vectorsSHA512 = - [ VectorDSA - { msg = "sample" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x09ECE7CA27D0F5A4DD4E556C9DF1D21D28104F8B - , r = 0x16C3491F9B8C3FBBDD5E7A7B667057F0D8EE8E1B - , s = 0x02C36A127A7B89EDBB72E4FFBC71DABC7D4FC69C - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "test" - , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 - , y = - 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B - , k = 0x65D2C2EEB175E370F28C75BFCDC028D22C7DBE9C - , r = 0x8EA47E475BA8AC6F2D821DA3BD212D11A3DEB9A0 - , s = 0x7C670C7AD72B6C050C109E1790008097125433E8 - , pgq = rfc6979Params1024 - } - , VectorDSA - { msg = "sample" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0x5A12994431785485B3F5F067221517791B85A597B7A9436995C89ED0374668FC - , r = 0x2016ED092DC5FB669B8EFB3D1F31A91EECB199879BE0CF78F02BA062CB4C942E - , s = 0xD0C76F84B5F091E141572A639A4FB8C230807EEA7D55C8A154A224400AFF2351 - , pgq = rfc6979Params2048 - } - , VectorDSA - { msg = "test" - , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC - , y = - 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF - , k = 0xAFF1651E4CD6036D57AA8B2A05CCF1A9D5A40166340ECBBDC55BE10B568AA0AA - , r = 0x89EC4BB1400ECCFF8E7D9AA515CD1DE7803F2DAFF09693EE7FD1353E90A68307 - , s = 0xC9F0BDABCC0D880BB137A994CC7F3980CE91CC10FAF529FC46565B15CEA854E1 - , pgq = rfc6979Params2048 - } - ] - -rfc6979Params1024 = - DSA.Params - { DSA.params_p = - 0x86F5CA03DCFEB225063FF830A0C769B9DD9D6153AD91D7CE27F787C43278B447E6533B86B18BED6E8A48B784A14C252C5BE0DBF60B86D6385BD2F12FB763ED8873ABFD3F5BA2E0A8C0A59082EAC056935E529DAF7C610467899C77ADEDFC846C881870B7B19B2B58F9BE0521A17002E3BDD6B86685EE90B3D9A1B02B782B1779 - , DSA.params_g = - 0x07B0F92546150B62514BB771E2A0C0CE387F03BDA6C56B505209FF25FD3C133D89BBCD97E904E09114D9A7DEFDEADFC9078EA544D2E401AEECC40BB9FBBF78FD87995A10A1C27CB7789B594BA7EFB5C4326A9FE59A070E136DB77175464ADCA417BE5DCE2F40D10A46A3A3943F26AB7FD9C0398FF8C76EE0A56826A8A88F1DBD - , DSA.params_q = 0x996F967F6C8E388D9E28D01E205FBA957A5698B1 - } - -rfc6979Params2048 = - DSA.Params - { DSA.params_p = - 0x9DB6FB5951B66BB6FE1E140F1D2CE5502374161FD6538DF1648218642F0B5C48C8F7A41AADFA187324B87674FA1822B00F1ECF8136943D7C55757264E5A1A44FFE012E9936E00C1D3E9310B01C7D179805D3058B2A9F4BB6F9716BFE6117C6B5B3CC4D9BE341104AD4A80AD6C94E005F4B993E14F091EB51743BF33050C38DE235567E1B34C3D6A5C0CEAA1A0F368213C3D19843D0B4B09DCB9FC72D39C8DE41F1BF14D4BB4563CA28371621CAD3324B6A2D392145BEBFAC748805236F5CA2FE92B871CD8F9C36D3292B5509CA8CAA77A2ADFC7BFD77DDA6F71125A7456FEA153E433256A2261C6A06ED3693797E7995FAD5AABBCFBE3EDA2741E375404AE25B - , DSA.params_g = - 0x5C7FF6B06F8F143FE8288433493E4769C4D988ACE5BE25A0E24809670716C613D7B0CEE6932F8FAA7C44D2CB24523DA53FBE4F6EC3595892D1AA58C4328A06C46A15662E7EAA703A1DECF8BBB2D05DBE2EB956C142A338661D10461C0D135472085057F3494309FFA73C611F78B32ADBB5740C361C9F35BE90997DB2014E2EF5AA61782F52ABEB8BD6432C4DD097BC5423B285DAFB60DC364E8161F4A2A35ACA3A10B1C4D203CC76A470A33AFDCBDD92959859ABD8B56E1725252D78EAC66E71BA9AE3F1DD2487199874393CD4D832186800654760E1E34C09E4D155179F9EC0DC4473F996BDCE6EED1CABED8B6F116F7AD9CF505DF0F998E34AB27514B0FFE7 - , DSA.params_q = - 0xF2C3119374CE76C9356990B465374A17F23F9ED35089BD969F61C6DDE9998C1F - } - -vectorToPrivate :: VectorDSA -> DSA.PrivateKey -vectorToPrivate vector = - DSA.PrivateKey - { DSA.private_x = x vector - , DSA.private_params = pgq vector - } - -vectorToPublic :: VectorDSA -> DSA.PublicKey -vectorToPublic vector = - DSA.PublicKey - { DSA.public_y = y vector - , DSA.public_params = pgq vector - } - -doSignatureTest hashAlg i vector = testCase (show i) (expected @=? actual) - where - expected = Just $ DSA.Signature (r vector) (s vector) - actual = DSA.signWith (k vector) (vectorToPrivate vector) hashAlg (msg vector) - -doVerifyTest hashAlg i vector = testCase (show i) (True @=? actual) - where - actual = - DSA.verify - hashAlg - (vectorToPublic vector) - (DSA.Signature (r vector) (s vector)) - (msg vector) - -dsaTests = - testGroup - "DSA" - [ testGroup - "SHA1" - [ testGroup "signature" $ zipWith (doSignatureTest SHA1) [katZero ..] vectorsSHA1 - , testGroup "verify" $ zipWith (doVerifyTest SHA1) [katZero ..] vectorsSHA1 - ] - , testGroup - "SHA224" - [ testGroup "signature" $ - zipWith (doSignatureTest SHA224) [katZero ..] vectorsSHA224 - , testGroup "verify" $ zipWith (doVerifyTest SHA224) [katZero ..] vectorsSHA224 - ] - , testGroup - "SHA256" - [ testGroup "signature" $ - zipWith (doSignatureTest SHA256) [katZero ..] vectorsSHA256 - , testGroup "verify" $ zipWith (doVerifyTest SHA256) [katZero ..] vectorsSHA256 - ] - , testGroup - "SHA384" - [ testGroup "signature" $ - zipWith (doSignatureTest SHA384) [katZero ..] vectorsSHA384 - , testGroup "verify" $ zipWith (doVerifyTest SHA384) [katZero ..] vectorsSHA384 - ] - , testGroup - "SHA512" - [ testGroup "signature" $ - zipWith (doSignatureTest SHA512) [katZero ..] vectorsSHA512 - , testGroup "verify" $ zipWith (doVerifyTest SHA512) [katZero ..] vectorsSHA512 - ] - ] diff --git a/tests/KAT_PubKey/ECC.hs b/tests/KAT_PubKey/ECC.hs deleted file mode 100644 --- a/tests/KAT_PubKey/ECC.hs +++ /dev/null @@ -1,235 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.ECC (eccTests, eccKatTests) where - -import Control.Arrow (second) - -import Data.List - -import qualified Crypto.PubKey.ECC.Prim as ECC -import qualified Crypto.PubKey.ECC.Types as ECC - -import Test.Tasty.KAT -import Test.Tasty.KAT.FileLoader - -import Imports - -instance Arbitrary ECC.Curve where - arbitrary = - ECC.getCurveByName - <$> elements - [ ECC.SEC_p112r1 - , ECC.SEC_p112r2 - , ECC.SEC_p128r1 - , ECC.SEC_p128r2 - , ECC.SEC_p160k1 - , ECC.SEC_p160r1 - , ECC.SEC_p160r2 - , ECC.SEC_p192k1 - , ECC.SEC_p192r1 - , ECC.SEC_p224k1 - , ECC.SEC_p224r1 - , ECC.SEC_p256k1 - , ECC.SEC_p256r1 - , ECC.SEC_p384r1 - , ECC.SEC_p521r1 - , ECC.SEC_t113r1 - , ECC.SEC_t113r2 - , ECC.SEC_t131r1 - , ECC.SEC_t131r2 - , ECC.SEC_t163k1 - , ECC.SEC_t163r1 - , ECC.SEC_t163r2 - , ECC.SEC_t193r1 - , ECC.SEC_t193r2 - , ECC.SEC_t233k1 - , ECC.SEC_t233r1 - , ECC.SEC_t239k1 - , ECC.SEC_t283k1 - , ECC.SEC_t283r1 - , ECC.SEC_t409k1 - , ECC.SEC_t409r1 - , ECC.SEC_t571k1 - , ECC.SEC_t571r1 - ] - -data VectorPoint = VectorPoint - { curve :: ECC.Curve - , x :: Integer - , y :: Integer - , valid :: Bool - } - -vectorsPoint :: [VectorPoint] -vectorsPoint = - [ VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x491c0c4761b0a4a147b5e4ce03a531546644f5d1e3d05e57 - , y = 0x6fa5addd47c5d6be3933fbff88f57a6c8ca0232c471965de - , valid = False -- point not on curve - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x646c22e8aa5f7833390e0399155ac198ae42470bba4fc834 - , y = 0x8d4afcfffd80e69a4d180178b37c44572495b7b267ee32a9 - , valid = True - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x4c6b9ea0dec92ecfff7799470be6a2277b9169daf45d54bb - , y = 0xf0eab42826704f51b26ae98036e83230becb639dd1964627 - , valid = False -- point not on curve - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x0673c8bb717b055c3d6f55c06acfcfb7260361ed3ec0f414 - , y = 0xba8b172826eb0b854026968d2338a180450a27906f6eddea - , valid = True - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x82c949295156192df0b52480e38c810751ac570daec460a3 - , y = 0x200057ada615c80b8ff256ce8d47f2562b74a438f1921ac3 - , valid = False -- point not on curve - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x284fbaa76ce0faae2ca4867d01092fa1ace5724cd12c8dd0 - , y = 0xe42af3dbf3206be3fcbcc3a7ccaf60c73dc29e7bb9b44fca - , valid = True - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x1b574acd4fb0f60dde3e3b5f3f0e94211f95112e43cba6fd2 - , y = 0xbcc1b8a770f01a22e84d7f14e44932ffe094d8e3b1e6ac26 - , valid = False -- x or y out of range - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x16ba109f1f1bb44e0d05b80181c03412ea764a59601d17e9f - , y = 0x0569a843dbb4e287db420d6b9fe30cd7b5d578b052315f56 - , valid = False -- x or y out of range - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x1333308a7c833ede5189d25ea3525919c9bd16370d904938d - , y = 0xb10fd01d67df75ff9b726c700c1b50596c9f0766ea56f80e - , valid = False -- x or y out of range - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x9671ec444cff24c8a5be80b018fa505ed6109a731e88c91a - , y = 0xfe79dae23008e46bf4230c895aab261a95845a77f06d0655 - , valid = True - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0x158e8b6f0b14216bc52fe8897b4305d870ede70436a96741d - , y = 0xfb3f970b19a313571a1a23be310923f85acc1cab0a157cbd - , valid = False -- x or y out of range - } - , VectorPoint - { curve = ECC.getCurveByName ECC.SEC_p192r1 - , x = 0xace95b650c08f73dbb4fa7b4bbdebd6b809a25b28ed135ef - , y = 0xe9b8679404166d1329dd539ad52aad9a1b6681f5f26bb9aa - , valid = False -- point not on curve - } - ] - -doPointValidTest :: Show a => a -> VectorPoint -> TestTree -doPointValidTest i vector = - testCase - (show i) - ( valid vector - @=? ECC.isPointValid (curve vector) (ECC.Point (x vector) (y vector)) - ) - -arbitraryPoint :: ECC.Curve -> Gen ECC.Point -arbitraryPoint aCurve = - frequency [(5, return ECC.PointO), (95, pointGen)] - where - n = ECC.ecc_n (ECC.common_curve aCurve) - pointGen = ECC.pointBaseMul aCurve <$> choose (1, n - 1) - -eccTests :: TestTree -eccTests = - testGroup - "ECC" - [ testGroup "valid-point" $ zipWith doPointValidTest [katZero ..] vectorsPoint - , localOption (QuickCheckTests 20) $ - testGroup - "property" - [ testProperty "point-add" $ \aCurve (QAInteger r1) (QAInteger r2) -> - let curveN = ECC.ecc_n . ECC.common_curve $ aCurve - curveGen = ECC.ecc_g . ECC.common_curve $ aCurve - p1 = ECC.pointMul aCurve r1 curveGen - p2 = ECC.pointMul aCurve r2 curveGen - pR = ECC.pointMul aCurve ((r1 + r2) `mod` curveN) curveGen - in pR `propertyEq` ECC.pointAdd aCurve p1 p2 - , testProperty "point-negate-add" $ \aCurve -> do - p <- arbitraryPoint aCurve - let o = ECC.pointAdd aCurve p (ECC.pointNegate aCurve p) - return $ ECC.PointO `propertyEq` o - , testProperty "point-negate-negate" $ \aCurve -> do - p <- arbitraryPoint aCurve - return $ p `propertyEq` ECC.pointNegate aCurve (ECC.pointNegate aCurve p) - , testProperty "point-mul-mul" $ \aCurve (QAInteger n1) (QAInteger n2) -> do - p <- arbitraryPoint aCurve - let pRes = ECC.pointMul aCurve (n1 * n2) p - let pDef = ECC.pointMul aCurve n1 (ECC.pointMul aCurve n2 p) - return $ pRes `propertyEq` pDef - , testProperty "double-scalar-mult" $ \aCurve (QAInteger n1) (QAInteger n2) -> do - p1 <- arbitraryPoint aCurve - p2 <- arbitraryPoint aCurve - let pRes = ECC.pointAddTwoMuls aCurve n1 p1 n2 p2 - let pDef = ECC.pointAdd aCurve (ECC.pointMul aCurve n1 p1) (ECC.pointMul aCurve n2 p2) - return $ pRes `propertyEq` pDef - ] - ] - -eccKatTests :: IO TestTree -eccKatTests = do - res <- - testKatLoad "KATs/ECC-PKV.txt" (map (second (map toVector)) . katLoaderSimple) - return $ - testKatDetailed {-Grouped-} - "ECC/valid-point" - res - ( \g vect -> do - let mCurve = - ECC.getCurveByName <$> case g of - "P-192" -> Just ECC.SEC_p192r1 - "P-224" -> Just ECC.SEC_p224r1 - "P-256" -> Just ECC.SEC_p256r1 - "P-384" -> Just ECC.SEC_p384r1 - "P-521" -> Just ECC.SEC_p521r1 - "B-163" -> Just ECC.SEC_t163r2 - "B-233" -> Just ECC.SEC_t233r1 - "B-283" -> Just ECC.SEC_t283r1 - "B-409" -> Just ECC.SEC_t409r1 - "B-571" -> Just ECC.SEC_t571r1 - "" -> Nothing - _ -> Nothing - {- - "K-163" -> Just ECC.SEC_t163k1 - "K-233" -> Just ECC.SEC_t233k1 - "K-283" -> Just ECC.SEC_t283k1 - "K-409" -> Just ECC.SEC_t409k1 - "K-571" -> Just ECC.SEC_t571k1 - -} - case mCurve of - Nothing -> return True - Just c -> do - return (ECC.isPointValid c (ECC.Point (x vect) (y vect)) == valid vect) - ) - where - toVector kvs = - case sequence $ map (flip lookup kvs) ["Qx", "Qy", "Result"] of - Just [qx, qy, res] -> - VectorPoint - undefined - (valueHexInteger qx) - (valueHexInteger qy) - ("F" `isPrefixOf` res) - Just _ -> error ("ERROR: " ++ show kvs) - Nothing -> error ("ERROR: " ++ show kvs) -- VectorPoint undefined 0 0 True diff --git a/tests/KAT_PubKey/ECDSA.hs b/tests/KAT_PubKey/ECDSA.hs deleted file mode 100644 --- a/tests/KAT_PubKey/ECDSA.hs +++ /dev/null @@ -1,1636 +0,0 @@ -{-# LANGUAGE ExistentialQuantification #-} -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.ECDSA (ecdsaTests) where - -import Crypto.Hash -import Crypto.Number.Serialize -import Crypto.PubKey.ECC.ECDSA ( - PrivateKey (..), - PublicKey (..), - Signature (..), - deterministicNonce, - signWith, - verify, - ) -import Crypto.PubKey.ECC.Generate -import Crypto.PubKey.ECC.Types -import Data.ByteString (ByteString) -import qualified Data.ByteString as B -import Test.Tasty -import Test.Tasty.HUnit -import Text.Printf - --- existential type allows storing different hash algorithms in the same value -data HashAlg = forall hash. (Show hash, HashAlgorithm hash) => HashAlg hash -instance Show HashAlg where show (HashAlg alg) = show alg - -data Entry = Entry - { curveName :: CurveName - , privateNumber :: PrivateNumber - , publicPoint :: PublicPoint - , hashAlgorithm :: HashAlg - , message :: ByteString - , nonce :: Integer - , signature :: Signature - } -instance Show Entry where - show entry = - printf - "%s.%s.%s" - (show $ curveName entry) - (show $ B.take 8 $ message entry) - (show $ hashAlgorithm entry) - -normalize :: Entry -> Entry -normalize entry - | s <= n `div` 2 = entry - | otherwise = entry{signature = Signature r (n - s)} - where - Signature r s = signature entry - n = ecc_n $ common_curve $ getCurveByName $ curveName entry - --- taken from GEC 2: Test Vectors for SEC 1 -gec2Entries :: [Entry] -gec2Entries = - [ Entry - { curveName = SEC_p160r1 - , privateNumber = 971761939728640320549601132085879836204587084162 - , publicPoint = - Point - 466448783855397898016055842232266600516272889280 - 1110706324081757720403272427311003102474457754220 - , hashAlgorithm = HashAlg SHA1 - , message = "abc" - , nonce = 702232148019446860144825009548118511996283736794 - , signature = - Signature - { sign_r = 1176954224688105769566774212902092897866168635793 - , sign_s = 299742580584132926933316745664091704165278518100 - } - } - , Entry - { curveName = SEC_t163k1 - , privateNumber = 0x00000011f2626d90d26cb4c0379043b26e64107fc - , publicPoint = - Point - 0x0389fa5ad7f8304325a8c060ef7dcb83042c045bc - 0x0eefa094a5054da196943cc80509dcb9f59e5bc2e - , hashAlgorithm = HashAlg SHA1 - , message = - i2osp - 0xa2c1a03fdd00521bb08fc88d20344321977aaf637ef9d5470dd7d2c8628fc8d0d1f1d3587c6b3fd02386f8c13db341b14748a9475cc63baf065df64054b27d5c2cdf0f98e3bbb81d0b5dc94f8cdb87acf75720f6163de394c8c6af360bc1acb85b923a493b7b27cc111a257e36337bd94eb0fab9d5e633befb1ae7f1b244bfaa - , nonce = 0x0000000c3a4ff97286126dab1e5089395fcc47ebb - , signature = - Signature - { sign_r = 0x0dbe6c3a1dc851e7f2338b5c26c62b4b37bf8035c - , sign_s = 0x1c76458135b1ff9fbd23009b8414a47996126b56a - } - } - , Entry - { curveName = SEC_t163k1 - , privateNumber = 0x00000006a3803301daee9af09bb5b6c991a4f49a4 - , publicPoint = - Point - 0x4b500f555e857da8c299780130c5c3f48f02ee322 - 0x5c1c0ae25b47f06cc46fb86b12d2d8c0ba6a4bf07 - , hashAlgorithm = HashAlg SHA1 - , message = - i2osp - 0x67048080daaeb77d3ac31babdf8be23dbe75ceb4dfb94aa8113db5c5dcb6fe14b70f717b7b0ed0881835a66a86e6d840ffcb7d976c75ef2d1d4322fbbc86357384e24707aef88cea2c41a01a9a3d1b9e72ce650c7fdecc4f9448d3a77df6cdf13647ab295bb3132de0b1b2c402d8d2de7d452f1e003e0695de1470d1064eee16 - , nonce = 0x0000002f39fbf77f3e0dc046116de692b6cf91b16 - , signature = - Signature - { sign_r = 0x3d3eeda42f65d727f4a564f1415654356c6c57a6c - , sign_s = 0x35e4d43c5f08baddf138449db1ad0b7872552b7cd - } - } - , Entry - { curveName = SEC_t163k1 - , privateNumber = 0x0000002e28676514bd93fea11b62db0f6e324b18d - , publicPoint = - Point - 0x3f9c90b71f6a1de20a2716f38ef1b5f98c757bd42 - 0x2ff0a5d266d447ef62d43fbca6c34c08c1ce35a40 - , hashAlgorithm = HashAlg SHA1 - , message = - i2osp - 0x77e007dc2acd7248256165a4b30e98986f51a81efd926b85f74c81bc2a6d2bcd030060a844091e22fbb0ff3db5a20caaefb5d58ccdcbc27f0ff8a4d940e78f303079ec1ca5b0ca3d4ecc7580f8b34a9f0496c9e719d2ec3e1614b7644bc11179e895d2c0b58a1da204fbf0f6e509f97f983eacb6487092caf6e8e4e6b3c458b2 - , nonce = 0x00000001233ae699883e74e7f4dfb5279ff22280a - , signature = - Signature - { sign_r = 0x39de3cd2cf04145e522b8fba3f23e9218226e0860 - , sign_s = 0x2af62bfb3cfa202e2342606ee5bb0934c3b0375b6 - } - } - , Entry - { curveName = SEC_t163k1 - , privateNumber = 0x000000361dd088e3a6d3c910686c8dce57e5d4d8e - , publicPoint = - Point - 0x064f905c1da9d7e9c32d81890ae6f30dcc7839d32 - 0x06f1faedb6d9032016d3b681e7cf69c29d29eb27b - , hashAlgorithm = HashAlg SHA1 - , message = - i2osp - 0xfbacfcce4688748406ddf5c3495021eef8fb399865b649eb2395a04a1ab28335da2c236d306fcc59f7b65ea931cf0139571e1538ede5688958c3ac69f47a285362f5ad201f89cc735b7b465408c2c41b310fc8908d0be45054df2a7351fae36b390e842f3b5cdd9ad832940df5b2d25c2ed43ce86eaf2508bcf401ae58bb1d47 - , nonce = 0x00000022f723e9f5da56d3d0837d5dca2f937395f - , signature = - Signature - { sign_r = 0x374cdc8571083fecfbd4e25e1cd69ecc66b715f2d - , sign_s = 0x313b10949222929b2f20b15d446c27d6dcae3f086 - } - } - ] - -data EntryCurve = EntryCurve - { ecName :: CurveName - , ecPrivate :: PrivateNumber - , ecPublic :: PublicPoint - , ecMessages :: [EntryMessage] - } -data EntryMessage = EntryMessage - { emMessage :: ByteString - , emHashes :: [EntryHash] - } -data EntryHash = EntryHash - { ehAlgorithm :: HashAlg - , ehK :: Integer - , ehR :: Integer - , ehS :: Integer - } - -flatten :: [EntryCurve] -> [Entry] -flatten hierarchy = do - entryCurve <- hierarchy - entryMessage <- ecMessages entryCurve - entryHash <- emHashes entryMessage - pure $ - Entry - { curveName = ecName entryCurve - , privateNumber = ecPrivate entryCurve - , publicPoint = ecPublic entryCurve - , hashAlgorithm = ehAlgorithm entryHash - , message = emMessage entryMessage - , nonce = ehK entryHash - , signature = Signature (ehR entryHash) (ehS entryHash) - } - --- taken from RFC 6979 -rfc6979Entries :: [EntryCurve] -rfc6979Entries = - [ EntryCurve - { ecName = SEC_p192r1 - , ecPrivate = 0x6FAB034934E4C0FC9AE67F5B5659A9D7D1FEFD187EE09FD4 - , ecPublic = - Point - 0xAC2C77F529F91689FEA0EA5EFEC7F210D8EEA0B9E047ED56 - 0x3BC723E57670BD4887EBC732C523063D0A7C957BC97C1C43 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x37D7CA00D2C7B0E5E412AC03BD44BA837FDD5B28CD3B0021 - , ehR = 0x98C6BD12B23EAF5E2A2045132086BE3EB8EBD62ABF6698FF - , ehS = 0x57A22B07DEA9530F8DE9471B1DC6624472E8E2844BC25B64 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x4381526B3FC1E7128F202E194505592F01D5FF4C5AF015D8 - , ehR = 0xA1F00DAD97AEEC91C95585F36200C65F3C01812AA60378F5 - , ehS = 0xE07EC1304C7C6C9DEBBE980B9692668F81D4DE7922A0F97A - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x32B1B6D7D42A05CB449065727A84804FB1A3E34D8F261496 - , ehR = 0x4B0B8CE98A92866A2820E20AA6B75B56382E0F9BFD5ECB55 - , ehS = 0xCCDB006926EA9565CBADC840829D8C384E06DE1F1E381B85 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x4730005C4FCB01834C063A7B6760096DBE284B8252EF4311 - , ehR = 0xDA63BF0B9ABCF948FBB1E9167F136145F7A20426DCC287D5 - , ehS = 0xC3AA2C960972BD7A2003A57E1C4C77F0578F8AE95E31EC5E - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0xA2AC7AB055E4F20692D49209544C203A7D1F2C0BFBC75DB1 - , ehR = 0x4D60C5AB1996BD848343B31C00850205E2EA6922DAC2E4B8 - , ehS = 0x3F6E837448F027A1BF4B34E796E32A811CBB4050908D8F67 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0xD9CF9C3D3297D3260773A1DA7418DB5537AB8DD93DE7FA25 - , ehR = 0x0F2141A0EBBC44D2E1AF90A50EBCFCE5E197B3B7D4DE036D - , ehS = 0xEB18BC9E1F3D7387500CB99CF5F7C157070A8961E38700B7 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0xF5DC805F76EF851800700CCE82E7B98D8911B7D510059FBE - , ehR = 0x6945A1C1D1B2206B8145548F633BB61CEF04891BAF26ED34 - , ehS = 0xB7FB7FDFC339C0B9BD61A9F5A8EAF9BE58FC5CBA2CB15293 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x5C4CE89CF56D9E7C77C8585339B006B97B5F0680B4306C6C - , ehR = 0x3A718BD8B4926C3B52EE6BBE67EF79B18CB6EB62B1AD97AE - , ehS = 0x5662E6848A4A19B1F1AE2F72ACD4B8BBE50F1EAC65D9124F - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x5AFEFB5D3393261B828DB6C91FBC68C230727B030C975693 - , ehR = 0xB234B60B4DB75A733E19280A7A6034BD6B1EE88AF5332367 - , ehS = 0x7994090B2D59BB782BE57E74A44C9A1C700413F8ABEFE77A - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x0758753A5254759C7CFBAD2E2D9B0792EEE44136C9480527 - , ehR = 0xFE4F4AE86A58B6507946715934FE2D8FF9D95B6B098FE739 - , ehS = 0x74CF5605C98FBA0E1EF34D4B5A1577A7DCF59457CAE52290 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_p224r1 - , ecPrivate = 0xF220266E1105BFE3083E03EC7A3A654651F45E37167E88600BF257C1 - , ecPublic = - Point - 0x00CF08DA5AD719E42707FA431292DEA11244D64FC51610D94B130D6C - 0xEEAB6F3DEBE455E3DBF85416F7030CBD94F34F2D6F232C69F3C1385A - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x7EEFADD91110D8DE6C2C470831387C50D3357F7F4D477054B8B426BC - , ehR = 0x22226F9D40A96E19C4A301CE5B74B115303C0F3A4FD30FC257FB57AC - , ehS = 0x66D1CDD83E3AF75605DD6E2FEFF196D30AA7ED7A2EDF7AF475403D69 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0xC1D1F2F10881088301880506805FEB4825FE09ACB6816C36991AA06D - , ehR = 0x1CDFE6662DDE1E4A1EC4CDEDF6A1F5A2FB7FBD9145C12113E6ABFD3E - , ehS = 0xA6694FD7718A21053F225D3F46197CA699D45006C06F871808F43EBC - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0xAD3029E0278F80643DE33917CE6908C70A8FF50A411F06E41DEDFCDC - , ehR = 0x61AA3DA010E8E8406C656BC477A7A7189895E7E840CDFE8FF42307BA - , ehS = 0xBC814050DAB5D23770879494F9E0A680DC1AF7161991BDE692B10101 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x52B40F5A9D3D13040F494E83D3906C6079F29981035C7BD51E5CAC40 - , ehR = 0x0B115E5E36F0F9EC81F1325A5952878D745E19D7BB3EABFABA77E953 - , ehS = 0x830F34CCDFE826CCFDC81EB4129772E20E122348A2BBD889A1B1AF1D - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x9DB103FFEDEDF9CFDBA05184F925400C1653B8501BAB89CEA0FBEC14 - , ehR = 0x074BD1D979D5F32BF958DDC61E4FB4872ADCAFEB2256497CDAC30397 - , ehS = 0xA4CECA196C3D5A1FF31027B33185DC8EE43F288B21AB342E5D8EB084 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x2519178F82C3F0E4F87ED5883A4E114E5B7A6E374043D8EFD329C253 - , ehR = 0xDEAA646EC2AF2EA8AD53ED66B2E2DDAA49A12EFD8356561451F3E21C - , ehS = 0x95987796F6CF2062AB8135271DE56AE55366C045F6D9593F53787BD2 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0xDF8B38D40DCA3E077D0AC520BF56B6D565134D9B5F2EAE0D34900524 - , ehR = 0xC441CE8E261DED634E4CF84910E4C5D1D22C5CF3B732BB204DBEF019 - , ehS = 0x902F42847A63BDC5F6046ADA114953120F99442D76510150F372A3F4 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0xFF86F57924DA248D6E44E8154EB69F0AE2AEBAEE9931D0B5A969F904 - , ehR = 0xAD04DDE87B84747A243A631EA47A1BA6D1FAA059149AD2440DE6FBA6 - , ehS = 0x178D49B1AE90E3D8B629BE3DB5683915F4E8C99FDF6E666CF37ADCFD - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x7046742B839478C1B5BD31DB2E862AD868E1A45C863585B5F22BDC2D - , ehR = 0x389B92682E399B26518A95506B52C03BC9379A9DADF3391A21FB0EA4 - , ehS = 0x414A718ED3249FF6DBC5B50C27F71F01F070944DA22AB1F78F559AAB - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0xE39C2AA4EA6BE2306C72126D40ED77BF9739BB4D6EF2BBB1DCB6169D - , ehR = 0x049F050477C5ADD858CAC56208394B5A55BAEBBE887FDF765047C17C - , ehS = 0x077EB13E7005929CEFA3CD0403C7CDCC077ADF4E44F3C41B2F60ECFF - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_p256r1 - , ecPrivate = 0xC9AFA9D845BA75166B5C215767B1D6934E50C3DB36E89B127B8A622B120F6721 - , ecPublic = - Point - 0x60FED4BA255A9D31C961EB74C6356D68C049B8923B61FA6CE669622E60F29FB6 - 0x7903FE1008B8BC99A41AE9E95628BC64F2F1B20C2D7E9F5177A3C294D4462299 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x882905F1227FD620FBF2ABF21244F0BA83D0DC3A9103DBBEE43A1FB858109DB4 - , ehR = 0x61340C88C3AAEBEB4F6D667F672CA9759A6CCAA9FA8811313039EE4A35471D32 - , ehS = 0x6D7F147DAC089441BB2E2FE8F7A3FA264B9C475098FDCF6E00D7C996E1B8B7EB - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x103F90EE9DC52E5E7FB5132B7033C63066D194321491862059967C715985D473 - , ehR = 0x53B2FFF5D1752B2C689DF257C04C40A587FABABB3F6FC2702F1343AF7CA9AA3F - , ehS = 0xB9AFB64FDC03DC1A131C7D2386D11E349F070AA432A4ACC918BEA988BF75C74C - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0xA6E3C57DD01ABE90086538398355DD4C3B17AA873382B0F24D6129493D8AAD60 - , ehR = 0xEFD48B2AACB6A8FD1140DD9CD45E81D69D2C877B56AAF991C34D0EA84EAF3716 - , ehS = 0xF7CB1C942D657C41D436C7A1B6E29F65F3E900DBB9AFF4064DC4AB2F843ACDA8 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x09F634B188CEFD98E7EC88B1AA9852D734D0BC272F7D2A47DECC6EBEB375AAD4 - , ehR = 0x0EAFEA039B20E9B42309FB1D89E213057CBF973DC0CFC8F129EDDDC800EF7719 - , ehS = 0x4861F0491E6998B9455193E34E7B0D284DDD7149A74B95B9261F13ABDE940954 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x5FA81C63109BADB88C1F367B47DA606DA28CAD69AA22C4FE6AD7DF73A7173AA5 - , ehR = 0x8496A60B5E9B47C825488827E0495B0E3FA109EC4568FD3F8D1097678EB97F00 - , ehS = 0x2362AB1ADBE2B8ADF9CB9EDAB740EA6049C028114F2460F96554F61FAE3302FE - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x8C9520267C55D6B980DF741E56B4ADEE114D84FBFA2E62137954164028632A2E - , ehR = 0x0CBCC86FD6ABD1D99E703E1EC50069EE5C0B4BA4B9AC60E409E8EC5910D81A89 - , ehS = 0x01B9D7B73DFAA60D5651EC4591A0136F87653E0FD780C3B1BC872FFDEAE479B1 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x669F4426F2688B8BE0DB3A6BD1989BDAEFFF84B649EEB84F3DD26080F667FAA7 - , ehR = 0xC37EDB6F0AE79D47C3C27E962FA269BB4F441770357E114EE511F662EC34A692 - , ehS = 0xC820053A05791E521FCAAD6042D40AEA1D6B1A540138558F47D0719800E18F2D - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0xD16B6AE827F17175E040871A1C7EC3500192C4C92677336EC2537ACAEE0008E0 - , ehR = 0xF1ABB023518351CD71D881567B1EA663ED3EFCF6C5132B354F28D3B0B7D38367 - , ehS = 0x019F4113742A2B14BD25926B49C649155F267E60D3814B4C0CC84250E46F0083 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x16AEFFA357260B04B1DD199693960740066C1A8F3E8EDD79070AA914D361B3B8 - , ehR = 0x83910E8B48BB0C74244EBDF7F07A1C5413D61472BD941EF3920E623FBCCEBEB6 - , ehS = 0x8DDBEC54CF8CD5874883841D712142A56A8D0F218F5003CB0296B6B509619F2C - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x6915D11632ACA3C40D5D51C08DAF9C555933819548784480E93499000D9F0B7F - , ehR = 0x461D93F31B6540894788FD206C07CFA0CC35F46FA3C91816FFF1040AD1581A04 - , ehS = 0x39AF9F15DE0DB8D97E72719C74820D304CE5226E32DEDAE67519E840D1194E55 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_p384r1 - , ecPrivate = - 0x6B9D3DAD2E1B8C1C05B19875B6659F4DE23C3B667BF297BA9AA47740787137D896D5724E4C70A825F872C9EA60D2EDF5 - , ecPublic = - Point - 0xEC3A4E415B4E19A4568618029F427FA5DA9A8BC4AE92E02E06AAE5286B300C64DEF8F0EA9055866064A254515480BC13 - 0x8015D9B72D7D57244EA8EF9AC0C621896708A59367F9DFB9F54CA84B3F1C9DB1288B231C3AE0D4FE7344FD2533264720 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x4471EF7518BB2C7C20F62EAE1C387AD0C5E8E470995DB4ACF694466E6AB096630F29E5938D25106C3C340045A2DB01A7 - , ehR = - 0xEC748D839243D6FBEF4FC5C4859A7DFFD7F3ABDDF72014540C16D73309834FA37B9BA002899F6FDA3A4A9386790D4EB2 - , ehS = - 0xA3BCFA947BEEF4732BF247AC17F71676CB31A847B9FF0CBC9C9ED4C1A5B3FACF26F49CA031D4857570CCB5CA4424A443 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0xA4E4D2F0E729EB786B31FC20AD5D849E304450E0AE8E3E341134A5C1AFA03CAB8083EE4E3C45B06A5899EA56C51B5879 - , ehR = - 0x42356E76B55A6D9B4631C865445DBE54E056D3B3431766D0509244793C3F9366450F76EE3DE43F5A125333A6BE060122 - , ehS = - 0x9DA0C81787064021E78DF658F2FBB0B042BF304665DB721F077A4298B095E4834C082C03D83028EFBF93A3C23940CA8D - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x180AE9F9AEC5438A44BC159A1FCB277C7BE54FA20E7CF404B490650A8ACC414E375572342863C899F9F2EDF9747A9B60 - , ehR = - 0x21B13D1E013C7FA1392D03C5F99AF8B30C570C6F98D4EA8E354B63A21D3DAA33BDE1E888E63355D92FA2B3C36D8FB2CD - , ehS = - 0xF3AA443FB107745BF4BD77CB3891674632068A10CA67E3D45DB2266FA7D1FEEBEFDC63ECCD1AC42EC0CB8668A4FA0AB0 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x94ED910D1A099DAD3254E9242AE85ABDE4BA15168EAF0CA87A555FD56D10FBCA2907E3E83BA95368623B8C4686915CF9 - , ehR = - 0x94EDBB92A5ECB8AAD4736E56C691916B3F88140666CE9FA73D64C4EA95AD133C81A648152E44ACF96E36DD1E80FABE46 - , ehS = - 0x99EF4AEB15F178CEA1FE40DB2603138F130E740A19624526203B6351D0A3A94FA329C145786E679E7B82C71A38628AC8 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x92FC3C7183A883E24216D1141F1A8976C5B0DD797DFA597E3D7B32198BD35331A4E966532593A52980D0E3AAA5E10EC3 - , ehR = - 0xED0959D5880AB2D869AE7F6C2915C6D60F96507F9CB3E047C0046861DA4A799CFE30F35CC900056D7C99CD7882433709 - , ehS = - 0x512C8CCEEE3890A84058CE1E22DBC2198F42323CE8ACA9135329F03C068E5112DC7CC3EF3446DEFCEB01A45C2667FDD5 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x66CC2C8F4D303FC962E5FF6A27BD79F84EC812DDAE58CF5243B64A4AD8094D47EC3727F3A3C186C15054492E30698497 - , ehR = - 0x4BC35D3A50EF4E30576F58CD96CE6BF638025EE624004A1F7789A8B8E43D0678ACD9D29876DAF46638645F7F404B11C7 - , ehS = - 0xD5A6326C494ED3FF614703878961C0FDE7B2C278F9A65FD8C4B7186201A2991695BA1C84541327E966FA7B50F7382282 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x18FA39DB95AA5F561F30FA3591DC59C0FA3653A80DAFFA0B48D1A4C6DFCBFF6E3D33BE4DC5EB8886A8ECD093F2935726 - , ehR = - 0xE8C9D0B6EA72A0E7837FEA1D14A1A9557F29FAA45D3E7EE888FC5BF954B5E62464A9A817C47FF78B8C11066B24080E72 - , ehS = - 0x07041D4A7A0379AC7232FF72E6F77B6DDB8F09B16CCE0EC3286B2BD43FA8C6141C53EA5ABEF0D8231077A04540A96B66 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x0CFAC37587532347DC3389FDC98286BBA8C73807285B184C83E62E26C401C0FAA48DD070BA79921A3457ABFF2D630AD7 - , ehR = - 0x6D6DEFAC9AB64DABAFE36C6BF510352A4CC27001263638E5B16D9BB51D451559F918EEDAF2293BE5B475CC8F0188636B - , ehS = - 0x2D46F3BECBCC523D5F1A1256BF0C9B024D879BA9E838144C8BA6BAEB4B53B47D51AB373F9845C0514EEFB14024787265 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x015EE46A5BF88773ED9123A5AB0807962D193719503C527B031B4C2D225092ADA71F4A459BC0DA98ADB95837DB8312EA - , ehR = - 0x8203B63D3C853E8D77227FB377BCF7B7B772E97892A80F36AB775D509D7A5FEB0542A7F0812998DA8F1DD3CA3CF023DB - , ehS = - 0xDDD0760448D42D8A43AF45AF836FCE4DE8BE06B485E9B61B827C2F13173923E06A739F040649A667BF3B828246BAA5A5 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x3780C4F67CB15518B6ACAE34C9F83568D2E12E47DEAB6C50A4E4EE5319D1E8CE0E2CC8A136036DC4B9C00E6888F66B6C - , ehR = - 0xA0D5D090C9980FAF3C2CE57B7AE951D31977DD11C775D314AF55F76C676447D06FB6495CD21B4B6E340FC236584FB277 - , ehS = - 0x976984E59B4C77B0E8E4460DCA3D9F20E07B9BB1F63BEEFAF576F6B2E8B224634A2092CD3792E0159AD9CEE37659C736 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_p521r1 - , ecPrivate = - 0x0FAD06DAA62BA3B25D2FB40133DA757205DE67F5BB0018FEE8C86E1B68C7E75CAA896EB32F1F47C70855836A6D16FCC1466F6D8FBEC67DB89EC0C08B0E996B83538 - , ecPublic = - Point - 0x1894550D0785932E00EAA23B694F213F8C3121F86DC97A04E5A7167DB4E5BCD371123D46E45DB6B5D5370A7F20FB633155D38FFA16D2BD761DCAC474B9A2F5023A4 - 0x0493101C962CD4D2FDDF782285E64584139C2F91B47F87FF82354D6630F746A28A0DB25741B5B34A828008B22ACC23F924FAAFBD4D33F81EA66956DFEAA2BFDFCF5 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x089C071B419E1C2820962321787258469511958E80582E95D8378E0C2CCDB3CB42BEDE42F50E3FA3C71F5A76724281D31D9C89F0F91FC1BE4918DB1C03A5838D0F9 - , ehR = - 0x0343B6EC45728975EA5CBA6659BBB6062A5FF89EEA58BE3C80B619F322C87910FE092F7D45BB0F8EEE01ED3F20BABEC079D202AE677B243AB40B5431D497C55D75D - , ehS = - 0x0E7B0E675A9B24413D448B8CC119D2BF7B2D2DF032741C096634D6D65D0DBE3D5694625FB9E8104D3B842C1B0E2D0B98BEA19341E8676AEF66AE4EBA3D5475D5D16 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x121415EC2CD7726330A61F7F3FA5DE14BE9436019C4DB8CB4041F3B54CF31BE0493EE3F427FB906393D895A19C9523F3A1D54BB8702BD4AA9C99DAB2597B92113F3 - , ehR = - 0x1776331CFCDF927D666E032E00CF776187BC9FDD8E69D0DABB4109FFE1B5E2A30715F4CC923A4A5E94D2503E9ACFED92857B7F31D7152E0F8C00C15FF3D87E2ED2E - , ehS = - 0x050CB5265417FE2320BBB5A122B8E1A32BD699089851128E360E620A30C7E17BA41A666AF126CE100E5799B153B60528D5300D08489CA9178FB610A2006C254B41F - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x0EDF38AFCAAECAB4383358B34D67C9F2216C8382AAEA44A3DAD5FDC9C32575761793FEF24EB0FC276DFC4F6E3EC476752F043CF01415387470BCBD8678ED2C7E1A0 - , ehR = - 0x1511BB4D675114FE266FC4372B87682BAECC01D3CC62CF2303C92B3526012659D16876E25C7C1E57648F23B73564D67F61C6F14D527D54972810421E7D87589E1A7 - , ehS = - 0x04A171143A83163D6DF460AAF61522695F207A58B95C0644D87E52AA1A347916E4F7A72930B1BC06DBE22CE3F58264AFD23704CBB63B29B931F7DE6C9D949A7ECFC - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x1546A108BC23A15D6F21872F7DED661FA8431DDBD922D0DCDB77CC878C8553FFAD064C95A920A750AC9137E527390D2D92F153E66196966EA554D9ADFCB109C4211 - , ehR = - 0x1EA842A0E17D2DE4F92C15315C63DDF72685C18195C2BB95E572B9C5136CA4B4B576AD712A52BE9730627D16054BA40CC0B8D3FF035B12AE75168397F5D50C67451 - , ehS = - 0x1F21A3CEE066E1961025FB048BD5FE2B7924D0CD797BABE0A83B66F1E35EEAF5FDE143FA85DC394A7DEE766523393784484BDF3E00114A1C857CDE1AA203DB65D61 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x1DAE2EA071F8110DC26882D4D5EAE0621A3256FC8847FB9022E2B7D28E6F10198B1574FDD03A9053C08A1854A168AA5A57470EC97DD5CE090124EF52A2F7ECBFFD3 - , ehR = - 0x0C328FAFCBD79DD77850370C46325D987CB525569FB63C5D3BC53950E6D4C5F174E25A1EE9017B5D450606ADD152B534931D7D4E8455CC91F9B15BF05EC36E377FA - , ehS = - 0x0617CCE7CF5064806C467F678D3B4080D6F1CC50AF26CA209417308281B68AF282623EAA63E5B5C0723D8B8C37FF0777B1A20F8CCB1DCCC43997F1EE0E44DA4A67A - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x0BB9F2BF4FE1038CCF4DABD7139A56F6FD8BB1386561BD3C6A4FC818B20DF5DDBA80795A947107A1AB9D12DAA615B1ADE4F7A9DC05E8E6311150F47F5C57CE8B222 - , ehR = - 0x13BAD9F29ABE20DE37EBEB823C252CA0F63361284015A3BF430A46AAA80B87B0693F0694BD88AFE4E661FC33B094CD3B7963BED5A727ED8BD6A3A202ABE009D0367 - , ehS = - 0x1E9BB81FF7944CA409AD138DBBEE228E1AFCC0C890FC78EC8604639CB0DBDC90F717A99EAD9D272855D00162EE9527567DD6A92CBD629805C0445282BBC916797FF - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x040D09FCF3C8A5F62CF4FB223CBBB2B9937F6B0577C27020A99602C25A01136987E452988781484EDBBCF1C47E554E7FC901BC3085E5206D9F619CFF07E73D6F706 - , ehR = - 0x1C7ED902E123E6815546065A2C4AF977B22AA8EADDB68B2C1110E7EA44D42086BFE4A34B67DDC0E17E96536E358219B23A706C6A6E16BA77B65E1C595D43CAE17FB - , ehS = - 0x177336676304FCB343CE028B38E7B4FBA76C1C1B277DA18CAD2A8478B2A9A9F5BEC0F3BA04F35DB3E4263569EC6AADE8C92746E4C82F8299AE1B8F1739F8FD519A4 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x01DE74955EFAABC4C4F17F8E84D881D1310B5392D7700275F82F145C61E843841AF09035BF7A6210F5A431A6A9E81C9323354A9E69135D44EBD2FCAA7731B909258 - , ehR = - 0x00E871C4A14F993C6C7369501900C4BC1E9C7B0B4BA44E04868B30B41D8071042EB28C4C250411D0CE08CD197E4188EA4876F279F90B3D8D74A3C76E6F1E4656AA8 - , ehS = - 0x0CD52DBAA33B063C3A6CD8058A1FB0A46A4754B034FCC644766CA14DA8CA5CA9FDE00E88C1AD60CCBA759025299079D7A427EC3CC5B619BFBC828E7769BCD694E86 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x1F1FC4A349A7DA9A9E116BFDD055DC08E78252FF8E23AC276AC88B1770AE0B5DCEB1ED14A4916B769A523CE1E90BA22846AF11DF8B300C38818F713DADD85DE0C88 - , ehR = - 0x14BEE21A18B6D8B3C93FAB08D43E739707953244FDBE924FA926D76669E7AC8C89DF62ED8975C2D8397A65A49DCC09F6B0AC62272741924D479354D74FF6075578C - , ehS = - 0x133330865C067A0EAF72362A65E2D7BC4E461E8C8995C3B6226A21BD1AA78F0ED94FE536A0DCA35534F0CD1510C41525D163FE9D74D134881E35141ED5E8E95B979 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x16200813020EC986863BEDFC1B121F605C1215645018AEA1A7B215A564DE9EB1B38A67AA1128B80CE391C4FB71187654AAA3431027BFC7F395766CA988C964DC56D - , ehR = - 0x13E99020ABF5CEE7525D16B69B229652AB6BDF2AFFCAEF38773B4B7D08725F10CDB93482FDCC54EDCEE91ECA4166B2A7C6265EF0CE2BD7051B7CEF945BABD47EE6D - , ehS = - 0x1FBD0013C674AA79CB39849527916CE301C66EA7CE8B80682786AD60F98F7E78A19CA69EFF5C57400E3B3A0AD66CE0978214D13BAF4E9AC60752F7B155E2DE4DCE3 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t163k1 - , ecPrivate = 0x09A4D6792295A7F730FC3F2B49CBC0F62E862272F - , ecPublic = - Point - 0x79AEE090DB05EC252D5CB4452F356BE198A4FF96F - 0x782E29634DDC9A31EF40386E896BAA18B53AFA5A3 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x09744429FA741D12DE2BE8316E35E84DB9E5DF1CD - , ehR = 0x30C45B80BA0E1406C4EFBBB7000D6DE4FA465D505 - , ehS = 0x38D87DF89493522FC4CD7DE1553BD9DBBA2123011 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x323E7B28BFD64E6082F5B12110AA87BC0D6A6E159 - , ehR = 0x38A2749F7EA13BD5DA0C76C842F512D5A65FFAF32 - , ehS = 0x064F841F70112B793FD773F5606BFA5AC2A04C1E8 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x23AF4074C90A02B3FE61D286D5C87F425E6BDD81B - , ehR = 0x113A63990598A3828C407C0F4D2438D990DF99A7F - , ehS = 0x1313A2E03F5412DDB296A22E2C455335545672D9F - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x2132ABE0ED518487D3E4FA7FD24F8BED1F29CCFCE - , ehR = 0x34D4DE955871BB84FEA4E7D068BA5E9A11BD8B6C4 - , ehS = 0x2BAAF4D4FD57F175C405A2F39F9755D9045C820BD - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x00BBCC2F39939388FDFE841892537EC7B1FF33AA3 - , ehR = 0x38E487F218D696A7323B891F0CCF055D895B77ADC - , ehS = 0x0972D7721093F9B3835A5EB7F0442FA8DCAA873C4 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x14CAB9192F39C8A0EA8E81B4B87574228C99CD681 - , ehR = 0x1375BEF93F21582F601497036A7DC8014A99C2B79 - , ehS = 0x254B7F1472FFFEE9002D081BB8CE819CCE6E687F9 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x091DD986F38EB936BE053DD6ACE3419D2642ADE8D - , ehR = 0x110F17EF209957214E35E8C2E83CBE73B3BFDEE2C - , ehS = 0x057D5022392D359851B95DEC2444012502A5349CB - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x193649CE51F0CFF0784CFC47628F4FA854A93F7A2 - , ehR = 0x0354D5CD24F9C41F85D02E856FA2B0001C83AF53E - , ehS = 0x020B200677731CD4FE48612A92F72A19853A82B65 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x37C73C6F8B404EC83DA17A6EBCA724B3FF1F7EEBA - , ehR = 0x11B6A84206515495AD8DBB2E5785D6D018D75817E - , ehS = 0x1A7D4C1E17D4030A5D748ADEA785C77A54581F6D0 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x331AD98D3186F73967B1E0B120C80B1E22EFC2988 - , ehR = 0x148934745B351F6367FF5BB56B1848A2F508902A9 - , ehS = 0x36214B19444FAB504DBA61D4D6FF2D2F9640F4837 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t233k1 - , ecPrivate = 0x103B2142BDC2A3C3B55080D09DF1808F79336DA2399F5CA7171D1BE9B0 - , ecPublic = - Point - 0x0682886F36C68473C1A221720C2B12B9BE13458BA907E1C4736595779F2 - 0x1B20639B41BE0927090999B7817A3B3928D20503A39546044EC13A10309 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x273179E3E12C69591AD3DD9C7CCE3985820E3913AB6696EB14486DDBCF - , ehR = 0x5474541C988A9A1F73899F55EF28963DFFBBF0C2B1A1EE787C6A76C6A4 - , ehS = 0x46301F9EC6624257BFC70D72186F17898EDBD0A3522560A88DD1B7D45A - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x71626A309D9CD80AD0B975D757FE6BF4B84E49F8F34C780070D7746F19 - , ehR = 0x667F2FCE3E1C497EBD8E4B7C6372A8234003FE4ED6D4515814E7E11430 - , ehS = 0x6A1C41340DAA730320DB9475F10E29A127D7AE3432F155E1F7954E1B57 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x73552F9CAC5774F74F485FA253871F2109A0C86040552EAA67DBA92DC9 - , ehR = 0x38AD9C1D2CB29906E7D63C24601AC55736B438FB14F4093D6C32F63A10 - , ehS = 0x647AAD2599C21B6EE89BE7FF957D98F684B7921DE1FD3CC82C079624F4 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x17D726A67539C609BD99E29AA3737EF247724B71455C3B6310034038C8 - , ehR = 0x0C6510F57559C36FBCFF8C7BA4B81853DC618AD0BAAB03CFFDF3FD09FD - , ehS = 0x0AD331EE1C9B91A88BA77997235769C60AD07EE69E11F7137E17C5CF67 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x0E535C328774CDE546BE3AF5D7FCD263872F107E807435105BA2FDC166 - , ehR = 0x47C4AC1B344028CC740BA7BB9F8AA59D6390E3158153D4F2ADE4B74950 - , ehS = 0x26CE0CDE18A1B884B3EE1A879C13B42F11BB7C85F7A3745C8BECEC8E6E - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x1D8BBF5CB6EFFA270A1CDC22C81E269F0CC16E27151E0A460BA9B51AFF - , ehR = 0x4780B2DE4BAA5613872179AD90664249842E8B96FCD5653B55DD63EED4 - , ehS = 0x6AF46BA322E21D4A88DAEC1650EF38774231276266D6A45ED6A64ECB44 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x67634D0ABA2C9BF7AE54846F26DCD166E7100654BCE6FDC96667631AA2 - , ehR = 0x61D9CC8C842DF19B3D9F4BDA0D0E14A957357ADABC239444610FB39AEA - , ehS = 0x66432278891CB594BA8D08A0C556053D15917E53449E03C2EF88474CF6 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x2CE5AEDC155ACC0DDC5E679EBACFD21308362E5EFC05C5E99B2557A8D7 - , ehR = 0x05E4E6B4DB0E13034E7F1F2E5DBAB766D37C15AE4056C7EE607C8AC7F4 - , ehS = 0x5FC46AA489BF828B34FBAD25EC432190F161BEA8F60D3FCADB0EE3B725 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x1B4BD3903E74FD0B31E23F956C70062014DFEFEE21832032EA5352A055 - , ehR = 0x50F1EFEDFFEC1088024620280EE0D7641542E4D4B5D61DB32358FC571B - , ehS = 0x4614EAE449927A9EB2FCC42EA3E955B43D194087719511A007EC9217A5 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x1775ED919CA491B5B014C5D5E86AF53578B5A7976378F192AF665CB705 - , ehR = 0x6FE6D0D3A953BB66BB01BC6B9EDFAD9F35E88277E5768D1B214395320F - , ehS = 0x7C01A236E4BFF0A771050AD01EC1D24025D3130BBD9E4E81978EB3EC09 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t283k1 - , ecPrivate = - 0x06A0777356E87B89BA1ED3A3D845357BE332173C8F7A65BDC7DB4FAB3C4CC79ACC8194E - , ecPublic = - Point - 0x25330D0A651D5A20DC6389BC02345117725640AEC3C126612CE444EDD19649BDECC03D6 - 0x505BD60A4B67182474EC4D1C668A73140F70504A68F39EFCD972487E9530E0508A76193 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x0A96F788DECAF6C9DBE24DC75ABA6EAAE85E7AB003C8D4F83CB1540625B2993BF445692 - , ehR = 0x1B66D1E33FBDB6E107A69B610995C93C744CEBAEAF623CB42737C27D60188BD1D045A68 - , ehS = 0x02E45B62C9C258643532FD536594B46C63B063946494F95DAFF8759FD552502324295C5 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x1B4C4E3B2F6B08B5991BD2BDDE277A7016DA527AD0AAE5BC61B64C5A0EE63E8B502EF61 - , ehR = 0x018CF2F371BE86BB62E02B27CDE56DDAC83CCFBB3141FC59AEE022B66AC1A60DBBD8B76 - , ehS = 0x1854E02A381295EA7F184CEE71AB7222D6974522D3B99B309B1A8025EB84118A28BF20E - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x1CEB9E8E0DFF53CE687DEB81339ACA3C98E7A657D5A9499EF779F887A934408ECBE5A38 - , ehR = 0x19E90AA3DE5FB20AED22879F92C6FED278D9C9B9293CC5E94922CD952C9DBF20DF1753A - , ehS = 0x135AA7443B6A25D11BB64AC482E04D47902D017752882BD72527114F46CF8BB56C5A8C3 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x1460A5C41745A5763A9D548AE62F2C3630BBED71B6AA549D7F829C22442A728C5D965DA - , ehR = 0x0F8C1CA9C221AD9907A136F787D33BA56B0495A40E86E671C940FD767EDD75EB6001A49 - , ehS = 0x1071A56915DEE89E22E511975AA09D00CDC4AA7F5054CBE83F5977EE6F8E1CC31EC43FD - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x00F3B59FCB5C1A01A1A2A0019E98C244DFF61502D6E6B9C4E957EDDCEB258EF4DBEF04A - , ehR = 0x1D0008CF4BA4A701BEF70771934C2A4A87386155A2354140E2ED52E18553C35B47D9E50 - , ehS = 0x0D15F4FA1B7A4D41D9843578E22EF98773179103DC4FF0DD1F74A6B5642841B91056F78 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x168B5F8C0881D4026C08AC5894A2239D219FA9F4DA0600ADAA56D5A1781AF81F08A726E - , ehR = 0x140932FA7307666A8CCB1E1A09656CC40F5932965841ABD5E8E43559D93CF2311B02767 - , ehS = 0x16A2FD46DA497E5E739DED67F426308C45C2E16528BF2A17EB5D65964FD88B770FBB9C6 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x045E13EA645CE01D9B25EA38C8A8A170E04C83BB7F231EE3152209FE10EC8B2E565536C - , ehR = 0x0E72AF7E39CD72EF21E61964D87C838F977485FA6A7E999000AFA97A381B2445FCEE541 - , ehS = 0x1644FF7D848DA1A040F77515082C27C763B1B4BF332BCF5D08251C6B57D806319778208 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x0B585A7A68F51089691D6EDE2B43FC4451F66C10E65F134B963D4CBD4EB844B0E1469A6 - , ehR = 0x158FAEB2470B306C57764AFC8528174589008449E11DB8B36994B607A65956A59715531 - , ehS = 0x0521BC667CA1CA42B5649E78A3D76823C678B7BB3CD58D2E93CD791D53043A6F83F1FD1 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x1E88738E14482A09EE16A73D490A7FE8739DF500039538D5C4B6C8D6D7F208D6CA56760 - , ehR = 0x1CC4DC5479E0F34C4339631A45AA690580060BF0EB518184C983E0E618C3B93AAB14BBE - , ehS = 0x0284D72FF8AFA83DE364502CBA0494BB06D40AE08F9D9746E747EA87240E589BA0683B7 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x00E5F24A223BD459653F682763C3BB322D4EE75DD89C63D4DC61518D543E76585076BBA - , ehR = 0x1E7912517C6899732E09756B1660F6B96635D638283DF9A8A11D30E008895D7F5C9C7F3 - , ehS = 0x0887E75CBD0B7DD9DE30ED79BDB3D78E4F1121C5EAFF5946918F594F88D363644789DA7 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t409k1 - , ecPrivate = - 0x29C16768F01D1B8A89FDA85E2EFD73A09558B92A178A2931F359E4D70AD853E569CDAF16DAA569758FB4E73089E4525D8BBFCF - , ecPublic = - Point - 0x0CF923F523FE34A6E863D8BA45FB1FE6D784C8F219C414EEF4DB8362DBBD3CA71AEB28F568668D5D7A0093E2B84F6FAD759DB42 - 0x13B1C374D5132978A1B1123EBBE9A5C54D1A9D56B09AFDB4ADE93CCD7C4D332E2916F7D4B9D18578EE3C2E2DE4D2ECE0DE63549 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x7866E5247F9A3556F983C86E81EDA696AC8489DB40A2862F278603982D304F08B2B6E1E7848534BEAF1330D37A1CF84C7994C1 - , ehR = - 0x7192EE99EC7AFE23E02CB1F9850D1ECE620475EDA6B65D04984029408EC1E5A6476BC940D81F218FC31D979814CAC6E78340FA - , ehS = - 0x1DE75DE97CBE740FC79A6B5B22BC2B7832C687E6960F0B8173D5D8BE2A75AC6CA43438BAF69C669CE6D64E0FB93BC5854E0F81 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x512340DB682C7B8EBE407BF1AA54194DFE85D49025FE0F632C9B8A06A996F2FCD0D73C752FB09D23DB8FBE50605DC25DF0745C - , ehR = - 0x41C8EDF39D5E4E76A04D24E6BFD4B2EC35F99CD2483478FD8B0A03E99379576EDACC4167590B7D9C387857A5130B1220CB771F - , ehS = - 0x659652EEAC9747BCAD58034B25362B6AA61836E1BA50E2F37630813050D43457E62EAB0F13AE197E6CFE0244F983107555E269 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x782385F18BAF5A36A588637A76DFAB05739A14163BF723A4417B74BD1469D37AC9E8CCE6AEC8FF63F37B815AAF14A876EED962 - , ehR = - 0x49EC220D6D24980693E6D33B191532EAB4C5D924E97E305E2C1CCFE6F1EAEF96C17F6EC27D1E06191023615368628A7E0BD6A9 - , ehS = - 0x1A4AB1DD9BAAA21F77C503E1B39E770FFD44718349D54BA4CF08F688CE89D7D7C5F7213F225944BE5F7C9BA42B8BEE382F8AF9 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x4DA637CB2E5C90E486744E45A73935DD698D4597E736DA332A06EDA8B26D5ABC6153EC2ECE14981CF3E5E023F36FFA55EEA6D7 - , ehR = - 0x562BB99EE027644EC04E493C5E81B41F261F6BD18FB2FAE3AFEAD91FAB8DD44AFA910B13B9C79C87555225219E44E72245BB7C - , ehS = - 0x25BA5F28047DDDBDA7ED7E49DA31B62B20FD9C7E5B8988817BBF738B3F4DFDD2DCD06EE6DF2A1B744C850DAF952C12B9A56774 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x57055B293ECFDFE983CEF716166091E573275C53906A39EADC25C89C5EC8D7A7E5629FCFDFAD514E1348161C9A34EA1C42D58C - , ehR = - 0x16C7E7FB33B5577F7CF6F77762F0F2D531C6E7A3528BD2CF582498C1A48F200789E9DF7B754029DA0D7E3CE96A2DC760932606 - , ehS = - 0x2729617EFBF80DA5D2F201AC7910D3404A992C39921C2F65F8CF4601392DFE933E6457EAFDBD13DFE160D243100378B55C290A - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x545453D8DC05D220F9A12EF322D0B855E664C72835FABE8A41211453EB8A7CFF950D80773839D0043A46852DDA5A536E02291F - , ehR = - 0x565648A5BAD24E747A7D7531FA9DBDFCB184ECFEFDB00A319459242B68D0989E52BED4107AED35C27D8ECA10E876ACA48006C9 - , ehS = - 0x7420BA6FF72ECC5C92B7CA0309258B5879F26393DB22753B9EC5DF905500A04228AC08880C485E2AC8834E13E8FA44FA57BF18 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x3C5352929D4EBE3CCE87A2DCE380F0D2B33C901E61ABC530DAF3506544AB0930AB9BFD553E51FCDA44F06CD2F49E17E07DB519 - , ehR = - 0x251DFE54EAEC8A781ADF8A623F7F36B4ABFC7EE0AE78C8406E93B5C3932A8120AB8DFC49D8E243C7C30CB5B1E021BADBDF9CA4 - , ehS = - 0x77854C2E72EAA6924CC0B5F6751379D132569843B1C7885978DBBAA6678967F643A50DBB06E6EA6102FFAB7766A57C3887BD22 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x251E32DEE10ED5EA4AD7370DF3EFF091E467D5531CA59DE3AA791763715E1169AB5E18C2A11CD473B0044FB45308E8542F2EB0 - , ehR = - 0x58075FF7E8D36844EED0FC3F78B7CFFDEEF6ADE5982D5636552A081923E24841C9E37DF2C8C4BF2F2F7A174927F3B7E6A0BEB2 - , ehS = - 0x0A737469D013A31B91E781CE201100FDE1FA488ABF2252C025C678462D715AD3078C9D049E06555CABDF37878CFB909553FF51 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x11C540EA46C5038FE28BB66E2E9E9A04C9FE9567ADF33D56745953D44C1DC8B5B92922F53A174E431C0ED8267D919329F19014 - , ehR = - 0x1C5C88642EA216682244E46E24B7CE9AAEF9B3F97E585577D158C3CBC3C598250A53F6D46DFB1E2DD9DC302E7DA4F0CAAFF291 - , ehS = - 0x1D3FD721C35872C74514359F88AD983E170E5DE5B31AFC0BE12E9F4AB2B2538C7797686BA955C1D042FD1F8CDC482775579F11 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x59527CE953BC09DF5E85155CAE7BB1D7F342265F41635545B06044F844ECB4FA6476E7D47420ADC8041E75460EC0A4EC760E95 - , ehR = - 0x1A32CD7764149DF79349DBF79451F4585BB490BD63A200700D7111B45DDA414000AE1B0A69AEACBA1364DD7719968AAD123F93 - , ehS = - 0x582AB1076CAFAE23A76244B82341AEFC4C6D8D8060A62A352C33187720C8A37F3DAC227E62758B11DF1562FD249941C1679F82 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t571k1 - , ecPrivate = - 0x0C16F58550D824ED7B95569D4445375D3A490BC7E0194C41A39DEB732C29396CDF1D66DE02DD1460A816606F3BEC0F32202C7BD18A32D87506466AA92032F1314ED7B19762B0D22 - , ecPublic = - Point - 0x6CFB0DF7541CDD4C41EF319EA88E849EFC8605D97779148082EC991C463ED32319596F9FDF4779C17CAF20EFD9BEB57E9F4ED55BFC52A2FA15CA23BC62B7BF019DB59793DD77318 - 0x1CFC91102F7759A561BD8D5B51AAAEEC7F40E659D67870361990D6DE29F6B4F7E18AE13BDE5EA5C1F77B23D676F44050C9DBFCCDD7B3756328DDA059779AAE8446FC5158A75C227 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x17F7E360B21BEAE4A757A19ACA77FB404D273F05719A86EAD9D7B3F4D5ED7B4630584BB153CF7DCD5A87CCA101BD7EA9ECA0CE5EE27CA985833560000BB52B6BBE068740A45B267 - , ehR = - 0x0767913F96C82E38B7146A505938B79EC07E9AA3214377651BE968B52C039D3E4837B4A2DE26C481C4E1DE96F4D9DE63845D9B32E26D0D332725678E3CE57F668A5E3108FB6CEA5 - , ehS = - 0x109F89F55FA39FF465E40EBCF869A9B1DB425AEA53AB4ECBCE3C310572F79315F5D4891461372A0C36E63871BEDDBB3BA2042C6410B67311F1A185589FF4C987DBA02F9D992B9DF - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x0B599D068A1A00498EE0B9AD6F388521F594BD3F234E47F7A1DB6490D7B57D60B0101B36F39CC22885F78641C69411279706F0989E6991E5D5B53619E43EFB397E25E0814EF02BC - , ehR = - 0x010774B9F14DE6C9525131AD61531FA30987170D43782E9FB84FF0D70F093946DF75ECB69D400FE39B12D58C67C19DCE96335CEC1D9AADE004FE5B498AB8A940D46C8444348686A - , ehS = - 0x06DFE9AA5FEA6CF2CEDC06EE1F9FD9853D411F0B958F1C9C519C90A85F6D24C1C3435B3CDF4E207B4A67467C87B7543F6C0948DD382D24D1E48B3763EC27D4D32A0151C240CC5E0 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x0F79D53E63D89FB87F4D9E6DC5949F5D9388BCFE9EBCB4C2F7CE497814CF40E845705F8F18DBF0F860DE0B1CC4A433EF74A5741F3202E958C082E0B76E16ECD5866AA0F5F3DF300 - , ehR = - 0x1604BE98D1A27CEC2D3FA4BD07B42799E07743071E4905D7DCE7F6992B21A27F14F55D0FE5A7810DF65CF07F2F2554658817E5A88D952282EA1B8310514C0B40FFF46F159965168 - , ehS = - 0x18249377C654B8588475510F7B797081F68C2F8CCCE49F730353B2DA3364B1CD3E984813E11BB791824038EA367BA74583AB97A69AF2D77FA691AA694E348E15DA76F5A44EC1F40 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x0308253C022D25F8A9EBCD24459DD6596590BDEC7895618EEE8A2623A98D2A2B2E7594EE6B7AD3A39D70D68CB4ED01CB28E2129F8E2CC0CC8DC7780657E28BCD655F0BE9B7D35A2 - , ehR = - 0x1E6D7FB237040EA1904CCBF0984B81B866DE10D8AA93B06364C4A46F6C9573FA288C8BDDCC0C6B984E6AA75B42E7BF82FF34D51DFFBD7C87FDBFAD971656185BD12E4B8372F4BF1 - , ehS = - 0x04F94550072ADA7E8C82B7E83577DD39959577799CDABCEA60E267F36F1BEB981ABF24E722A7F031582D2CC5D80DAA7C0DEEBBE1AC5E729A6DBB34A5D645B698719FCA409FBA370 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x0C5EE7070AF55F84EBC43A0D481458CEDE1DCEBB57720A3C92F59B4941A044FECFF4F703940F3121773595E880333772ACF822F2449E17C64DA286BCD65711DD5DA44D7155BF004 - , ehR = - 0x086C9E048EADD7D3D2908501086F3AF449A01AF6BEB2026DC381B39530BCDDBE8E854251CBD5C31E6976553813C11213E4761CB8CA2E5352240AD9FB9C635D55FAB13AE42E4EE4F - , ehS = - 0x09FEE0A68F322B380217FCF6ABFF15D78C432BD8DD82E18B6BA877C01C860E24410F5150A44F979920147826219766ECB4E2E11A151B6A15BB8E2E825AC95BCCA228D8A1C9D3568 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x1D056563469E933E4BE064585D84602D430983BFBFD6885A94BA484DF9A7AB031AD6AC090A433D8EEDC0A7643EA2A9BC3B6299E8ABA933B4C1F2652BB49DAEE833155C8F1319908 - , ehR = - 0x1D055F499A3F7E3FC73D6E7D517B470879BDCB14ABC938369F23643C7B96D0242C1FF326FDAF1CCC8593612ACE982209658E73C24C9EC493B785608669DA74A5B7C9A1D8EA843BC - , ehS = - 0x1621376C53CFE3390A0520D2C657B1FF0EBB10E4B9C2510EDC39D04FEBAF12B8502B098A8B8F842EA6E8EB9D55CFEF94B7FF6D145AC3FFCE71BD978FEA3EF8194D4AB5293A8F3EA - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x1DA875065B9D94DBE75C61848D69578BCC267935792624F9887B53C9AF9E43CABFC42E4C3F9A456BA89E717D24F1412F33CFD297A7A4D403B18B5438654C74D592D5022125E0C6B - , ehR = - 0x18709BDE4E9B73D046CE0D48842C97063DA54DCCA28DCB087168FA37DA2BF5FDBE4720EE48D49EDE4DD5BD31AC0149DB8297BD410F9BC02A11EB79B60C8EE63AF51B65267D71881 - , ehS = - 0x12D8B9E98FBF1D264D78669E236319D8FFD8426C56AFB10C76471EE88D7F0AB1B158E685B6D93C850D47FB1D02E4B24527473DB60B8D1AEF26CEEBD3467B65A70FFDDC0DBB64D5F - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x04DDD0707E81BB56EA2D1D45D7FAFDBDD56912CAE224086802FEA1018DB306C4FB8D93338DBF6841CE6C6AB1506E9A848D2C0463E0889268843DEE4ACB552CFFCB858784ED116B2 - , ehR = - 0x1F5BF6B044048E0E310309FFDAC825290A69634A0D3592DBEE7BE71F69E45412F766AC92E174CC99AABAA5C9C89FCB187DFDBCC7A26765DB6D9F1EEC8A6127BBDFA5801E44E3BEC - , ehS = - 0x1B44CBFB233BFA2A98D5E8B2F0B2C27F9494BEAA77FEB59CDE3E7AE9CB2E385BE8DA7B80D7944AA71E0654E5067E9A70E88E68833054EED49F28283F02B229123995AF37A6089F0 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x0141B53DC6E569D8C0C0718A58A5714204502FDA146E7E2133E56D19E905B79413457437095DE13CF68B5CF5C54A1F2E198A55D974FC3E507AFC0ACF95ED391C93CC79E3B3FE37C - , ehR = - 0x11F61A6EFAB6D83053D9C52665B3542FF3F63BD5913E527BDBA07FBAF34BC766C2EC83163C5273243AA834C75FDDD1BC8A2BEAD388CD06C4EBA1962D645EEB35E92D44E8F2E081D - , ehS = - 0x16BF6341876F051DF224770CC8BA0E4D48B3332568A2B014BC80827BAA89DE18D1AEBC73E3BE8F85A8008C682AAC7D5F0E9FB5ECBEFBB637E30E4A0F226D2C2AA3E569BB54AB72B - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x14842F97F263587A164B215DD0F912C588A88DC4AB6AF4C530ADC1226F16E086D62C14435E6BFAB56F019886C88922D2321914EE41A8F746AAA2B964822E4AC6F40EE2492B66824 - , ehR = - 0x0F1E50353A39EA64CDF23081D6BB4B2A91DD73E99D3DD5A1AA1C49B4F6E34A665EAD24FD530B9103D522609A395AF3EF174C85206F67EF84835ED1632E0F6BAB718EA90DF9E2DA0 - , ehS = - 0x0B385004D7596625028E3FDE72282DE4EDC5B4CE33C1127F21CC37527C90B7307AE7D09281B840AEBCECAA711B00718103DDB32B3E9F6A9FBC6AF23E224A73B9435F619D9C62527 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t163r2 - , ecPrivate = 0x35318FC447D48D7E6BC93B48617DDDEDF26AA658F - , ecPublic = - Point - 0x126CF562D95A1D77D387BA75A3EA3A1407F23425A - 0x7D7CB5273C94DA8CA93049AFDA18721C24672BD71 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x0707A94C3D352E0A9FE49FB12F264992152A20004 - , ehR = 0x153FEBD179A69B6122DEBF5BC61EB947B24C93526 - , ehS = 0x37AC9C670F8CF18045049BAE7DD35553545C19E49 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x3B24C5E2C2D935314EABF57A6484289B291ADFE3F - , ehR = 0x0A379E69C44F9C16EA3215EA39EB1A9B5D58CC955 - , ehS = 0x04BAFF5308DA2A7FE2C1742769265AD3ED1D24E74 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x3D7086A59E6981064A9CDB684653F3A81B6EC0F0B - , ehR = 0x134E00F78FC1CB9501675D91C401DE20DDF228CDC - , ehS = 0x373273AEC6C36CB7BAFBB1903A5F5EA6A1D50B624 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x3B1E4443443486C7251A68EF184A936F05F8B17C7 - , ehR = 0x29430B935AF8E77519B0CA4F6903B0B82E6A21A66 - , ehS = 0x1EA1415306E9353FA5AA54BC7C2581DFBB888440D - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x2EDF5CFCAC7553C17421FDF54AD1D2EF928A879D2 - , ehR = 0x0B2F177A99F9DF2D51CCAF55F015F326E4B65E7A0 - , ehS = 0x0DF1FB4487E9B120C5E970EFE48F55E406306C3A1 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x10024F5B324CBC8954BA6ADB320CD3AB9296983B4 - , ehR = 0x256D4079C6C7169B8BC92529D701776A269D56308 - , ehS = 0x341D3FFEC9F1EB6A6ACBE88E3C86A1C8FDEB8B8E1 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x34F46DE59606D56C75406BFB459537A7CC280AA62 - , ehR = 0x28ECC6F1272CE80EA59DCF32F7AC2D861BA803393 - , ehS = 0x0AD4AE2C06E60183C1567D2B82F19421FE3053CE2 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x38145E3FFCA94E4DDACC20AD6E0997BD0E3B669D2 - , ehR = 0x227DF377B3FA50F90C1CB3CDCBBDBA552C1D35104 - , ehS = 0x1F7BEAD92583FE920D353F368C1960D0E88B46A56 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x375813210ECE9C4D7AB42DDC3C55F89189CF6DFFD - , ehR = 0x11811DAFEEA441845B6118A0DFEE8A0061231337D - , ehS = 0x36258301865EE48C5C6F91D63F62695002AB55B57 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x25AD8B393BC1E9363600FDA1A2AB6DF40079179A3 - , ehR = 0x3B6BB95CA823BE2ED8E3972FF516EB8972D765571 - , ehS = 0x13DC6F420628969DF900C3FCC48220B38BE24A541 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t233r1 - , ecPrivate = 0x07ADC13DD5BF34D1DDEEB50B2CE23B5F5E6D18067306D60C5F6FF11E5D3 - , ecPublic = - Point - 0x0FB348B3246B473AA7FBB2A01B78D61B62C4221D0F9AB55FC72DB3DF478 - 0x1162FA1F6C6ACF7FD8D19FC7D74BDD9104076E833898BC4C042A6E6BEBF - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x0A4E0B67A3A081C1B35D7BECEB5FE72A918B422B907145DB5416ED751CE - , ehR = 0x015CC6FD78BB06E0878E71465515EA5A21A2C18E6FC77B4B158DBEB3944 - , ehS = 0x0822A4A6C2EB2DF213A5E90BF40377956365EE8C4B4A5A4E2EB9270CB6A - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x0F2B1C1E80BEB58283AAA79857F7B83BDF724120D0913606FD07F7FFB2C - , ehR = 0x05D9920B53471148E10502AB49AB7A3F11084820A074FD89883CF51BC1A - , ehS = 0x04D3938900C0A9AAA7080D1DFEB56CFB0FADABE4214536C7ED5117ED13A - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x034A53897B0BBDB484302E19BF3F9B34A2ABFED639D109A388DC52006B5 - , ehR = 0x0A797F3B8AEFCE7456202DF1E46CCC291EA5A49DA3D4BDDA9A4B62D5E0D - , ehS = 0x01F6F81DA55C22DA4152134C661588F4BD6F82FDBAF0C5877096B070DC2 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x04D4670B28990BC92EEB49840B482A1FA03FE028D09F3D21F89C67ECA85 - , ehR = 0x015E85A8D46225DD7E314A1C4289731FC14DECE949349FE535D11043B85 - , ehS = 0x03F189D37F50493EFD5111A129443A662AB3C6B289129AD8C0CAC85119C - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x0DE108AAADA760A14F42C057EF81C0A31AF6B82E8FBCA8DC86E443AB549 - , ehR = 0x03B62A4BF783919098B1E42F496E65F7621F01D1D466C46940F0F132A95 - , ehS = 0x0F4BE031C6E5239E7DAA014CBBF1ED19425E49DAEB426EC9DF4C28A2E30 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x0250C5C90A4E2A3F8849FEBA87F0D0AE630AB18CBABB84F4FFFB36CEAC0 - , ehR = 0x02F1FEDC57BE203E4C8C6B8C1CEB35E13C1FCD956AB41E3BD4C8A6EFB1F - , ehS = 0x05738EC8A8EDEA8E435EE7266AD3EDE1EEFC2CEBE2BE1D614008D5D2951 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x07BDB6A7FD080D9EC2FC84BFF9E3E15750789DC04290C84FED00E109BBD - , ehR = 0x0CCE175124D3586BA7486F7146894C65C2A4A5A1904658E5C7F9DF5FA5D - , ehS = 0x08804B456D847ACE5CA86D97BF79FD6335E5B17F6C0D964B5D0036C867E - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x00376886E89013F7FF4B5214D56A30D49C99F53F211A3AFE01AA2BDE12D - , ehR = 0x035C3D6DFEEA1CFB29B93BE3FDB91A7B130951770C2690C16833A159677 - , ehS = 0x0600F7301D12AB376B56D4459774159ADB51F97E282FF384406AFD53A02 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x03726870DE75613C5E529E453F4D92631C03D08A7F63813E497D4CB3877 - , ehR = 0x061602FC8068BFD5FB86027B97455D200EC603057446CCE4D76DB8EF42C - , ehS = 0x03396DD0D59C067BB999B422D9883736CF9311DFD6951F91033BD03CA8D - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x09CE5810F1AC68810B0DFFBB6BEEF2E0053BB937969AE7886F9D064A8C4 - , ehR = 0x07E12CB60FDD614958E8E34B3C12DDFF35D85A9C5800E31EA2CC2EF63B1 - , ehS = 0x0E8970FD99D836F3CC1C807A2C58760DE6EDAA23705A82B9CB1CE93FECC - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t283r1 - , ecPrivate = - 0x14510D4BC44F2D26F4553942C98073C1BD35545CEABB5CC138853C5158D2729EA408836 - , ecPublic = - Point - 0x17E3409A13C399F0CA8A192F028D46E3446BCFFCDF51FF8A905ED2DED786E74F9C3E8A9 - 0x47EFCBCC31C01D86D1992F7BFAC0277DBD02A6D289274099A2C0F039C8F59F318371B0E - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x277F389559667E8AE4B65DC056F8CE2872E1917E7CC59D17D485B0B98343206FBCCD441 - , ehR = 0x201E18D48C6DB3D5D097C4DCE1E25587E1501FC3CF47BDB5B4289D79E273D6A9ACB8285 - , ehS = 0x151AE05712B024CE617358260774C8CA8B0E7A7E72EF8229BF2ACE7609560CB30322C4F - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x14CC8FCFEECD6B999B4DC6084EBB06FDED0B44D5C507802CC7A5E9ECF36E69DA6AE23C6 - , ehR = 0x143E878DDFD4DF40D97B8CD638B3C4706501C2201CF7108F2FB91478C11D69473246925 - , ehS = 0x0CBF1B9717FEEA3AABB09D9654110144267098E0E1E8D0289A6211BE0EEDFDD86A3DB79 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x38C9D662188982943E080B794A4CFB0732DBA37C6F40D5B8CFADED6FF31C5452BA3F877 - , ehR = 0x29FD82497FB3E5CEF65579272138DE59E2B666B8689466572B3B69A172CEE83BE145659 - , ehS = 0x05A89D9166B40795AF0FE5958201B9C0523E500013CA12B4840EA2BC53F25F9B3CE87C0 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x21B7265DEBF90E6F988CFFDB62B121A02105226C652807CC324ED6FB119A287A72680AB - , ehR = 0x2F00689C1BFCD2A8C7A41E0DE55AE182E6463A152828EF89FE3525139B6603294E69353 - , ehS = 0x1744514FE0A37447250C8A329EAAADA81572226CABA16F39270EE5DD03F27B1F665EB5D - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x20583259DC179D9DA8E5387E89BFF2A3090788CF1496BCABFE7D45BB120B0C811EB8980 - , ehR = 0x0DA43A9ADFAA6AD767998A054C6A8F1CF77A562924628D73C62761847AD8286E0D91B47 - , ehS = 0x1D118733AE2C88357827CAFC6F68ABC25C80C640532925E95CFE66D40F8792F3AC44C42 - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = 0x0185C57A743D5BA06193CE2AA47B07EF3D6067E5AE1A6469BCD3FC510128BA564409D82 - , ehR = 0x05A408133919F2CDCDBE5E4C14FBC706C1F71BADAFEF41F5DE4EC27272FC1CA9366FBB2 - , ehS = 0x012966272872C097FEA7BCE64FAB1A81982A773E26F6E4EF7C99969846E67CA9CBE1692 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = 0x2E5C1F00677A0E015EC3F799FA9E9A004309DBD784640EAAF5E1CE64D3045B9FE9C1FA1 - , ehR = 0x08F3824E40C16FF1DDA8DC992776D26F4A5981AB5092956C4FDBB4F1AE0A711EEAA10E5 - , ehS = 0x0A64B91EFADB213E11483FB61C73E3EF63D3B44EEFC56EA401B99DCC60CC28E99F0F1FA - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = 0x018A7D44F2B4341FEFE68F6BD8894960F97E08124AAB92C1FFBBE90450FCC9356C9AAA5 - , ehR = 0x3597B406F5329D11A79E887847E5EC60861CCBB19EC61F252DB7BD549C699951C182796 - , ehS = 0x0A6A100B997BC622D91701D9F5C6F6D3815517E577622DA69D3A0E8917C1CBE63ACD345 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = 0x3C75397BA4CF1B931877076AF29F2E2F4231B117AB4B8E039F7F9704DE1BD3522F150B6 - , ehR = 0x1BB490926E5A1FDC7C5AA86D0835F9B994EDA315CA408002AF54A298728D422EBF59E4C - , ehS = 0x36C682CFC9E2C89A782BFD3A191609D1F0C1910D5FD6981442070393159D65FBCC0A8BA - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = 0x14E66B18441FA54C21E3492D0611D2B48E19DE3108D915FD5CA08E786327A2675F11074 - , ehR = 0x19944AA68F9778C2E3D6E240947613E6DA60EFCE9B9B2C063FF5466D72745B5A0B25BA2 - , ehS = 0x03F1567B3C5B02DF15C874F0EE22850824693D5ADC4663BAA19E384E550B1DD41F31EE6 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t409r1 - , ecPrivate = - 0x0494994CC325B08E7B4CE038BD9436F90B5E59A2C13C3140CD3AE07C04A01FC489F572CE0569A6DB7B8060393DE76330C624177 - , ecPublic = - Point - 0x1A7055961CF1DA4B9A015B18B1524EF01FDD9B93FAEFC26FB1F2F828A7227B7031925DA0AC1A8A075C3B33554B222EA859C17E7 - 0x18105C042F290736088F30AEC7AE7732A45DE47BCE0940113AB8132516D1E059B0F581FD581A9A3CB3A0AC42A1962738ADB86E6 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x042D8A2B34402757EB2CCFDDC3E6E96A7ADD3FDA547FC10A0CB77CFC720B4F9E16EEAAA2A8CC4E4A4B5DBF7D8AC4EA491859E60 - , ehR = - 0x0D8783188E1A540E2022D389E1D35B32F56F8C2BB5636B8ABF7718806B27A713EBAE37F63ECD4B61445CEF5801B62594EF3E982 - , ehS = - 0x03A6B4A80E204DB0DE12E7415C13C9EC091C52935658316B4A0C591216A3879154BEB1712560E346E7EF26517707435B55C3141 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x0C933F1DC4C70838C2AD16564715ACAF545BCDD8DC203D25AF3EC63949C65CB2E68AC1F60CA7EACA2A823F4E240927AA82CEEC5 - , ehR = - 0x0EE4F39ACC2E03CE96C3D9FCBAFA5C22C89053662F8D4117752A9B10F09ADFDA59DB061E247FE5321D6B170EE758ACE1BE4D157 - , ehS = - 0x00A2B83265B456A430A8BF27DCC8A9488B3F126C10F0D6D64BF7B8A218FAAF20E51A295A3AE78F205E5A4A6AE224C3639F1BB34 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x08EC42D13A3909A20C41BEBD2DFED8CACCE56C7A7D1251DF43F3E9E289DAE00E239F6960924AC451E125B784CB687C7F23283FD - , ehR = - 0x02D8B1B31E33E74D7EB46C30FDE5AD2CA04EC8FE08FBA0E73BA5E568953AC5EA307C072942238DFC07F4A4D7C7C6A9F86436D17 - , ehS = - 0x079F7D471E6CB73234AF7F7C381D2CE15DE35BAF8BB68393B73235B3A26EC2DF4842CE433FB492D6E074E604D4870024D42189A - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x0DA881BCE3BA851485879EF8AC585A63F1540B9198ECB8A1096D70CB25A104E2F8A96B108AE76CB49CF34491ABC70E9D2AAD450 - , ehR = - 0x07BC638B7E7CE6FEE5E9C64A0F966D722D01BB4BC3F3A35F30D4CDDA92DFC5F7F0B4BBFE8065D9AD452FD77A1914BE3A2440C18 - , ehS = - 0x06D904429850521B28A32CBF55C7C0FDF35DC4E0BDA2552C7BF68A171E970E6788ACC0B9521EACB4796E057C70DD9B95FED5BFB - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x0750926FFAD7FF5DE85DF7960B3A4F9E3D38CF5A049BFC89739C48D42B34FBEE03D2C047025134CC3145B60AFD22A68DF0A7FB2 - , ehR = - 0x05D178DECAFD2D02A3DA0D8BA1C4C1D95EE083C760DF782193A9F7B4A8BE6FC5C21FD60613BCA65C063A61226E050A680B3ABD4 - , ehS = - 0x013B7581E98F6A63FBBCB3E49BCDA60F816DB230B888506D105DC229600497C3B46588C784BE3AA9343BEF82F7C9C80AEB63C3B - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x017E167EAB1850A3B38EE66BFE2270F2F6BFDAC5E2D227D47B20E75F0719161E6C74E9F23088F0C58B1E63BC6F185AD2EF4EAE6 - , ehR = - 0x049F54E7C10D2732B4638473053782C6919218BBEFCEC8B51640FC193E832291F05FA12371E9B448417B3290193F08EE9319195 - , ehS = - 0x0499E267DEC84E02F6F108B10E82172C414F15B1B7364BE8BFD66ADC0C5DE23FEE3DF0D811134C25AFE0E05A6672F98889F28F1 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x01ADEB94C19951B460A146B8275D81638C07735B38A525D76023AAF26AA8A058590E1D5B1E78AB3C91608BDA67CFFBE6FC8A6CC - , ehR = - 0x0B1527FFAA7DD7C7E46B628587A5BEC0539A2D04D3CF27C54841C2544E1BBDB42FDBDAAF8671A4CA86DFD619B1E3732D7BB56F2 - , ehS = - 0x0442C68C044868DF4832C807F1EDDEBF7F5052A64B826FD03451440794063F52B022DF304F47403D4069234CA9EB4C964B37C02 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x06EBA3D58D0E0DFC406D67FC72EF0C943624CF40019D1E48C3B54CCAB0594AFD5DEE30AEBAA22E693DBCFECAD1A85D774313DAD - , ehR = - 0x0BB27755B991D6D31757BCBF68CB01225A38E1CFA20F775E861055DD108ED7EA455E4B96B2F6F7CD6C6EC2B3C70C3EDDEB9743B - , ehS = - 0x0C5BE90980E7F444B5F7A12C9E9AC7A04CA81412822DD5AD1BE7C45D5032555EA070864245CF69266871FEB8CD1B7EDC30EF6D5 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x0A45B787DB44C06DEAB846511EEDBF7BFCFD3BD2C11D965C92FC195F67328F36A2DC83C0352885DAB96B55B02FCF49DCCB0E2DA - , ehR = - 0x04EFEB7098772187907C87B33E0FBBA4584226C50C11E98CA7AAC6986F8D3BE044E5B52D201A410B852536527724CA5F8CE6549 - , ehS = - 0x09574102FEB3EF87E6D66B94119F5A6062950FF4F902EA1E6BD9E2037F33FF991E31F5956C23AFE48FCDC557FD6F088C7C9B2B3 - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x0B90F8A0E757E81D4EA6891766729C96A6D01F9AEDC0D334932D1F81CC4E1973A4F01C33555FF08530A5098CADB6EDAE268ABB5 - , ehR = - 0x07E0249C68536AE2AEC2EC30090340DA49E6DC9E9EEC8F85E5AABFB234B6DA7D2E9524028CF821F21C6019770474CC40B01FAF6 - , ehS = - 0x08125B5A03FB44AE81EA46D446130C2A415ECCA265910CA69D55F2453E16CD7B2DFA4E28C50FA8137F9C0C6CEE4CD37ABCCF6D8 - } - ] - } - ] - } - , EntryCurve - { ecName = SEC_t571r1 - , ecPrivate = - 0x028A04857F24C1C082DF0D909C0E72F453F2E2340CCB071F0E389BCA2575DA19124198C57174929AD26E348CF63F78D28021EF5A9BF2D5CBEAF6B7CCB6C4DA824DD5C82CFB24E11 - , ecPublic = - Point - 0x4B4B3CE9377550140B62C1061763AA524814DDCEF37B00CD5CDE94F7792BB0E96758E55DA2E9FEA8FF2A8B6830AE1D57A9CA7A77FCB0836BF43EA5454CDD9FEAD5CCFE7375C6A83 - 0x4453B18F261E7A0E7570CD72F235EA750438E43946FBEBD2518B696954767AA7849C1719E18E1C51652C28CA853426F15C09AA4B579487338ABC7F33768FADD61B5A3A6443A8189 - , ecMessages = - [ EntryMessage - { emMessage = "sample" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x2669FAFEF848AF67D437D4A151C3C5D3F9AA8BB66EDC35F090C9118F95BA0041B0993BE2EF55DAAF36B5B3A737C40DB1F6E3D93D97B8419AD6E1BB8A5D4A0E9B2E76832D4E7B862 - , ehR = - 0x147D3EB0EDA9F2152DFD014363D6A9CE816D7A1467D326A625FC4AB0C786E1B74DDF7CD4D0E99541391B266C704BB6B6E8DCCD27B460802E0867143727AA415555454321EFE5CB6 - , ehS = - 0x17319571CAF533D90D2E78A64060B9C53169AB7FC908947B3EDADC54C79CCF0A7920B4C64A4EAB6282AFE9A459677CDA37FD6DD50BEF18709590FE18B923BDF74A66B189A850819 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x2EAFAD4AC8644DEB29095BBAA88D19F31316434F1766AD4423E0B54DD2FE0C05E307758581B0DAED2902683BBC7C47B00E63E3E429BA54EA6BA3AEC33A94C9A24A6EF8E27B7677A - , ehR = - 0x10F4B63E79B2E54E4F4F6A2DBC786D8F4A143ECA7B2AD97810F6472AC6AE20853222854553BE1D44A7974599DB7061AE8560DF57F2675BE5F9DD94ABAF3D47F1582B318E459748B - , ehS = - 0x3BBEA07C6B269C2B7FE9AE4DDB118338D0C2F0022920A7F9DCFCB7489594C03B536A9900C4EA6A10410007222D3DAE1A96F291C4C9275D75D98EB290DC0EEF176037B2C7A7A39A3 - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x15C2C6B7D1A070274484774E558B69FDFA193BDB7A23F27C2CD24298CE1B22A6CC9B7FB8CABFD6CF7C6B1CF3251E5A1CDDD16FBFED28DE79935BB2C631B8B8EA9CC4BCC937E669E - , ehR = - 0x213EF9F3B0CFC4BF996B8AF3A7E1F6CACD2B87C8C63820000800AC787F17EC99C04BCEDF29A8413CFF83142BB88A50EF8D9A086AF4EB03E97C567500C21D865714D832E03C6D054 - , ehS = - 0x3D32322559B094E20D8935E250B6EC139AC4AAB77920812C119AF419FB62B332C8D226C6C9362AE3C1E4AABE19359B8428EA74EC8FBE83C8618C2BCCB6B43FBAA0F2CCB7D303945 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x0FEF0B68CB49453A4C6ECBF1708DBEEFC885C57FDAFB88417AAEFA5B1C35017B4B498507937ADCE2F1D9EFFA5FE8F5AEB116B804FD182A6CF1518FDB62D53F60A0FF6EB707D856B - , ehR = - 0x375D8F49C656A0BBD21D3F54CDA287D853C4BB1849983CD891EF6CD6BB56A62B687807C16685C2C9BCA2663C33696ACCE344C45F3910B1DF806204FF731ECB289C100EF4D1805EC - , ehS = - 0x1CDEC6F46DFEEE44BCE71D41C60550DC67CF98D6C91363625AC2553E4368D2DFB734A8E8C72E118A76ACDB0E58697940A0F3DF49E72894BD799450FC9E550CC04B9FF9B0380021C - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x3FF373833A06C791D7AD586AFA3990F6EF76999C35246C4AD0D519BFF180CA1880E11F2FB38B764854A0AE3BECDDB50F05AC4FCEE542F207C0A6229E2E19652F0E647B9C4882193 - , ehR = - 0x1C26F40D940A7EAA0EB1E62991028057D91FEDA0366B606F6C434C361F04E545A6A51A435E26416F6838FFA260C617E798E946B57215284182BE55F29A355E6024FE32A47289CF0 - , ehS = - 0x3691DE4369D921FE94EDDA67CB71FBBEC9A436787478063EB1CC778B3DCDC1C4162662752D28DEEDF6F32A269C82D1DB80C87CE4D3B662E03AC347806E3F19D18D6D4DE7358DF7E - } - ] - } - , EntryMessage - { emMessage = "test" - , emHashes = - [ EntryHash - { ehAlgorithm = HashAlg SHA1 - , ehK = - 0x019B506FD472675A7140E429AA5510DCDDC21004206EEC1B39B28A688A8FD324138F12503A4EFB64F934840DFBA2B4797CFC18B8BD0B31BBFF3CA66A4339E4EF9D771B15279D1DC - , ehR = - 0x133F5414F2A9BC41466D339B79376038A64D045E5B0F792A98E5A7AA87E0AD016419E5F8D176007D5C9C10B5FD9E2E0AB8331B195797C0358BA05ECBF24ACE59C5F368A6C0997CC - , ehS = - 0x3D16743AE9F00F0B1A500F738719C5582550FEB64689DA241665C4CE4F328BA0E34A7EF527ED13BFA5889FD2D1D214C11EB17D6BC338E05A56F41CAFF1AF7B8D574DB62EF0D0F21 - } - , EntryHash - { ehAlgorithm = HashAlg SHA224 - , ehK = - 0x333C711F8C62F205F926593220233B06228285261D34026232F6F729620C6DE12220F282F4206D223226705608688B20B8BA86D8DFE54F07A37EC48F253283AC33C3F5102C8CC3E - , ehR = - 0x3048E76506C5C43D92B2E33F62B33E3111CEEB87F6C7DF7C7C01E3CDA28FA5E8BE04B5B23AA03C0C70FEF8F723CBCEBFF0B7A52A3F5C8B84B741B4F6157E69A5FB0524B48F31828 - , ehS = - 0x2C99078CCFE5C82102B8D006E3703E020C46C87C75163A2CD839C885550BA5CB501AC282D29A1C26D26773B60FBE05AAB62BFA0BA32127563D42F7669C97784C8897C22CFB4B8FA - } - , EntryHash - { ehAlgorithm = HashAlg SHA256 - , ehK = - 0x328E02CF07C7B5B6D3749D8302F1AE5BFAA8F239398459AF4A2C859C7727A8123A7FE9BE8B228413FC8DC0E9DE16AF3F8F43005107F9989A5D97A5C4455DA895E81336710A3FB2C - , ehR = - 0x184BC808506E11A65D628B457FDA60952803C604CC7181B59BD25AEE1411A66D12A777F3A0DC99E1190C58D0037807A95E5080FA1B2E5CCAA37B50D401CFFC3417C005AEE963469 - , ehS = - 0x27280D45F81B19334DBDB07B7E63FE8F39AC7E9AE14DE1D2A6884D2101850289D70EE400F26ACA5E7D73F534A14568478E59D00594981ABE6A1BA18554C13EB5E03921E4DC98333 - } - , EntryHash - { ehAlgorithm = HashAlg SHA384 - , ehK = - 0x2A77E29EAD9E811A9FDA0284C14CDFA1D9F8FA712DA59D530A06CDE54187E250AD1D4FB5788161938B8DE049616399C5A56B0737C9564C9D4D845A4C6A7CDFCBFF0F01A82BE672E - , ehR = - 0x319EE57912E7B0FAA1FBB145B0505849A89C6DB1EC06EA20A6A7EDE072A6268AF6FD9C809C7E422A5F33C6C3326EAD7402467DF3272A1B2726C1C20975950F0F50D8324578F13EC - , ehS = - 0x2CF3EA27EADD0612DD2F96F46E89AB894B01A10DF985C5FC099CFFE0EA083EB44BE682B08BFE405DAD5F37D0A2C59015BA41027E24B99F8F75A70B6B7385BF39BBEA02513EB880C - } - , EntryHash - { ehAlgorithm = HashAlg SHA512 - , ehK = - 0x21CE6EE4A2C72C9F93BDB3B552F4A633B8C20C200F894F008643240184BE57BB282A1645E47FBBE131E899B4C61244EFC2486D88CDBD1DD4A65EBDD837019D02628D0DCD6ED8FB5 - , ehR = - 0x2AA1888EAB05F7B00B6A784C4F7081D2C833D50794D9FEAF6E22B8BE728A2A90BFCABDC803162020AA629718295A1489EE7ED0ECB8AAA197B9BDFC49D18DDD78FC85A48F9715544 - , ehS = - 0x0AA5371FE5CA671D6ED9665849C37F394FED85D51FEF72DA2B5F28EDFB2C6479CA63320C19596F5E1101988E2C619E302DD05112F47E8823040CE540CD3E90DCF41DBC461744EE9 - } - ] - } - ] - } - ] - -testPublic :: PrivateKey -> PublicPoint -> TestTree -testPublic (PrivateKey curve key) pub = - testCase "public" $ - pub @=? generateQ curve key - -testNonce :: PrivateKey -> HashAlg -> ByteString -> Integer -> TestTree -testNonce key (HashAlg alg) msg nonc = - testCase "nonce" $ - nonc @=? deterministicNonce alg key (hashWith alg msg) Just - -testSignature - :: PrivateKey -> HashAlg -> ByteString -> Integer -> Signature -> TestTree -testSignature key (HashAlg alg) msg nonc sig = testCase "signature" $ - case signWith nonc key alg msg of - Nothing -> assertFailure "could not sign message" - Just result -> sig @=? result - -testVerify :: PublicKey -> HashAlg -> ByteString -> Signature -> TestTree -testVerify pub (HashAlg alg) msg sig = - testCase "verify" $ - assertBool "signature verification failed" $ - verify alg pub sig msg - -testEntry :: Entry -> TestTree -testEntry entry = testGroup (show entry) tests - where - tests = - [ testPublic key $ publicPoint entry - , testSignature - key - (hashAlgorithm entry) - (message entry) - (nonce entry) - (signature entry) - , testVerify pub (hashAlgorithm entry) (message entry) (signature entry) - ] - pub = PublicKey curve $ publicPoint entry - key = PrivateKey curve $ privateNumber entry - curve = getCurveByName $ curveName entry - -testEntryNonce :: Entry -> TestTree -testEntryNonce entry = testGroup (show entry) tests - where - tests = - [ testPublic key $ publicPoint entry - , testNonce key (hashAlgorithm entry) (message entry) (nonce entry) - , testSignature - key - (hashAlgorithm entry) - (message entry) - (nonce entry) - (signature entry) - , testVerify pub (hashAlgorithm entry) (message entry) (signature entry) - ] - pub = PublicKey curve $ publicPoint entry - key = PrivateKey curve $ privateNumber entry - curve = getCurveByName $ curveName entry - -ecdsaTests :: TestTree -ecdsaTests = - testGroup - "ECDSA" - [ testGroup "GEC 2" $ testEntry . normalize <$> gec2Entries - , testGroup "RFC 6979" $ testEntryNonce . normalize <$> flatten rfc6979Entries - ] diff --git a/tests/KAT_PubKey/OAEP.hs b/tests/KAT_PubKey/OAEP.hs deleted file mode 100644 --- a/tests/KAT_PubKey/OAEP.hs +++ /dev/null @@ -1,142 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.OAEP (oaepTests) where - -import Crypto.Hash -import Crypto.PubKey.RSA -import qualified Crypto.PubKey.RSA.OAEP as OAEP - -import Imports - -rsaKeyInt = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0xbbf82f090682ce9c2338ac2b9da871f7368d07eed41043a440d6b6f07454f51fb8dfbaaf035c02ab61ea48ceeb6fcd4876ed520d60e1ec4619719d8a5b8b807fafb8e0a3dfc737723ee6b4b7d93a2584ee6a649d060953748834b2454598394ee0aab12d7b61a51f527a9a41f6c1687fe2537298ca2a8f5946f8e5fd091dbdcb - , public_e = 0x11 - , public_size = 128 - } - , private_d = - 0xa5dafc5341faf289c4b988db30c1cdf83f31251e0668b42784813801579641b29410b3c7998d6bc465745e5c392669d6870da2c082a939e37fdcb82ec93edac97ff3ad5950accfbc111c76f1a9529444e56aaf68c56c092cd38dc3bef5d20a939926ed4f74a13eddfbe1a1cecc4894af9428c2b7b8883fe4463a4bc85b1cb3c1 - , private_p = - 0xeecfae81b1b9b3c908810b10a1b5600199eb9f44aef4fda493b81a9e3d84f632124ef0236e5d1e3b7e28fae7aa040a2d5b252176459d1f397541ba2a58fb6599 - , private_q = - 0xc97fb1f027f453f6341233eaaad1d9353f6c42d08866b1d05a0f2035028b9d869840b41666b42e92ea0da3b43204b5cfce3352524d0416a5a441e700af461503 - , private_dP = - 0x54494ca63eba0337e4e24023fcd69a5aeb07dddc0183a4d0ac9b54b051f2b13ed9490975eab77414ff59c1f7692e9a2e202b38fc910a474174adc93c1f67c981 - , private_dQ = - 0x471e0290ff0af0750351b7f878864ca961adbd3a8a7e991c5c0556a94c3146a7f9803f8f6f8ae342e931fd8ae47a220d1b99a495849807fe39f9245a9836da3d - , private_qinv = - 0xb06c4fdabb6301198d265bdbae9423b380f271f73453885093077fcd39e2119fc98632154f5883b167a967bf402b4e9e2e0f9656e698ea3666edfb25798039f7 - } - -rsaKey1 = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0xa8b3b284af8eb50b387034a860f146c4919f318763cd6c5598c8ae4811a1e0abc4c7e0b082d693a5e7fced675cf4668512772c0cbc64a742c6c630f533c8cc72f62ae833c40bf25842e984bb78bdbf97c0107d55bdb662f5c4e0fab9845cb5148ef7392dd3aaff93ae1e6b667bb3d4247616d4f5ba10d4cfd226de88d39f16fb - , public_e = 0x010001 - , public_size = 128 - } - , private_d = - 0x53339cfdb79fc8466a655c7316aca85c55fd8f6dd898fdaf119517ef4f52e8fd8e258df93fee180fa0e4ab29693cd83b152a553d4ac4d1812b8b9fa5af0e7f55fe7304df41570926f3311f15c4d65a732c483116ee3d3d2d0af3549ad9bf7cbfb78ad884f84d5beb04724dc7369b31def37d0cf539e9cfcdd3de653729ead5d1 - , private_p = - 0xd32737e7267ffe1341b2d5c0d150a81b586fb3132bed2f8d5262864a9cb9f30af38be448598d413a172efb802c21acf1c11c520c2f26a471dcad212eac7ca39d - , private_q = - 0xcc8853d1d54da630fac004f471f281c7b8982d8224a490edbeb33d3e3d5cc93c4765703d1dd791642f1f116a0dd852be2419b2af72bfe9a030e860b0288b5d77 - , private_dP = - 0x0e12bf1718e9cef5599ba1c3882fe8046a90874eefce8f2ccc20e4f2741fb0a33a3848aec9c9305fbecbd2d76819967d4671acc6431e4037968db37878e695c1 - , private_dQ = - 0x95297b0f95a2fa67d00707d609dfd4fc05c89dafc2ef6d6ea55bec771ea333734d9251e79082ecda866efef13c459e1a631386b7e354c899f5f112ca85d71583 - , private_qinv = - 0x4f456c502493bdc0ed2ab756a3a6ed4d67352a697d4216e93212b127a63d5411ce6fa98d5dbefd73263e3728142743818166ed7dd63687dd2a8ca1d2f4fbd8e1 - } - -data VectorOAEP = VectorOAEP - { seed :: ByteString - , message :: ByteString - , cipherText :: ByteString - } -vectorInt = - VectorOAEP - { message = "\xd4\x36\xe9\x95\x69\xfd\x32\xa7\xc8\xa0\x5b\xbc\x90\xd3\x2c\x49" - , seed = - "\xaa\xfd\x12\xf6\x59\xca\xe6\x34\x89\xb4\x79\xe5\x07\x6d\xde\xc2\xf0\x6c\xb5\x8f" - , cipherText = - "\x12\x53\xe0\x4d\xc0\xa5\x39\x7b\xb4\x4a\x7a\xb8\x7e\x9b\xf2\xa0\x39\xa3\x3d\x1e\x99\x6f\xc8\x2a\x94\xcc\xd3\x00\x74\xc9\x5d\xf7\x63\x72\x20\x17\x06\x9e\x52\x68\xda\x5d\x1c\x0b\x4f\x87\x2c\xf6\x53\xc1\x1d\xf8\x23\x14\xa6\x79\x68\xdf\xea\xe2\x8d\xef\x04\xbb\x6d\x84\xb1\xc3\x1d\x65\x4a\x19\x70\xe5\x78\x3b\xd6\xeb\x96\xa0\x24\xc2\xca\x2f\x4a\x90\xfe\x9f\x2e\xf5\xc9\xc1\x40\xe5\xbb\x48\xda\x95\x36\xad\x87\x00\xc8\x4f\xc9\x13\x0a\xde\xa7\x4e\x55\x8d\x51\xa7\x4d\xdf\x85\xd8\xb5\x0d\xe9\x68\x38\xd6\x06\x3e\x09\x55" - } - -vectorsKey1 = - [ VectorOAEP -- 1.1 - { message = - "\x66\x28\x19\x4e\x12\x07\x3d\xb0\x3b\xa9\x4c\xda\x9e\xf9\x53\x23\x97\xd5\x0d\xba\x79\xb9\x87\x00\x4a\xfe\xfe\x34" - , seed = - "\x18\xb7\x76\xea\x21\x06\x9d\x69\x77\x6a\x33\xe9\x6b\xad\x48\xe1\xdd\xa0\xa5\xef" - , cipherText = - "\x35\x4f\xe6\x7b\x4a\x12\x6d\x5d\x35\xfe\x36\xc7\x77\x79\x1a\x3f\x7b\xa1\x3d\xef\x48\x4e\x2d\x39\x08\xaf\xf7\x22\xfa\xd4\x68\xfb\x21\x69\x6d\xe9\x5d\x0b\xe9\x11\xc2\xd3\x17\x4f\x8a\xfc\xc2\x01\x03\x5f\x7b\x6d\x8e\x69\x40\x2d\xe5\x45\x16\x18\xc2\x1a\x53\x5f\xa9\xd7\xbf\xc5\xb8\xdd\x9f\xc2\x43\xf8\xcf\x92\x7d\xb3\x13\x22\xd6\xe8\x81\xea\xa9\x1a\x99\x61\x70\xe6\x57\xa0\x5a\x26\x64\x26\xd9\x8c\x88\x00\x3f\x84\x77\xc1\x22\x70\x94\xa0\xd9\xfa\x1e\x8c\x40\x24\x30\x9c\xe1\xec\xcc\xb5\x21\x00\x35\xd4\x7a\xc7\x2e\x8a" - } - , VectorOAEP -- 1.2 - { message = - "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" - , seed = - "\x0c\xc7\x42\xce\x4a\x9b\x7f\x32\xf9\x51\xbc\xb2\x51\xef\xd9\x25\xfe\x4f\xe3\x5f" - , cipherText = - "\x64\x0d\xb1\xac\xc5\x8e\x05\x68\xfe\x54\x07\xe5\xf9\xb7\x01\xdf\xf8\xc3\xc9\x1e\x71\x6c\x53\x6f\xc7\xfc\xec\x6c\xb5\xb7\x1c\x11\x65\x98\x8d\x4a\x27\x9e\x15\x77\xd7\x30\xfc\x7a\x29\x93\x2e\x3f\x00\xc8\x15\x15\x23\x6d\x8d\x8e\x31\x01\x7a\x7a\x09\xdf\x43\x52\xd9\x04\xcd\xeb\x79\xaa\x58\x3a\xdc\xc3\x1e\xa6\x98\xa4\xc0\x52\x83\xda\xba\x90\x89\xbe\x54\x91\xf6\x7c\x1a\x4e\xe4\x8d\xc7\x4b\xbb\xe6\x64\x3a\xef\x84\x66\x79\xb4\xcb\x39\x5a\x35\x2d\x5e\xd1\x15\x91\x2d\xf6\x96\xff\xe0\x70\x29\x32\x94\x6d\x71\x49\x2b\x44" - } - , VectorOAEP -- 1.3 - { message = - "\xd9\x4a\xe0\x83\x2e\x64\x45\xce\x42\x33\x1c\xb0\x6d\x53\x1a\x82\xb1\xdb\x4b\xaa\xd3\x0f\x74\x6d\xc9\x16\xdf\x24\xd4\xe3\xc2\x45\x1f\xff\x59\xa6\x42\x3e\xb0\xe1\xd0\x2d\x4f\xe6\x46\xcf\x69\x9d\xfd\x81\x8c\x6e\x97\xb0\x51" - , seed = - "\x25\x14\xdf\x46\x95\x75\x5a\x67\xb2\x88\xea\xf4\x90\x5c\x36\xee\xc6\x6f\xd2\xfd" - , cipherText = - "\x42\x37\x36\xed\x03\x5f\x60\x26\xaf\x27\x6c\x35\xc0\xb3\x74\x1b\x36\x5e\x5f\x76\xca\x09\x1b\x4e\x8c\x29\xe2\xf0\xbe\xfe\xe6\x03\x59\x5a\xa8\x32\x2d\x60\x2d\x2e\x62\x5e\x95\xeb\x81\xb2\xf1\xc9\x72\x4e\x82\x2e\xca\x76\xdb\x86\x18\xcf\x09\xc5\x34\x35\x03\xa4\x36\x08\x35\xb5\x90\x3b\xc6\x37\xe3\x87\x9f\xb0\x5e\x0e\xf3\x26\x85\xd5\xae\xc5\x06\x7c\xd7\xcc\x96\xfe\x4b\x26\x70\xb6\xea\xc3\x06\x6b\x1f\xcf\x56\x86\xb6\x85\x89\xaa\xfb\x7d\x62\x9b\x02\xd8\xf8\x62\x5c\xa3\x83\x36\x24\xd4\x80\x0f\xb0\x81\xb1\xcf\x94\xeb" - } - , VectorOAEP - { message = - "\x52\xe6\x50\xd9\x8e\x7f\x2a\x04\x8b\x4f\x86\x85\x21\x53\xb9\x7e\x01\xdd\x31\x6f\x34\x6a\x19\xf6\x7a\x85" - , seed = - "\xc4\x43\x5a\x3e\x1a\x18\xa6\x8b\x68\x20\x43\x62\x90\xa3\x7c\xef\xb8\x5d\xb3\xfb" - , cipherText = - "\x45\xea\xd4\xca\x55\x1e\x66\x2c\x98\x00\xf1\xac\xa8\x28\x3b\x05\x25\xe6\xab\xae\x30\xbe\x4b\x4a\xba\x76\x2f\xa4\x0f\xd3\xd3\x8e\x22\xab\xef\xc6\x97\x94\xf6\xeb\xbb\xc0\x5d\xdb\xb1\x12\x16\x24\x7d\x2f\x41\x2f\xd0\xfb\xa8\x7c\x6e\x3a\xcd\x88\x88\x13\x64\x6f\xd0\xe4\x8e\x78\x52\x04\xf9\xc3\xf7\x3d\x6d\x82\x39\x56\x27\x22\xdd\xdd\x87\x71\xfe\xc4\x8b\x83\xa3\x1e\xe6\xf5\x92\xc4\xcf\xd4\xbc\x88\x17\x4f\x3b\x13\xa1\x12\xaa\xe3\xb9\xf7\xb8\x0e\x0f\xc6\xf7\x25\x5b\xa8\x80\xdc\x7d\x80\x21\xe2\x2a\xd6\xa8\x5f\x07\x55" - } - , VectorOAEP - { message = - "\x8d\xa8\x9f\xd9\xe5\xf9\x74\xa2\x9f\xef\xfb\x46\x2b\x49\x18\x0f\x6c\xf9\xe8\x02" - , seed = - "\xb3\x18\xc4\x2d\xf3\xbe\x0f\x83\xfe\xa8\x23\xf5\xa7\xb4\x7e\xd5\xe4\x25\xa3\xb5" - , cipherText = - "\x36\xf6\xe3\x4d\x94\xa8\xd3\x4d\xaa\xcb\xa3\x3a\x21\x39\xd0\x0a\xd8\x5a\x93\x45\xa8\x60\x51\xe7\x30\x71\x62\x00\x56\xb9\x20\xe2\x19\x00\x58\x55\xa2\x13\xa0\xf2\x38\x97\xcd\xcd\x73\x1b\x45\x25\x7c\x77\x7f\xe9\x08\x20\x2b\xef\xdd\x0b\x58\x38\x6b\x12\x44\xea\x0c\xf5\x39\xa0\x5d\x5d\x10\x32\x9d\xa4\x4e\x13\x03\x0f\xd7\x60\xdc\xd6\x44\xcf\xef\x20\x94\xd1\x91\x0d\x3f\x43\x3e\x1c\x7c\x6d\xd1\x8b\xc1\xf2\xdf\x7f\x64\x3d\x66\x2f\xb9\xdd\x37\xea\xd9\x05\x91\x90\xf4\xfa\x66\xca\x39\xe8\x69\xc4\xeb\x44\x9c\xbd\xc4\x39" - } - , VectorOAEP -- 1.6 - { message = "\x26\x52\x10\x50\x84\x42\x71" - , seed = - "\xe4\xec\x09\x82\xc2\x33\x6f\x3a\x67\x7f\x6a\x35\x61\x74\xeb\x0c\xe8\x87\xab\xc2" - , cipherText = - "\x42\xce\xe2\x61\x7b\x1e\xce\xa4\xdb\x3f\x48\x29\x38\x6f\xbd\x61\xda\xfb\xf0\x38\xe1\x80\xd8\x37\xc9\x63\x66\xdf\x24\xc0\x97\xb4\xab\x0f\xac\x6b\xdf\x59\x0d\x82\x1c\x9f\x10\x64\x2e\x68\x1a\xd0\x5b\x8d\x78\xb3\x78\xc0\xf4\x6c\xe2\xfa\xd6\x3f\x74\xe0\xad\x3d\xf0\x6b\x07\x5d\x7e\xb5\xf5\x63\x6f\x8d\x40\x3b\x90\x59\xca\x76\x1b\x5c\x62\xbb\x52\xaa\x45\x00\x2e\xa7\x0b\xaa\xce\x08\xde\xd2\x43\xb9\xd8\xcb\xd6\x2a\x68\xad\xe2\x65\x83\x2b\x56\x56\x4e\x43\xa6\xfa\x42\xed\x19\x9a\x09\x97\x69\x74\x2d\xf1\x53\x9e\x82\x55" - } - ] - -doEncryptionTest key i vec = testCase (show i) (Right (cipherText vec) @=? actual) - where - actual = - OAEP.encryptWithSeed (seed vec) (OAEP.defaultOAEPParams SHA1) key (message vec) - -doDecryptionTest key i vec = testCase (show i) (Right (message vec) @=? actual) - where - actual = OAEP.decrypt Nothing (OAEP.defaultOAEPParams SHA1) key (cipherText vec) - -oaepTests = - testGroup - "RSA-OAEP" - [ testGroup - "internal" - [ doEncryptionTest (private_pub rsaKeyInt) (0 :: Int) vectorInt - , doDecryptionTest rsaKeyInt (0 :: Int) vectorInt - ] - , testGroup "encryption key 1024 bits" $ - zipWith (doEncryptionTest $ private_pub rsaKey1) [katZero ..] vectorsKey1 - , testGroup "decryption key 1024 bits" $ - zipWith (doDecryptionTest rsaKey1) [katZero ..] vectorsKey1 - ] diff --git a/tests/KAT_PubKey/P256.hs b/tests/KAT_PubKey/P256.hs deleted file mode 100644 --- a/tests/KAT_PubKey/P256.hs +++ /dev/null @@ -1,265 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE ScopedTypeVariables #-} - -module KAT_PubKey.P256 (tests) where - -import qualified Crypto.PubKey.ECC.P256 as P256 -import qualified Crypto.PubKey.ECC.Prim as ECC -import qualified Crypto.PubKey.ECC.Types as ECC - -import Crypto.Error -import Crypto.Number.ModArithmetic (inverseCoprimes) -import Crypto.Number.Serialize (i2ospOf, os2ip) -import Data.ByteArray (Bytes) - -import Imports - -newtype P256Scalar = P256Scalar Integer - deriving (Show, Eq, Ord) - -instance Arbitrary P256Scalar where - -- Cover the full range up to 2^256-1 except 0 and curveN. To test edge - -- cases with arithmetic functions, some values close to 0, curveN and - -- 2^256 are given higher frequency. - arbitrary = - P256Scalar - <$> oneof - [ choose (1, w) - , choose (w + 1, curveN - w - 1) - , choose (curveN - w, curveN - 1) - , choose (curveN + 1, curveN + w) - , choose (curveN + w + 1, high - w - 1) - , choose (high - w, high - 1) - ] - where - high = 2 ^ (256 :: Int) - w = 100 - -curve = ECC.getCurveByName ECC.SEC_p256r1 -curveN = ECC.ecc_n . ECC.common_curve $ curve -curveGen = ECC.ecc_g . ECC.common_curve $ curve - -pointP256ToECC :: P256.Point -> ECC.Point -pointP256ToECC p - | P256.pointIsAtInfinity p = ECC.PointO - | otherwise = uncurry ECC.Point (P256.pointToIntegers p) - -i2ospScalar :: Integer -> Bytes -i2ospScalar i = - case i2ospOf 32 i of - Nothing -> error "invalid size of P256 scalar" - Just b -> b - -unP256Scalar :: P256Scalar -> P256.Scalar -unP256Scalar (P256Scalar r) = - let rBytes = i2ospScalar r - in case P256.scalarFromBinary rBytes of - CryptoFailed err -> error ("cannot convert scalar: " ++ show err) - CryptoPassed scalar -> scalar - -unP256 :: P256Scalar -> Integer -unP256 (P256Scalar r) = r - -modP256Scalar :: P256Scalar -> P256Scalar -modP256Scalar (P256Scalar r) = P256Scalar (r `mod` curveN) - -p256ScalarToInteger :: P256.Scalar -> Integer -p256ScalarToInteger s = os2ip (P256.scalarToBinary s :: Bytes) - -xS = 0xde2444bebc8d36e682edd27e0f271508617519b3221a8fa0b77cab3989da97c9 -yS = 0xc093ae7ff36e5380fc01a5aad1e66659702de80f53cec576b6350b243042a256 -xT = 0x55a8b00f8da1d44e62f6b3b25316212e39540dc861c89575bb8cf92e35e0986b -yT = 0x5421c3209c2d6c704835d82ac4c3dd90f61a8a52598b9e7ab656e9d8c8b24316 -xR = 0x72b13dd4354b6b81745195e98cc5ba6970349191ac476bd4553cf35a545a067e -yR = 0x8d585cbb2e1327d75241a8a122d7620dc33b13315aa5c9d46d013011744ac264 - --- Two points on the curve whose validation reduces a product whose top --- digit has a zero low half: x = 2^96, and an x with a repeating bit --- pattern. Wycheproof ecdh_secp256r1_ecpoint tcId 74 and 93. -xU = 0x0000000000000000000000000000000000000001000000000000000000000000 -yU = 0x7d12de58d54423eb85ae8d157ae416fb004a7eb522ac1b67047ef3cdf9acdc3f -xV = 0x8000003ffffff0000007fffffe000000ffffffc000001ffffff8000003fffffc -yV = 0x0c3527bd081c1c07b313bc1a0c3f845fb2fe22557699ccc8f1354e61a27b7f88 - -tests = - testGroup - "P256" - [ testGroup - "scalar" - [ testProperty "marshalling" $ \(QAInteger r) -> - let rBytes = i2ospScalar r - in case P256.scalarFromBinary rBytes of - CryptoFailed err -> error (show err) - CryptoPassed scalar -> rBytes `propertyEq` P256.scalarToBinary scalar - , testProperty "add" $ \r1 r2 -> - let r = (unP256 r1 + unP256 r2) `mod` curveN - r' = P256.scalarAdd (unP256Scalar r1) (unP256Scalar r2) - in r `propertyEq` p256ScalarToInteger r' - , testProperty "add0" $ \r -> - let v = unP256 r `mod` curveN - v' = P256.scalarAdd (unP256Scalar r) P256.scalarZero - in v `propertyEq` p256ScalarToInteger v' - , testProperty "sub" $ \r1 r2 -> - let r = (unP256 r1 - unP256 r2) `mod` curveN - r' = P256.scalarSub (unP256Scalar r1) (unP256Scalar r2) - v = (unP256 r2 - unP256 r1) `mod` curveN - v' = P256.scalarSub (unP256Scalar r2) (unP256Scalar r1) - in propertyHold - [ eqTest "r1-r2" r (p256ScalarToInteger r') - , eqTest "r2-r1" v (p256ScalarToInteger v') - ] - , testProperty "sub0" $ \r -> - let v = unP256 r `mod` curveN - v' = P256.scalarSub (unP256Scalar r) P256.scalarZero - in v `propertyEq` p256ScalarToInteger v' - , testProperty "mul" $ \r1 r2 -> - let r = (unP256 r1 * unP256 r2) `mod` curveN - r' = P256.scalarMul (unP256Scalar r1) (unP256Scalar r2) - in r `propertyEq` p256ScalarToInteger r' - , testProperty "inv" $ \r' -> - let inv = inverseCoprimes (unP256 r') curveN - inv' = P256.scalarInv (unP256Scalar r') - in unP256 r' /= 0 ==> inv `propertyEq` p256ScalarToInteger inv' - , testProperty "inv-safe" $ \r' -> - let inv = P256.scalarInv (unP256Scalar r') - inv' = P256.scalarInvSafe (unP256Scalar r') - in unP256 r' /= 0 ==> inv `propertyEq` inv' - , testProperty "inv-safe-mul" $ \r' -> - let inv = P256.scalarInvSafe (unP256Scalar r') - res = P256.scalarMul (unP256Scalar r') inv - in unP256 r' /= 0 ==> 1 `propertyEq` p256ScalarToInteger res - , testProperty "inv-safe-zero" $ - let inv0 = P256.scalarInvSafe P256.scalarZero - invN = P256.scalarInvSafe P256.scalarN - in propertyHold - [ eqTest "scalarZero" P256.scalarZero inv0 - , eqTest "scalarN" P256.scalarZero invN - ] - ] - , testGroup - "point" - [ testProperty "marshalling" $ \rx ry -> - let p = P256.pointFromIntegers (unP256 rx, unP256 ry) - b = P256.pointToBinary p :: Bytes - p' = P256.unsafePointFromBinary b - in propertyHold [eqTest "point" (CryptoPassed p) p'] - , testProperty "marshalling-integer" $ \rx ry -> - let p = P256.pointFromIntegers (unP256 rx, unP256 ry) - (x, y) = P256.pointToIntegers p - in propertyHold [eqTest "x" (unP256 rx) x, eqTest "y" (unP256 ry) y] - , testCase "valid-point-1" $ casePointIsValid (xS, yS) - , testCase "valid-point-2" $ casePointIsValid (xR, yR) - , testCase "valid-point-3" $ casePointIsValid (xT, yT) - , -- The quotient estimate in crypton_p256_modmul can exceed the - -- true quotient, and the resulting borrow used to abort the - -- process on an assertion inside the reduction rather than - -- being corrected. Both points below are on the curve. - testCase "valid-point-reduction-1" $ casePointIsValid (xU, yU) - , testCase "valid-point-reduction-2" $ casePointIsValid (xV, yV) - , testCase "point-add-1" $ - let s = P256.pointFromIntegers (xS, yS) - t = P256.pointFromIntegers (xT, yT) - r = P256.pointFromIntegers (xR, yR) - in r @=? P256.pointAdd s t - , testProperty "point-add-infinity" casePointAddInfinity - , testProperty "lift-to-curve" propertyLiftToCurve - , testProperty "point-add" propertyPointAdd - , testProperty "point-add-infinity-identity" propertyPointAddInfinityIdentity - , testProperty "point-add-inverse" propertyPointAddInverse - , testProperty "point-negate" propertyPointNegate - , testProperty "point-mul" propertyPointMul - , testProperty "infinity" $ - let gN = P256.toPoint P256.scalarN - g1 = P256.pointBase - in propertyHold - [ eqTest "zero" True (P256.pointIsAtInfinity gN) - , eqTest "base" False (P256.pointIsAtInfinity g1) - ] - ] - ] - where - casePointIsValid pointTuple = - let s = P256.pointFromIntegers pointTuple in True @=? P256.pointIsValid s - - propertyLiftToCurve r = - let p = P256.toPoint (unP256Scalar r) - (x, y) = P256.pointToIntegers p - pEcc = ECC.pointMul curve (unP256 r) curveGen - in pEcc `propertyEq` ECC.Point x y - - propertyPointAdd r1 r2 = - let p1 = P256.toPoint (unP256Scalar r1) - p2 = P256.toPoint (unP256Scalar r2) - pe1 = ECC.pointMul curve (unP256 r1) curveGen - pe2 = ECC.pointMul curve (unP256 r2) curveGen - pR = P256.toPoint (P256.scalarAdd (unP256Scalar r1) (unP256Scalar r2)) - peR = ECC.pointAdd curve pe1 pe2 - in (unP256 r1 + unP256 r2) `mod` curveN - /= 0 - ==> propertyHold - [ eqTest "p256" pR (P256.pointAdd p1 p2) - , eqTest "ecc" peR (pointP256ToECC pR) - ] - - propertyPointNegate r = - let p = P256.toPoint (unP256Scalar r) - pe = ECC.pointMul curve (unP256 r) curveGen - pR = P256.pointNegate p - in ECC.pointNegate curve pe `propertyEq` pointP256ToECC pR - - propertyPointMul s' r' = - let s = modP256Scalar s' - r = modP256Scalar r' - p = P256.toPoint (unP256Scalar r) - pe = ECC.pointMul curve (unP256 r) curveGen - pR = P256.toPoint (P256.scalarMul (unP256Scalar s) (unP256Scalar r)) - peR = ECC.pointMul curve (unP256 s) pe - in propertyHold - [ eqTest "p256" pR (P256.pointMul (unP256Scalar s) p) - , eqTest "ecc" peR (pointP256ToECC pR) - ] - - pointInfinity :: P256.Point - pointInfinity = P256.pointFromIntegers (0, 0) - - casePointAddInfinity = - propertyHold - [ eqTest - "infinity + base" - P256.pointBase - (P256.pointAdd pointInfinity P256.pointBase) - , eqTest - "base + infinity" - P256.pointBase - (P256.pointAdd P256.pointBase pointInfinity) - , eqTest - "infinity + infinity" - pointInfinity - (P256.pointAdd pointInfinity pointInfinity) - ] - - propertyPointAddInfinityIdentity r = - let p = P256.toPoint (unP256Scalar r) - in propertyHold - [ eqTest - "infinity + p" - p - (P256.pointAdd pointInfinity p) - , eqTest - "p + infinity" - p - (P256.pointAdd p pointInfinity) - ] - - propertyPointAddInverse r = - let p = P256.toPoint (unP256Scalar r) - in propertyHold - [ eqTest - "p + negate p" - True - (P256.pointIsAtInfinity (P256.pointAdd p (P256.pointNegate p))) - , eqTest - "negate p + p" - True - (P256.pointIsAtInfinity (P256.pointAdd (P256.pointNegate p) p)) - ] diff --git a/tests/KAT_PubKey/PSS.hs b/tests/KAT_PubKey/PSS.hs deleted file mode 100644 --- a/tests/KAT_PubKey/PSS.hs +++ /dev/null @@ -1,491 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.PSS (pssTests) where - -import Crypto.PubKey.RSA -import qualified Crypto.PubKey.RSA.PSS as PSS - -import Imports - --- Module contains one vector generated by the implementation itself and other --- vectors from - -data VectorPSS = VectorPSS - { message :: ByteString - , salt :: ByteString - , signature :: ByteString - } - -rsaKeyInt = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0xa2ba40ee07e3b2bd2f02ce227f36a195024486e49c19cb41bbbdfbba98b22b0e577c2eeaffa20d883a76e65e394c69d4b3c05a1e8fadda27edb2a42bc000fe888b9b32c22d15add0cd76b3e7936e19955b220dd17d4ea904b1ec102b2e4de7751222aa99151024c7cb41cc5ea21d00eeb41f7c800834d2c6e06bce3bce7ea9a5 - , public_e = 0x010001 - , public_size = 128 - } - , private_d = - 0x50e2c3e38d886110288dfc68a9533e7e12e27d2aa56d2cdb3fb6efa990bcff29e1d2987fb711962860e7391b1ce01ebadb9e812d2fbdfaf25df4ae26110a6d7a26f0b810f54875e17dd5c9fb6d641761245b81e79f8c88f0e55a6dcd5f133abd35f8f4ec80adf1bf86277a582894cb6ebcd2162f1c7534f1f4947b129151b71 - , private_p = - 0xd17f655bf27c8b16d35462c905cc04a26f37e2a67fa9c0ce0dced472394a0df743fe7f929e378efdb368eddff453cf007af6d948e0ade757371f8a711e278f6b - , private_q = - 0xc6d92b6fee7414d1358ce1546fb62987530b90bd15e0f14963a5e2635adb69347ec0c01b2ab1763fd8ac1a592fb22757463a982425bb97a3a437c5bf86d03f2f - , private_dP = - 0x9d0dbf83e5ce9e4b1754dcd5cd05bcb7b55f1508330ea49f14d4e889550f8256cb5f806dff34b17ada44208853577d08e4262890acf752461cea05547601bc4f - , private_dQ = - 0x1291a524c6b7c059e90e46dc83b2171eb3fa98818fd179b6c8bf6cecaa476303abf283fe05769cfc495788fe5b1ddfde9e884a3cd5e936b7e955ebf97eb563b1 - , private_qinv = - 0xa63f1da38b950c9ad1c67ce0d677ec2914cd7d40062df42a67eb198a176f9742aac7c5fea14f2297662b84812c4defc49a8025ab4382286be4c03788dd01d69f - } - -rsaKey1 = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0xa56e4a0e701017589a5187dc7ea841d156f2ec0e36ad52a44dfeb1e61f7ad991d8c51056ffedb162b4c0f283a12a88a394dff526ab7291cbb307ceabfce0b1dfd5cd9508096d5b2b8b6df5d671ef6377c0921cb23c270a70e2598e6ff89d19f105acc2d3f0cb35f29280e1386b6f64c4ef22e1e1f20d0ce8cffb2249bd9a2137 - , public_e = 0x010001 - , public_size = 128 - } - , private_d = - 0x33a5042a90b27d4f5451ca9bbbd0b44771a101af884340aef9885f2a4bbe92e894a724ac3c568c8f97853ad07c0266c8c6a3ca0929f1e8f11231884429fc4d9ae55fee896a10ce707c3ed7e734e44727a39574501a532683109c2abacaba283c31b4bd2f53c3ee37e352cee34f9e503bd80c0622ad79c6dcee883547c6a3b325 - , private_p = - 0xe7e8942720a877517273a356053ea2a1bc0c94aa72d55c6e86296b2dfc967948c0a72cbccca7eacb35706e09a1df55a1535bd9b3cc34160b3b6dcd3eda8e6443 - , private_q = - 0xb69dca1cf7d4d7ec81e75b90fcca874abcde123fd2700180aa90479b6e48de8d67ed24f9f19d85ba275874f542cd20dc723e6963364a1f9425452b269a6799fd - , private_dP = - 0x28fa13938655be1f8a159cbaca5a72ea190c30089e19cd274a556f36c4f6e19f554b34c077790427bbdd8dd3ede2448328f385d81b30e8e43b2fffa027861979 - , private_dQ = - 0x1a8b38f398fa712049898d7fb79ee0a77668791299cdfa09efc0e507acb21ed74301ef5bfd48be455eaeb6e1678255827580a8e4e8e14151d1510a82a3f2e729 - , private_qinv = - 0x27156aba4126d24a81f3a528cbfb27f56886f840a9f6e86e17a44b94fe9319584b8e22fdde1e5a2e3bd8aa5ba8d8584194eb2190acf832b847f13a3d24a79f4d - } - -vectorInt = - VectorPSS - { message = - "\x85\x9e\xef\x2f\xd7\x8a\xca\x00\x30\x8b\xdc\x47\x11\x93\xbf\x55\xbf\x9d\x78\xdb\x8f\x8a\x67\x2b\x48\x46\x34\xf3\xc9\xc2\x6e\x64\x78\xae\x10\x26\x0f\xe0\xdd\x8c\x08\x2e\x53\xa5\x29\x3a\xf2\x17\x3c\xd5\x0c\x6d\x5d\x35\x4f\xeb\xf7\x8b\x26\x02\x1c\x25\xc0\x27\x12\xe7\x8c\xd4\x69\x4c\x9f\x46\x97\x77\xe4\x51\xe7\xf8\xe9\xe0\x4c\xd3\x73\x9c\x6b\xbf\xed\xae\x48\x7f\xb5\x56\x44\xe9\xca\x74\xff\x77\xa5\x3c\xb7\x29\x80\x2f\x6e\xd4\xa5\xff\xa8\xba\x15\x98\x90\xfc" - , salt = - "\xe3\xb5\xd5\xd0\x02\xc1\xbc\xe5\x0c\x2b\x65\xef\x88\xa1\x88\xd8\x3b\xce\x7e\x61" - , signature = - "\x8d\xaa\x62\x7d\x3d\xe7\x59\x5d\x63\x05\x6c\x7e\xc6\x59\xe5\x44\x06\xf1\x06\x10\x12\x8b\xaa\xe8\x21\xc8\xb2\xa0\xf3\x93\x6d\x54\xdc\x3b\xdc\xe4\x66\x89\xf6\xb7\x95\x1b\xb1\x8e\x84\x05\x42\x76\x97\x18\xd5\x71\x5d\x21\x0d\x85\xef\xbb\x59\x61\x92\x03\x2c\x42\xbe\x4c\x29\x97\x2c\x85\x62\x75\xeb\x6d\x5a\x45\xf0\x5f\x51\x87\x6f\xc6\x74\x3d\xed\xdd\x28\xca\xec\x9b\xb3\x0e\xa9\x9e\x02\xc3\x48\x82\x69\x60\x4f\xe4\x97\xf7\x4c\xcd\x7c\x7f\xca\x16\x71\x89\x71\x23\xcb\xd3\x0d\xef\x5d\x54\xa2\xb5\x53\x6a\xd9\x0a\x74\x7e" - } - -{- -# mHash = Hash(M) -# salt = random string of octets -# M' = Padding || mHash || salt -# H = Hash(M') -# DB = Padding || salt -# dbMask = MGF(H, length(DB)) -# maskedDB = DB xor dbMask (leftmost bit set to -# zero) -# EM = maskedDB || H || 0xbc - -# mHash: -37 b6 6a e0 44 58 43 35 3d 47 ec b0 b4 fd 14 c1 -10 e6 2d 6a - -# salt: - -# M': -00 00 00 00 00 00 00 00 37 b6 6a e0 44 58 43 35 -3d 47 ec b0 b4 fd 14 c1 10 e6 2d 6a e3 b5 d5 d0 -02 c1 bc e5 0c 2b 65 ef 88 a1 88 d8 3b ce 7e 61 - -# H: -df 1a 89 6f 9d 8b c8 16 d9 7c d7 a2 c4 3b ad 54 -6f be 8c fe - -# DB: -00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -00 00 00 00 00 00 01 e3 b5 d5 d0 02 c1 bc e5 0c -2b 65 ef 88 a1 88 d8 3b ce 7e 61 - -# dbMask: -66 e4 67 2e 83 6a d1 21 ba 24 4b ed 65 76 b8 67 -d9 a4 47 c2 8a 6e 66 a5 b8 7d ee 7f bc 7e 65 af -50 57 f8 6f ae 89 84 d9 ba 7f 96 9a d6 fe 02 a4 -d7 5f 74 45 fe fd d8 5b 6d 3a 47 7c 28 d2 4b a1 -e3 75 6f 79 2d d1 dc e8 ca 94 44 0e cb 52 79 ec -d3 18 3a 31 1f c8 97 39 a9 66 43 13 6e 8b 0f 46 -5e 87 a4 53 5c d4 c5 9b 10 02 8d - -# maskedDB: -66 e4 67 2e 83 6a d1 21 ba 24 4b ed 65 76 b8 67 -d9 a4 47 c2 8a 6e 66 a5 b8 7d ee 7f bc 7e 65 af -50 57 f8 6f ae 89 84 d9 ba 7f 96 9a d6 fe 02 a4 -d7 5f 74 45 fe fd d8 5b 6d 3a 47 7c 28 d2 4b a1 -e3 75 6f 79 2d d1 dc e8 ca 94 44 0e cb 52 79 ec -d3 18 3a 31 1f c8 96 da 1c b3 93 11 af 37 ea 4a -75 e2 4b db fd 5c 1d a0 de 7c ec - -# Encoded message EM: -66 e4 67 2e 83 6a d1 21 ba 24 4b ed 65 76 b8 67 -d9 a4 47 c2 8a 6e 66 a5 b8 7d ee 7f bc 7e 65 af -50 57 f8 6f ae 89 84 d9 ba 7f 96 9a d6 fe 02 a4 -d7 5f 74 45 fe fd d8 5b 6d 3a 47 7c 28 d2 4b a1 -e3 75 6f 79 2d d1 dc e8 ca 94 44 0e cb 52 79 ec -d3 18 3a 31 1f c8 96 da 1c b3 93 11 af 37 ea 4a -75 e2 4b db fd 5c 1d a0 de 7c ec df 1a 89 6f 9d -8b c8 16 d9 7c d7 a2 c4 3b ad 54 6f be 8c fe bc --} - -vectorsKey1 = - [ -- Example 1.1 - VectorPSS - { message = - "\xcd\xc8\x7d\xa2\x23\xd7\x86\xdf\x3b\x45\xe0\xbb\xbc\x72\x13\x26\xd1\xee\x2a\xf8\x06\xcc\x31\x54\x75\xcc\x6f\x0d\x9c\x66\xe1\xb6\x23\x71\xd4\x5c\xe2\x39\x2e\x1a\xc9\x28\x44\xc3\x10\x10\x2f\x15\x6a\x0d\x8d\x52\xc1\xf4\xc4\x0b\xa3\xaa\x65\x09\x57\x86\xcb\x76\x97\x57\xa6\x56\x3b\xa9\x58\xfe\xd0\xbc\xc9\x84\xe8\xb5\x17\xa3\xd5\xf5\x15\xb2\x3b\x8a\x41\xe7\x4a\xa8\x67\x69\x3f\x90\xdf\xb0\x61\xa6\xe8\x6d\xfa\xae\xe6\x44\x72\xc0\x0e\x5f\x20\x94\x57\x29\xcb\xeb\xe7\x7f\x06\xce\x78\xe0\x8f\x40\x98\xfb\xa4\x1f\x9d\x61\x93\xc0\x31\x7e\x8b\x60\xd4\xb6\x08\x4a\xcb\x42\xd2\x9e\x38\x08\xa3\xbc\x37\x2d\x85\xe3\x31\x17\x0f\xcb\xf7\xcc\x72\xd0\xb7\x1c\x29\x66\x48\xb3\xa4\xd1\x0f\x41\x62\x95\xd0\x80\x7a\xa6\x25\xca\xb2\x74\x4f\xd9\xea\x8f\xd2\x23\xc4\x25\x37\x02\x98\x28\xbd\x16\xbe\x02\x54\x6f\x13\x0f\xd2\xe3\x3b\x93\x6d\x26\x76\xe0\x8a\xed\x1b\x73\x31\x8b\x75\x0a\x01\x67\xd0" - , salt = - "\xde\xe9\x59\xc7\xe0\x64\x11\x36\x14\x20\xff\x80\x18\x5e\xd5\x7f\x3e\x67\x76\xaf" - , signature = - "\x90\x74\x30\x8f\xb5\x98\xe9\x70\x1b\x22\x94\x38\x8e\x52\xf9\x71\xfa\xac\x2b\x60\xa5\x14\x5a\xf1\x85\xdf\x52\x87\xb5\xed\x28\x87\xe5\x7c\xe7\xfd\x44\xdc\x86\x34\xe4\x07\xc8\xe0\xe4\x36\x0b\xc2\x26\xf3\xec\x22\x7f\x9d\x9e\x54\x63\x8e\x8d\x31\xf5\x05\x12\x15\xdf\x6e\xbb\x9c\x2f\x95\x79\xaa\x77\x59\x8a\x38\xf9\x14\xb5\xb9\xc1\xbd\x83\xc4\xe2\xf9\xf3\x82\xa0\xd0\xaa\x35\x42\xff\xee\x65\x98\x4a\x60\x1b\xc6\x9e\xb2\x8d\xeb\x27\xdc\xa1\x2c\x82\xc2\xd4\xc3\xf6\x6c\xd5\x00\xf1\xff\x2b\x99\x4d\x8a\x4e\x30\xcb\xb3\x3c" - } - , -- Example 1.2 - VectorPSS - { message = - "\x85\x13\x84\xcd\xfe\x81\x9c\x22\xed\x6c\x4c\xcb\x30\xda\xeb\x5c\xf0\x59\xbc\x8e\x11\x66\xb7\xe3\x53\x0c\x4c\x23\x3e\x2b\x5f\x8f\x71\xa1\xcc\xa5\x82\xd4\x3e\xcc\x72\xb1\xbc\xa1\x6d\xfc\x70\x13\x22\x6b\x9e" - , salt = - "\xef\x28\x69\xfa\x40\xc3\x46\xcb\x18\x3d\xab\x3d\x7b\xff\xc9\x8f\xd5\x6d\xf4\x2d" - , signature = - "\x3e\xf7\xf4\x6e\x83\x1b\xf9\x2b\x32\x27\x41\x42\xa5\x85\xff\xce\xfb\xdc\xa7\xb3\x2a\xe9\x0d\x10\xfb\x0f\x0c\x72\x99\x84\xf0\x4e\xf2\x9a\x9d\xf0\x78\x07\x75\xce\x43\x73\x9b\x97\x83\x83\x90\xdb\x0a\x55\x05\xe6\x3d\xe9\x27\x02\x8d\x9d\x29\xb2\x19\xca\x2c\x45\x17\x83\x25\x58\xa5\x5d\x69\x4a\x6d\x25\xb9\xda\xb6\x60\x03\xc4\xcc\xcd\x90\x78\x02\x19\x3b\xe5\x17\x0d\x26\x14\x7d\x37\xb9\x35\x90\x24\x1b\xe5\x1c\x25\x05\x5f\x47\xef\x62\x75\x2c\xfb\xe2\x14\x18\xfa\xfe\x98\xc2\x2c\x4d\x4d\x47\x72\x4f\xdb\x56\x69\xe8\x43" - } - , -- Example 1.3 - VectorPSS - { message = - "\xa4\xb1\x59\x94\x17\x61\xc4\x0c\x6a\x82\xf2\xb8\x0d\x1b\x94\xf5\xaa\x26\x54\xfd\x17\xe1\x2d\x58\x88\x64\x67\x9b\x54\xcd\x04\xef\x8b\xd0\x30\x12\xbe\x8d\xc3\x7f\x4b\x83\xaf\x79\x63\xfa\xff\x0d\xfa\x22\x54\x77\x43\x7c\x48\x01\x7f\xf2\xbe\x81\x91\xcf\x39\x55\xfc\x07\x35\x6e\xab\x3f\x32\x2f\x7f\x62\x0e\x21\xd2\x54\xe5\xdb\x43\x24\x27\x9f\xe0\x67\xe0\x91\x0e\x2e\x81\xca\x2c\xab\x31\xc7\x45\xe6\x7a\x54\x05\x8e\xb5\x0d\x99\x3c\xdb\x9e\xd0\xb4\xd0\x29\xc0\x6d\x21\xa9\x4c\xa6\x61\xc3\xce\x27\xfa\xe1\xd6\xcb\x20\xf4\x56\x4d\x66\xce\x47\x67\x58\x3d\x0e\x5f\x06\x02\x15\xb5\x90\x17\xbe\x85\xea\x84\x89\x39\x12\x7b\xd8\xc9\xc4\xd4\x7b\x51\x05\x6c\x03\x1c\xf3\x36\xf1\x7c\x99\x80\xf3\xb8\xf5\xb9\xb6\x87\x8e\x8b\x79\x7a\xa4\x3b\x88\x26\x84\x33\x3e\x17\x89\x3f\xe9\xca\xa6\xaa\x29\x9f\x7e\xd1\xa1\x8e\xe2\xc5\x48\x64\xb7\xb2\xb9\x9b\x72\x61\x8f\xb0\x25\x74\xd1\x39\xef\x50\xf0\x19\xc9\xee\xf4\x16\x97\x13\x38\xe7\xd4\x70" - , salt = - "\x71\x0b\x9c\x47\x47\xd8\x00\xd4\xde\x87\xf1\x2a\xfd\xce\x6d\xf1\x81\x07\xcc\x77" - , signature = - "\x66\x60\x26\xfb\xa7\x1b\xd3\xe7\xcf\x13\x15\x7c\xc2\xc5\x1a\x8e\x4a\xa6\x84\xaf\x97\x78\xf9\x18\x49\xf3\x43\x35\xd1\x41\xc0\x01\x54\xc4\x19\x76\x21\xf9\x62\x4a\x67\x5b\x5a\xbc\x22\xee\x7d\x5b\xaa\xff\xaa\xe1\xc9\xba\xca\x2c\xc3\x73\xb3\xf3\x3e\x78\xe6\x14\x3c\x39\x5a\x91\xaa\x7f\xac\xa6\x64\xeb\x73\x3a\xfd\x14\xd8\x82\x72\x59\xd9\x9a\x75\x50\xfa\xca\x50\x1e\xf2\xb0\x4e\x33\xc2\x3a\xa5\x1f\x4b\x9e\x82\x82\xef\xdb\x72\x8c\xc0\xab\x09\x40\x5a\x91\x60\x7c\x63\x69\x96\x1b\xc8\x27\x0d\x2d\x4f\x39\xfc\xe6\x12\xb1" - } - , -- Example 1.4 - VectorPSS - { message = "\xbc\x65\x67\x47\xfa\x9e\xaf\xb3\xf0" - , salt = - "\x05\x6f\x00\x98\x5d\xe1\x4d\x8e\xf5\xce\xa9\xe8\x2f\x8c\x27\xbe\xf7\x20\x33\x5e" - , signature = - "\x46\x09\x79\x3b\x23\xe9\xd0\x93\x62\xdc\x21\xbb\x47\xda\x0b\x4f\x3a\x76\x22\x64\x9a\x47\xd4\x64\x01\x9b\x9a\xea\xfe\x53\x35\x9c\x17\x8c\x91\xcd\x58\xba\x6b\xcb\x78\xbe\x03\x46\xa7\xbc\x63\x7f\x4b\x87\x3d\x4b\xab\x38\xee\x66\x1f\x19\x96\x34\xc5\x47\xa1\xad\x84\x42\xe0\x3d\xa0\x15\xb1\x36\xe5\x43\xf7\xab\x07\xc0\xc1\x3e\x42\x25\xb8\xde\x8c\xce\x25\xd4\xf6\xeb\x84\x00\xf8\x1f\x7e\x18\x33\xb7\xee\x6e\x33\x4d\x37\x09\x64\xca\x79\xfd\xb8\x72\xb4\xd7\x52\x23\xb5\xee\xb0\x81\x01\x59\x1f\xb5\x32\xd1\x55\xa6\xde\x87" - } - , -- Example 1.5 - VectorPSS - { message = - "\xb4\x55\x81\x54\x7e\x54\x27\x77\x0c\x76\x8e\x8b\x82\xb7\x55\x64\xe0\xea\x4e\x9c\x32\x59\x4d\x6b\xff\x70\x65\x44\xde\x0a\x87\x76\xc7\xa8\x0b\x45\x76\x55\x0e\xee\x1b\x2a\xca\xbc\x7e\x8b\x7d\x3e\xf7\xbb\x5b\x03\xe4\x62\xc1\x10\x47\xea\xdd\x00\x62\x9a\xe5\x75\x48\x0a\xc1\x47\x0f\xe0\x46\xf1\x3a\x2b\xf5\xaf\x17\x92\x1d\xc4\xb0\xaa\x8b\x02\xbe\xe6\x33\x49\x11\x65\x1d\x7f\x85\x25\xd1\x0f\x32\xb5\x1d\x33\xbe\x52\x0d\x3d\xdf\x5a\x70\x99\x55\xa3\xdf\xe7\x82\x83\xb9\xe0\xab\x54\x04\x6d\x15\x0c\x17\x7f\x03\x7f\xdc\xcc\x5b\xe4\xea\x5f\x68\xb5\xe5\xa3\x8c\x9d\x7e\xdc\xcc\xc4\x97\x5f\x45\x5a\x69\x09\xb4" - , salt = - "\x80\xe7\x0f\xf8\x6a\x08\xde\x3e\xc6\x09\x72\xb3\x9b\x4f\xbf\xdc\xea\x67\xae\x8e" - , signature = - "\x1d\x2a\xad\x22\x1c\xa4\xd3\x1d\xdf\x13\x50\x92\x39\x01\x93\x98\xe3\xd1\x4b\x32\xdc\x34\xdc\x5a\xf4\xae\xae\xa3\xc0\x95\xaf\x73\x47\x9c\xf0\xa4\x5e\x56\x29\x63\x5a\x53\xa0\x18\x37\x76\x15\xb1\x6c\xb9\xb1\x3b\x3e\x09\xd6\x71\xeb\x71\xe3\x87\xb8\x54\x5c\x59\x60\xda\x5a\x64\x77\x6e\x76\x8e\x82\xb2\xc9\x35\x83\xbf\x10\x4c\x3f\xdb\x23\x51\x2b\x7b\x4e\x89\xf6\x33\xdd\x00\x63\xa5\x30\xdb\x45\x24\xb0\x1c\x3f\x38\x4c\x09\x31\x0e\x31\x5a\x79\xdc\xd3\xd6\x84\x02\x2a\x7f\x31\xc8\x65\xa6\x64\xe3\x16\x97\x8b\x75\x9f\xad" - } - , -- Example 1.6 - VectorPSS - { message = - "\x10\xaa\xe9\xa0\xab\x0b\x59\x5d\x08\x41\x20\x7b\x70\x0d\x48\xd7\x5f\xae\xdd\xe3\xb7\x75\xcd\x6b\x4c\xc8\x8a\xe0\x6e\x46\x94\xec\x74\xba\x18\xf8\x52\x0d\x4f\x5e\xa6\x9c\xbb\xe7\xcc\x2b\xeb\xa4\x3e\xfd\xc1\x02\x15\xac\x4e\xb3\x2d\xc3\x02\xa1\xf5\x3d\xc6\xc4\x35\x22\x67\xe7\x93\x6c\xfe\xbf\x7c\x8d\x67\x03\x57\x84\xa3\x90\x9f\xa8\x59\xc7\xb7\xb5\x9b\x8e\x39\xc5\xc2\x34\x9f\x18\x86\xb7\x05\xa3\x02\x67\xd4\x02\xf7\x48\x6a\xb4\xf5\x8c\xad\x5d\x69\xad\xb1\x7a\xb8\xcd\x0c\xe1\xca\xf5\x02\x5a\xf4\xae\x24\xb1\xfb\x87\x94\xc6\x07\x0c\xc0\x9a\x51\xe2\xf9\x91\x13\x11\xe3\x87\x7d\x00\x44\xc7\x1c\x57\xa9\x93\x39\x50\x08\x80\x6b\x72\x3a\xc3\x83\x73\xd3\x95\x48\x18\x18\x52\x8c\x1e\x70\x53\x73\x92\x82\x05\x35\x29\x51\x0e\x93\x5c\xd0\xfa\x77\xb8\xfa\x53\xcc\x2d\x47\x4b\xd4\xfb\x3c\xc5\xc6\x72\xd6\xff\xdc\x90\xa0\x0f\x98\x48\x71\x2c\x4b\xcf\xe4\x6c\x60\x57\x36\x59\xb1\x1e\x64\x57\xe8\x61\xf0\xf6\x04\xb6\x13\x8d\x14\x4f\x8c\xe4\xe2\xda\x73" - , salt = - "\xa8\xab\x69\xdd\x80\x1f\x00\x74\xc2\xa1\xfc\x60\x64\x98\x36\xc6\x16\xd9\x96\x81" - , signature = - "\x2a\x34\xf6\x12\x5e\x1f\x6b\x0b\xf9\x71\xe8\x4f\xbd\x41\xc6\x32\xbe\x8f\x2c\x2a\xce\x7d\xe8\xb6\x92\x6e\x31\xff\x93\xe9\xaf\x98\x7f\xbc\x06\xe5\x1e\x9b\xe1\x4f\x51\x98\xf9\x1f\x3f\x95\x3b\xd6\x7d\xa6\x0a\x9d\xf5\x97\x64\xc3\xdc\x0f\xe0\x8e\x1c\xbe\xf0\xb7\x5f\x86\x8d\x10\xad\x3f\xba\x74\x9f\xef\x59\xfb\x6d\xac\x46\xa0\xd6\xe5\x04\x36\x93\x31\x58\x6f\x58\xe4\x62\x8f\x39\xaa\x27\x89\x82\x54\x3b\xc0\xee\xb5\x37\xdc\x61\x95\x80\x19\xb3\x94\xfb\x27\x3f\x21\x58\x58\xa0\xa0\x1a\xc4\xd6\x50\xb9\x55\xc6\x7f\x4c\x58" - } - ] - --- ================================== --- Example 2: A 1025-bit RSA Key Pair --- ================================== - -rsaKey2 = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0x01d40c1bcf97a68ae7cdbd8a7bf3e34fa19dcca4ef75a47454375f94514d88fed006fb829f8419ff87d6315da68a1ff3a0938e9abb3464011c303ad99199cf0c7c7a8b477dce829e8844f625b115e5e9c4a59cf8f8113b6834336a2fd2689b472cbb5e5cabe674350c59b6c17e176874fb42f8fc3d176a017edc61fd326c4b33c9 - , public_e = 0x010001 - , public_size = 129 - } - , private_d = - 0x027d147e4673057377fd1ea201565772176a7dc38358d376045685a2e787c23c15576bc16b9f444402d6bfc5d98a3e88ea13ef67c353eca0c0ddba9255bd7b8bb50a644afdfd1dd51695b252d22e7318d1b6687a1c10ff75545f3db0fe602d5f2b7f294e3601eab7b9d1cecd767f64692e3e536ca2846cb0c2dd486a39fa75b1 - , private_p = - 0x016601e926a0f8c9e26ecab769ea65a5e7c52cc9e080ef519457c644da6891c5a104d3ea7955929a22e7c68a7af9fcad777c3ccc2b9e3d3650bce404399b7e59d1 - , private_q = - 0x014eafa1d4d0184da7e31f877d1281ddda625664869e8379e67ad3b75eae74a580e9827abd6eb7a002cb5411f5266797768fb8e95ae40e3e8a01f35ff89e56c079 - , private_dP = - 0xe247cce504939b8f0a36090de200938755e2444b29539a7da7a902f6056835c0db7b52559497cfe2c61a8086d0213c472c78851800b171f6401de2e9c2756f31 - , private_dQ = - 0xb12fba757855e586e46f64c38a70c68b3f548d93d787b399999d4c8f0bbd2581c21e19ed0018a6d5d3df86424b3abcad40199d31495b61309f27c1bf55d487c1 - , private_qinv = - 0x564b1e1fa003bda91e89090425aac05b91da9ee25061e7628d5f51304a84992fdc33762bd378a59f030a334d532bd0dae8f298ea9ed844636ad5fb8cbdc03cad - } - -vectorsKey2 = - [ -- Example 2.1 - VectorPSS - { message = - "\xda\xba\x03\x20\x66\x26\x3f\xae\xdb\x65\x98\x48\x11\x52\x78\xa5\x2c\x44\xfa\xa3\xa7\x6f\x37\x51\x5e\xd3\x36\x32\x10\x72\xc4\x0a\x9d\x9b\x53\xbc\x05\x01\x40\x78\xad\xf5\x20\x87\x51\x46\xaa\xe7\x0f\xf0\x60\x22\x6d\xcb\x7b\x1f\x1f\xc2\x7e\x93\x60" - , salt = - "\x57\xbf\x16\x0b\xcb\x02\xbb\x1d\xc7\x28\x0c\xf0\x45\x85\x30\xb7\xd2\x83\x2f\xf7" - , signature = - "\x01\x4c\x5b\xa5\x33\x83\x28\xcc\xc6\xe7\xa9\x0b\xf1\xc0\xab\x3f\xd6\x06\xff\x47\x96\xd3\xc1\x2e\x4b\x63\x9e\xd9\x13\x6a\x5f\xec\x6c\x16\xd8\x88\x4b\xdd\x99\xcf\xdc\x52\x14\x56\xb0\x74\x2b\x73\x68\x68\xcf\x90\xde\x09\x9a\xdb\x8d\x5f\xfd\x1d\xef\xf3\x9b\xa4\x00\x7a\xb7\x46\xce\xfd\xb2\x2d\x7d\xf0\xe2\x25\xf5\x46\x27\xdc\x65\x46\x61\x31\x72\x1b\x90\xaf\x44\x53\x63\xa8\x35\x8b\x9f\x60\x76\x42\xf7\x8f\xab\x0a\xb0\xf4\x3b\x71\x68\xd6\x4b\xae\x70\xd8\x82\x78\x48\xd8\xef\x1e\x42\x1c\x57\x54\xdd\xf4\x2c\x25\x89\xb5\xb3" - } - , -- Example 2.2 - VectorPSS - { message = - "\xe4\xf8\x60\x1a\x8a\x6d\xa1\xbe\x34\x44\x7c\x09\x59\xc0\x58\x57\x0c\x36\x68\xcf\xd5\x1d\xd5\xf9\xcc\xd6\xad\x44\x11\xfe\x82\x13\x48\x6d\x78\xa6\xc4\x9f\x93\xef\xc2\xca\x22\x88\xce\xbc\x2b\x9b\x60\xbd\x04\xb1\xe2\x20\xd8\x6e\x3d\x48\x48\xd7\x09\xd0\x32\xd1\xe8\xc6\xa0\x70\xc6\xaf\x9a\x49\x9f\xcf\x95\x35\x4b\x14\xba\x61\x27\xc7\x39\xde\x1b\xb0\xfd\x16\x43\x1e\x46\x93\x8a\xec\x0c\xf8\xad\x9e\xb7\x2e\x83\x2a\x70\x35\xde\x9b\x78\x07\xbd\xc0\xed\x8b\x68\xeb\x0f\x5a\xc2\x21\x6b\xe4\x0c\xe9\x20\xc0\xdb\x0e\xdd\xd3\x86\x0e\xd7\x88\xef\xac\xca\xca\x50\x2d\x8f\x2b\xd6\xd1\xa7\xc1\xf4\x1f\xf4\x6f\x16\x81\xc8\xf1\xf8\x18\xe9\xc4\xf6\xd9\x1a\x0c\x78\x03\xcc\xc6\x3d\x76\xa6\x54\x4d\x84\x3e\x08\x4e\x36\x3b\x8a\xcc\x55\xaa\x53\x17\x33\xed\xb5\xde\xe5\xb5\x19\x6e\x9f\x03\xe8\xb7\x31\xb3\x77\x64\x28\xd9\xe4\x57\xfe\x3f\xbc\xb3\xdb\x72\x74\x44\x2d\x78\x58\x90\xe9\xcb\x08\x54\xb6\x44\x4d\xac\xe7\x91\xd7\x27\x3d\xe1\x88\x97\x19\x33\x8a\x77\xfe" - , salt = - "\x7f\x6d\xd3\x59\xe6\x04\xe6\x08\x70\xe8\x98\xe4\x7b\x19\xbf\x2e\x5a\x7b\x2a\x90" - , signature = - "\x01\x09\x91\x65\x6c\xca\x18\x2b\x7f\x29\xd2\xdb\xc0\x07\xe7\xae\x0f\xec\x15\x8e\xb6\x75\x9c\xb9\xc4\x5c\x5f\xf8\x7c\x76\x35\xdd\x46\xd1\x50\x88\x2f\x4d\xe1\xe9\xae\x65\xe7\xf7\xd9\x01\x8f\x68\x36\x95\x4a\x47\xc0\xa8\x1a\x8a\x6b\x6f\x83\xf2\x94\x4d\x60\x81\xb1\xaa\x7c\x75\x9b\x25\x4b\x2c\x34\xb6\x91\xda\x67\xcc\x02\x26\xe2\x0b\x2f\x18\xb4\x22\x12\x76\x1d\xcd\x4b\x90\x8a\x62\xb3\x71\xb5\x91\x8c\x57\x42\xaf\x4b\x53\x7e\x29\x69\x17\x67\x4f\xb9\x14\x19\x47\x61\x62\x1c\xc1\x9a\x41\xf6\xfb\x95\x3f\xbc\xbb\x64\x9d\xea" - } - , -- Example 2.3 - VectorPSS - { message = - "\x52\xa1\xd9\x6c\x8a\xc3\x9e\x41\xe4\x55\x80\x98\x01\xb9\x27\xa5\xb4\x45\xc1\x0d\x90\x2a\x0d\xcd\x38\x50\xd2\x2a\x66\xd2\xbb\x07\x03\xe6\x7d\x58\x67\x11\x45\x95\xaa\xbf\x5a\x7a\xeb\x5a\x8f\x87\x03\x4b\xbb\x30\xe1\x3c\xfd\x48\x17\xa9\xbe\x76\x23\x00\x23\x60\x6d\x02\x86\xa3\xfa\xf8\xa4\xd2\x2b\x72\x8e\xc5\x18\x07\x9f\x9e\x64\x52\x6e\x3a\x0c\xc7\x94\x1a\xa3\x38\xc4\x37\x99\x7c\x68\x0c\xca\xc6\x7c\x66\xbf\xa1" - , salt = - "\xfc\xa8\x62\x06\x8b\xce\x22\x46\x72\x4b\x70\x8a\x05\x19\xda\x17\xe6\x48\x68\x8c" - , signature = - "\x00\x7f\x00\x30\x01\x8f\x53\xcd\xc7\x1f\x23\xd0\x36\x59\xfd\xe5\x4d\x42\x41\xf7\x58\xa7\x50\xb4\x2f\x18\x5f\x87\x57\x85\x20\xc3\x07\x42\xaf\xd8\x43\x59\xb6\xe6\xe8\xd3\xed\x95\x9d\xc6\xfe\x48\x6b\xed\xc8\xe2\xcf\x00\x1f\x63\xa7\xab\xe1\x62\x56\xa1\xb8\x4d\xf0\xd2\x49\xfc\x05\xd3\x19\x4c\xe5\xf0\x91\x27\x42\xdb\xbf\x80\xdd\x17\x4f\x6c\x51\xf6\xba\xd7\xf1\x6c\xf3\x36\x4e\xba\x09\x5a\x06\x26\x7d\xc3\x79\x38\x03\xac\x75\x26\xae\xbe\x0a\x47\x5d\x38\xb8\xc2\x24\x7a\xb5\x1c\x48\x98\xdf\x70\x47\xdc\x6a\xdf\x52\xc6\xc4" - } - , -- Example 2.4 - VectorPSS - { message = - "\xa7\x18\x2c\x83\xac\x18\xbe\x65\x70\xa1\x06\xaa\x9d\x5c\x4e\x3d\xbb\xd4\xaf\xae\xb0\xc6\x0c\x4a\x23\xe1\x96\x9d\x79\xff" - , salt = - "\x80\x70\xef\x2d\xe9\x45\xc0\x23\x87\x68\x4b\xa0\xd3\x30\x96\x73\x22\x35\xd4\x40" - , signature = - "\x00\x9c\xd2\xf4\xed\xbe\x23\xe1\x23\x46\xae\x8c\x76\xdd\x9a\xd3\x23\x0a\x62\x07\x61\x41\xf1\x6c\x15\x2b\xa1\x85\x13\xa4\x8e\xf6\xf0\x10\xe0\xe3\x7f\xd3\xdf\x10\xa1\xec\x62\x9a\x0c\xb5\xa3\xb5\xd2\x89\x30\x07\x29\x8c\x30\x93\x6a\x95\x90\x3b\x6b\xa8\x55\x55\xd9\xec\x36\x73\xa0\x61\x08\xfd\x62\xa2\xfd\xa5\x6d\x1c\xe2\xe8\x5c\x4d\xb6\xb2\x4a\x81\xca\x3b\x49\x6c\x36\xd4\xfd\x06\xeb\x7c\x91\x66\xd8\xe9\x48\x77\xc4\x2b\xea\x62\x2b\x3b\xfe\x92\x51\xfd\xc2\x1d\x8d\x53\x71\xba\xda\xd7\x8a\x48\x82\x14\x79\x63\x35\xb4\x0b" - } - , -- Example 2.5 - VectorPSS - { message = - "\x86\xa8\x3d\x4a\x72\xee\x93\x2a\x4f\x56\x30\xaf\x65\x79\xa3\x86\xb7\x8f\xe8\x89\x99\xe0\xab\xd2\xd4\x90\x34\xa4\xbf\xc8\x54\xdd\x94\xf1\x09\x4e\x2e\x8c\xd7\xa1\x79\xd1\x95\x88\xe4\xae\xfc\x1b\x1b\xd2\x5e\x95\xe3\xdd\x46\x1f" - , salt = - "\x17\x63\x9a\x4e\x88\xd7\x22\xc4\xfc\xa2\x4d\x07\x9a\x8b\x29\xc3\x24\x33\xb0\xc9" - , signature = - "\x00\xec\x43\x08\x24\x93\x1e\xbd\x3b\xaa\x43\x03\x4d\xae\x98\xba\x64\x6b\x8c\x36\x01\x3d\x16\x71\xc3\xcf\x1c\xf8\x26\x0c\x37\x4b\x19\xf8\xe1\xcc\x8d\x96\x50\x12\x40\x5e\x7e\x9b\xf7\x37\x86\x12\xdf\xcc\x85\xfc\xe1\x2c\xda\x11\xf9\x50\xbd\x0b\xa8\x87\x67\x40\x43\x6c\x1d\x25\x95\xa6\x4a\x1b\x32\xef\xcf\xb7\x4a\x21\xc8\x73\xb3\xcc\x33\xaa\xf4\xe3\xdc\x39\x53\xde\x67\xf0\x67\x4c\x04\x53\xb4\xfd\x9f\x60\x44\x06\xd4\x41\xb8\x16\x09\x8c\xb1\x06\xfe\x34\x72\xbc\x25\x1f\x81\x5f\x59\xdb\x2e\x43\x78\xa3\xad\xdc\x18\x1e\xcf" - } - , -- Example 2.6 - VectorPSS - { message = - "\x04\x9f\x91\x54\xd8\x71\xac\x4a\x7c\x7a\xb4\x53\x25\xba\x75\x45\xa1\xed\x08\xf7\x05\x25\xb2\x66\x7c\xf1" - , salt = - "\x37\x81\x0d\xef\x10\x55\xed\x92\x2b\x06\x3d\xf7\x98\xde\x5d\x0a\xab\xf8\x86\xee" - , signature = - "\x00\x47\x5b\x16\x48\xf8\x14\xa8\xdc\x0a\xbd\xc3\x7b\x55\x27\xf5\x43\xb6\x66\xbb\x6e\x39\xd3\x0e\x5b\x49\xd3\xb8\x76\xdc\xcc\x58\xea\xc1\x4e\x32\xa2\xd5\x5c\x26\x16\x01\x44\x56\xad\x2f\x24\x6f\xc8\xe3\xd5\x60\xda\x3d\xdf\x37\x9a\x1c\x0b\xd2\x00\xf1\x02\x21\xdf\x07\x8c\x21\x9a\x15\x1b\xc8\xd4\xec\x9d\x2f\xc2\x56\x44\x67\x81\x10\x14\xef\x15\xd8\xea\x01\xc2\xeb\xbf\xf8\xc2\xc8\xef\xab\x38\x09\x6e\x55\xfc\xbe\x32\x85\xc7\xaa\x55\x88\x51\x25\x4f\xaf\xfa\x92\xc1\xc7\x2b\x78\x75\x86\x63\xef\x45\x82\x84\x31\x39\xd7\xa6" - } - ] - --- ================================== --- Example 3: A 1026-bit RSA Key Pair --- ================================== - -rsaKey3 = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0x02f246ef451ed3eebb9a310200cc25859c048e4be798302991112eb68ce6db674e280da21feded1ae74880ca522b18db249385012827c515f0e466a1ffa691d98170574e9d0eadb087586ca48933da3cc953d95bd0ed50de10ddcb6736107d6c831c7f663e833ca4c097e700ce0fb945f88fb85fe8e5a773172565b914a471a443 - , public_e = 0x010001 - , public_size = 129 - } - , private_d = - 0x651451733b56de5ac0a689a4aeb6e6894a69014e076c88dd7a667eab3232bbccd2fc44ba2fa9c31db46f21edd1fdb23c5c128a5da5bab91e7f952b67759c7cff705415ac9fa0907c7ca6178f668fb948d869da4cc3b7356f4008dfd5449d32ee02d9a477eb69fc29266e5d9070512375a50fbbcc27e238ad98425f6ebbf88991 - , private_p = - 0x01bd36e18ece4b0fdb2e9c9d548bd1a7d6e2c21c6fdc35074a1d05b1c6c8b3d558ea2639c9a9a421680169317252558bd148ad215aac550e2dcf12a82d0ebfe853 - , private_q = - 0x01b1b656ad86d8e19d5dc86292b3a192fdf6e0dd37877bad14822fa00190cab265f90d3f02057b6f54d6ecb14491e5adeacebc48bf0ebd2a2ad26d402e54f61651 - , private_dP = - 0x1f2779fd2e3e5e6bae05539518fba0cd0ead1aa4513a7cba18f1cf10e3f68195693d278a0f0ee72f89f9bc760d80e2f9d0261d516501c6ae39f14a476ce2ccf5 - , private_dQ = - 0x011a0d36794b04a854aab4b2462d439a5046c91d940b2bc6f75b62956fef35a2a6e63c5309817f307bbff9d59e7e331bd363f6d66849b18346adea169f0ae9aec1 - , private_qinv = - 0x0b30f0ecf558752fb3a6ce4ba2b8c675f659eba6c376585a1b39712d038ae3d2b46fcb418ae15d0905da6440e1513a30b9b7d6668fbc5e88e5ab7a175e73ba35 - } - -vectorsKey3 = - [ -- Example 3.1 - VectorPSS - { message = - "\x59\x4b\x37\x33\x3b\xbb\x2c\x84\x52\x4a\x87\xc1\xa0\x1f\x75\xfc\xec\x0e\x32\x56\xf1\x08\xe3\x8d\xca\x36\xd7\x0d\x00\x57" - , salt = - "\xf3\x1a\xd6\xc8\xcf\x89\xdf\x78\xed\x77\xfe\xac\xbc\xc2\xf8\xb0\xa8\xe4\xcf\xaa" - , signature = - "\x00\x88\xb1\x35\xfb\x17\x94\xb6\xb9\x6c\x4a\x3e\x67\x81\x97\xf8\xca\xc5\x2b\x64\xb2\xfe\x90\x7d\x6f\x27\xde\x76\x11\x24\x96\x4a\x99\xa0\x1a\x88\x27\x40\xec\xfa\xed\x6c\x01\xa4\x74\x64\xbb\x05\x18\x23\x13\xc0\x13\x38\xa8\xcd\x09\x72\x14\xcd\x68\xca\x10\x3b\xd5\x7d\x3b\xc9\xe8\x16\x21\x3e\x61\xd7\x84\xf1\x82\x46\x7a\xbf\x8a\x01\xcf\x25\x3e\x99\xa1\x56\xea\xa8\xe3\xe1\xf9\x0e\x3c\x6e\x4e\x3a\xa2\xd8\x3e\xd0\x34\x5b\x89\xfa\xfc\x9c\x26\x07\x7c\x14\xb6\xac\x51\x45\x4f\xa2\x6e\x44\x6e\x3a\x2f\x15\x3b\x2b\x16\x79\x7f" - } - , -- Example 3.2 - VectorPSS - { message = - "\x8b\x76\x95\x28\x88\x4a\x0d\x1f\xfd\x09\x0c\xf1\x02\x99\x3e\x79\x6d\xad\xcf\xbd\xdd\x38\xe4\x4f\xf6\x32\x4c\xa4\x51" - , salt = - "\xfc\xf9\xf0\xe1\xf1\x99\xa3\xd1\xd0\xda\x68\x1c\x5b\x86\x06\xfc\x64\x29\x39\xf7" - , signature = - "\x02\xa5\xf0\xa8\x58\xa0\x86\x4a\x4f\x65\x01\x7a\x7d\x69\x45\x4f\x3f\x97\x3a\x29\x99\x83\x9b\x7b\xbc\x48\xbf\x78\x64\x11\x69\x17\x95\x56\xf5\x95\xfa\x41\xf6\xff\x18\xe2\x86\xc2\x78\x30\x79\xbc\x09\x10\xee\x9c\xc3\x4f\x49\xba\x68\x11\x24\xf9\x23\xdf\xa8\x8f\x42\x61\x41\xa3\x68\xa5\xf5\xa9\x30\xc6\x28\xc2\xc3\xc2\x00\xe1\x8a\x76\x44\x72\x1a\x0c\xbe\xc6\xdd\x3f\x62\x79\xbd\xe3\xe8\xf2\xbe\x5e\x2d\x4e\xe5\x6f\x97\xe7\xce\xaf\x33\x05\x4b\xe7\x04\x2b\xd9\x1a\x63\xbb\x09\xf8\x97\xbd\x41\xe8\x11\x97\xde\xe9\x9b\x11\xaf" - } - , -- Example 3.3 - VectorPSS - { message = - "\x1a\xbd\xba\x48\x9c\x5a\xda\x2f\x99\x5e\xd1\x6f\x19\xd5\xa9\x4d\x9e\x6e\xc3\x4a\x8d\x84\xf8\x45\x57\xd2\x6e\x5e\xf9\xb0\x2b\x22\x88\x7e\x3f\x9a\x4b\x69\x0a\xd1\x14\x92\x09\xc2\x0c\x61\x43\x1f\x0c\x01\x7c\x36\xc2\x65\x7b\x35\xd7\xb0\x7d\x3f\x5a\xd8\x70\x85\x07\xa9\xc1\xb8\x31\xdf\x83\x5a\x56\xf8\x31\x07\x18\x14\xea\x5d\x3d\x8d\x8f\x6a\xde\x40\xcb\xa3\x8b\x42\xdb\x7a\x2d\x3d\x7a\x29\xc8\xf0\xa7\x9a\x78\x38\xcf\x58\xa9\x75\x7f\xa2\xfe\x4c\x40\xdf\x9b\xaa\x19\x3b\xfc\x6f\x92\xb1\x23\xad\x57\xb0\x7a\xce\x3e\x6a\xc0\x68\xc9\xf1\x06\xaf\xd9\xee\xb0\x3b\x4f\x37\xc2\x5d\xbf\xbc\xfb\x30\x71\xf6\xf9\x77\x17\x66\xd0\x72\xf3\xbb\x07\x0a\xf6\x60\x55\x32\x97\x3a\xe2\x50\x51" - , salt = - "\x98\x6e\x7c\x43\xdb\xb6\x71\xbd\x41\xb9\xa7\xf4\xb6\xaf\xc8\x0e\x80\x5f\x24\x23" - , signature = - "\x02\x44\xbc\xd1\xc8\xc1\x69\x55\x73\x6c\x80\x3b\xe4\x01\x27\x2e\x18\xcb\x99\x08\x11\xb1\x4f\x72\xdb\x96\x41\x24\xd5\xfa\x76\x06\x49\xcb\xb5\x7a\xfb\x87\x55\xdb\xb6\x2b\xf5\x1f\x46\x6c\xf2\x3a\x0a\x16\x07\x57\x6e\x98\x3d\x77\x8f\xce\xff\xa9\x2d\xf7\x54\x8a\xea\x8e\xa4\xec\xad\x2c\x29\xdd\x9f\x95\xbc\x07\xfe\x91\xec\xf8\xbe\xe2\x55\xbf\xe8\x76\x2f\xd7\x69\x0a\xa9\xbf\xa4\xfa\x08\x49\xef\x72\x8c\x2c\x42\xc4\x53\x23\x64\x52\x2d\xf2\xab\x7f\x9f\x8a\x03\xb6\x3f\x7a\x49\x91\x75\x82\x86\x68\xf5\xef\x5a\x29\xe3\x80\x2c" - } - , -- Example 3.4 - VectorPSS - { message = - "\x8f\xb4\x31\xf5\xee\x79\x2b\x6c\x2a\xc7\xdb\x53\xcc\x42\x86\x55\xae\xb3\x2d\x03\xf4\xe8\x89\xc5\xc2\x5d\xe6\x83\xc4\x61\xb5\x3a\xcf\x89\xf9\xf8\xd3\xaa\xbd\xf6\xb9\xf0\xc2\xa1\xde\x12\xe1\x5b\x49\xed\xb3\x91\x9a\x65\x2f\xe9\x49\x1c\x25\xa7\xfc\xe1\xf7\x22\xc2\x54\x36\x08\xb6\x9d\xc3\x75\xec" - , salt = - "\xf8\x31\x2d\x9c\x8e\xea\x13\xec\x0a\x4c\x7b\x98\x12\x0c\x87\x50\x90\x87\xc4\x78" - , signature = - "\x01\x96\xf1\x2a\x00\x5b\x98\x12\x9c\x8d\xf1\x3c\x4c\xb1\x6f\x8a\xa8\x87\xd3\xc4\x0d\x96\xdf\x3a\x88\xe7\x53\x2e\xf3\x9c\xd9\x92\xf2\x73\xab\xc3\x70\xbc\x1b\xe6\xf0\x97\xcf\xeb\xbf\x01\x18\xfd\x9e\xf4\xb9\x27\x15\x5f\x3d\xf2\x2b\x90\x4d\x90\x70\x2d\x1f\x7b\xa7\xa5\x2b\xed\x8b\x89\x42\xf4\x12\xcd\x7b\xd6\x76\xc9\xd1\x8e\x17\x03\x91\xdc\xd3\x45\xc0\x6a\x73\x09\x64\xb3\xf3\x0b\xcc\xe0\xbb\x20\xba\x10\x6f\x9a\xb0\xee\xb3\x9c\xf8\xa6\x60\x7f\x75\xc0\x34\x7f\x0a\xf7\x9f\x16\xaf\xa0\x81\xd2\xc9\x2d\x1e\xe6\xf8\x36\xb8" - } - , -- Example 3.5 - VectorPSS - { message = - "\xfe\xf4\x16\x1d\xfa\xaf\x9c\x52\x95\x05\x1d\xfc\x1f\xf3\x81\x0c\x8c\x9e\xc2\xe8\x66\xf7\x07\x54\x22\xc8\xec\x42\x16\xa9\xc4\xff\x49\x42\x7d\x48\x3c\xae\x10\xc8\x53\x4a\x41\xb2\xfd\x15\xfe\xe0\x69\x60\xec\x6f\xb3\xf7\xa7\xe9\x4a\x2f\x8a\x2e\x3e\x43\xdc\x4a\x40\x57\x6c\x30\x97\xac\x95\x3b\x1d\xe8\x6f\x0b\x4e\xd3\x6d\x64\x4f\x23\xae\x14\x42\x55\x29\x62\x24\x64\xca\x0c\xbf\x0b\x17\x41\x34\x72\x38\x15\x7f\xab\x59\xe4\xde\x55\x24\x09\x6d\x62\xba\xec\x63\xac\x64" - , salt = - "\x50\x32\x7e\xfe\xc6\x29\x2f\x98\x01\x9f\xc6\x7a\x2a\x66\x38\x56\x3e\x9b\x6e\x2d" - , signature = - "\x02\x1e\xca\x3a\xb4\x89\x22\x64\xec\x22\x41\x1a\x75\x2d\x92\x22\x10\x76\xd4\xe0\x1c\x0e\x6f\x0d\xde\x9a\xfd\x26\xba\x5a\xcf\x6d\x73\x9e\xf9\x87\x54\x5d\x16\x68\x3e\x56\x74\xc9\xe7\x0f\x1d\xe6\x49\xd7\xe6\x1d\x48\xd0\xca\xeb\x4f\xb4\xd8\xb2\x4f\xba\x84\xa6\xe3\x10\x8f\xee\x7d\x07\x05\x97\x32\x66\xac\x52\x4b\x4a\xd2\x80\xf7\xae\x17\xdc\x59\xd9\x6d\x33\x51\x58\x6b\x5a\x3b\xdb\x89\x5d\x1e\x1f\x78\x20\xac\x61\x35\xd8\x75\x34\x80\x99\x83\x82\xba\x32\xb7\x34\x95\x59\x60\x8c\x38\x74\x52\x90\xa8\x5e\xf4\xe9\xf9\xbd\x83" - } - , -- Example 3.6 - VectorPSS - { message = - "\xef\xd2\x37\xbb\x09\x8a\x44\x3a\xee\xb2\xbf\x6c\x3f\x8c\x81\xb8\xc0\x1b\x7f\xcb\x3f\xeb" - , salt = - "\xb0\xde\x3f\xc2\x5b\x65\xf5\xaf\x96\xb1\xd5\xcc\x3b\x27\xd0\xc6\x05\x30\x87\xb3" - , signature = - "\x01\x2f\xaf\xec\x86\x2f\x56\xe9\xe9\x2f\x60\xab\x0c\x77\x82\x4f\x42\x99\xa0\xca\x73\x4e\xd2\x6e\x06\x44\xd5\xd2\x22\xc7\xf0\xbd\xe0\x39\x64\xf8\xe7\x0a\x5c\xb6\x5e\xd4\x4e\x44\xd5\x6a\xe0\xed\xf1\xff\x86\xca\x03\x2c\xc5\xdd\x44\x04\xdb\xb7\x6a\xb8\x54\x58\x6c\x44\xee\xd8\x33\x6d\x08\xd4\x57\xce\x6c\x03\x69\x3b\x45\xc0\xf1\xef\xef\x93\x62\x4b\x95\xb8\xec\x16\x9c\x61\x6d\x20\xe5\x53\x8e\xbc\x0b\x67\x37\xa6\xf8\x2b\x4b\xc0\x57\x09\x24\xfc\x6b\x35\x75\x9a\x33\x48\x42\x62\x79\xf8\xb3\xd7\x74\x4e\x2d\x22\x24\x26\xce" - } - ] - --- ================================== --- Example 8: A 1031-bit RSA Key Pair --- ================================== - -rsaKey8 = - PrivateKey - { private_pub = - PublicKey - { public_n = - 0x495370a1fb18543c16d3631e3163255df62be6eee890d5f25509e4f778a8ea6fbbbcdf85dff64e0d972003ab3681fbba6dd41fd541829b2e582de9f2a4a4e0a2d0900bef4753db3cee0ee06c7dfae8b1d53b5953218f9cceea695b08668edeaadced9463b1d790d5ebf27e9115b46cad4d9a2b8efab0561b0810344739ada0733f - , public_e = 0x010001 - , public_size = 129 - } - , private_d = - 0x6c66ffe98980c38fcdeab5159898836165f4b4b817c4f6a8d486ee4ea9130fe9b9092bd136d184f95f504a607eac565846d2fdd6597a8967c7396ef95a6eeebb4578a643966dca4d8ee3de842de63279c618159c1ab54a89437b6a6120e4930afb52a4ba6ced8a4947ac64b30a3497cbe701c2d6266d517219ad0ec6d347dbe9 - , private_p = - 0x08dad7f11363faa623d5d6d5e8a319328d82190d7127d2846c439b0ab72619b0a43a95320e4ec34fc3a9cea876422305bd76c5ba7be9e2f410c8060645a1d29edb - , private_q = - 0x0847e732376fc7900f898ea82eb2b0fc418565fdae62f7d9ec4ce2217b97990dd272db157f99f63c0dcbb9fbacdbd4c4dadb6df67756358ca4174825b48f49706d - , private_dP = - 0x05c2a83c124b3621a2aa57ea2c3efe035eff4560f33ddebb7adab81fce69a0c8c2edc16520dda83d59a23be867963ac65f2cc710bbcfb96ee103deb771d105fd85 - , private_dQ = - 0x04cae8aa0d9faa165c87b682ec140b8ed3b50b24594b7a3b2c220b3669bb819f984f55310a1ae7823651d4a02e99447972595139363434e5e30a7e7d241551e1b9 - , private_qinv = - 0x07d3e47bf686600b11ac283ce88dbb3f6051e8efd04680e44c171ef531b80b2b7c39fc766320e2cf15d8d99820e96ff30dc69691839c4b40d7b06e45307dc91f3f - } - -vectorsKey8 = - [ -- Example 8.1 - VectorPSS - { message = - "\x81\x33\x2f\x4b\xe6\x29\x48\x41\x5e\xa1\xd8\x99\x79\x2e\xea\xcf\x6c\x6e\x1d\xb1\xda\x8b\xe1\x3b\x5c\xea\x41\xdb\x2f\xed\x46\x70\x92\xe1\xff\x39\x89\x14\xc7\x14\x25\x97\x75\xf5\x95\xf8\x54\x7f\x73\x56\x92\xa5\x75\xe6\x92\x3a\xf7\x8f\x22\xc6\x99\x7d\xdb\x90\xfb\x6f\x72\xd7\xbb\x0d\xd5\x74\x4a\x31\xde\xcd\x3d\xc3\x68\x58\x49\x83\x6e\xd3\x4a\xec\x59\x63\x04\xad\x11\x84\x3c\x4f\x88\x48\x9f\x20\x97\x35\xf5\xfb\x7f\xda\xf7\xce\xc8\xad\xdc\x58\x18\x16\x8f\x88\x0a\xcb\xf4\x90\xd5\x10\x05\xb7\xa8\xe8\x4e\x43\xe5\x42\x87\x97\x75\x71\xdd\x99\xee\xa4\xb1\x61\xeb\x2d\xf1\xf5\x10\x8f\x12\xa4\x14\x2a\x83\x32\x2e\xdb\x05\xa7\x54\x87\xa3\x43\x5c\x9a\x78\xce\x53\xed\x93\xbc\x55\x08\x57\xd7\xa9\xfb" - , salt = - "\x1d\x65\x49\x1d\x79\xc8\x64\xb3\x73\x00\x9b\xe6\xf6\xf2\x46\x7b\xac\x4c\x78\xfa" - , signature = - "\x02\x62\xac\x25\x4b\xfa\x77\xf3\xc1\xac\xa2\x2c\x51\x79\xf8\xf0\x40\x42\x2b\x3c\x5b\xaf\xd4\x0a\x8f\x21\xcf\x0f\xa5\xa6\x67\xcc\xd5\x99\x3d\x42\xdb\xaf\xb4\x09\xc5\x20\xe2\x5f\xce\x2b\x1e\xe1\xe7\x16\x57\x7f\x1e\xfa\x17\xf3\xda\x28\x05\x2f\x40\xf0\x41\x9b\x23\x10\x6d\x78\x45\xaa\xf0\x11\x25\xb6\x98\xe7\xa4\xdf\xe9\x2d\x39\x67\xbb\x00\xc4\xd0\xd3\x5b\xa3\x55\x2a\xb9\xa8\xb3\xee\xf0\x7c\x7f\xec\xdb\xc5\x42\x4a\xc4\xdb\x1e\x20\xcb\x37\xd0\xb2\x74\x47\x69\x94\x0e\xa9\x07\xe1\x7f\xbb\xca\x67\x3b\x20\x52\x23\x80\xc5" - } - , -- Example 8.2 - VectorPSS - { message = - "\xe2\xf9\x6e\xaf\x0e\x05\xe7\xba\x32\x6e\xcc\xa0\xba\x7f\xd2\xf7\xc0\x23\x56\xf3\xce\xde\x9d\x0f\xaa\xbf\x4f\xcc\x8e\x60\xa9\x73\xe5\x59\x5f\xd9\xea\x08" - , salt = - "\x43\x5c\x09\x8a\xa9\x90\x9e\xb2\x37\x7f\x12\x48\xb0\x91\xb6\x89\x87\xff\x18\x38" - , signature = - "\x27\x07\xb9\xad\x51\x15\xc5\x8c\x94\xe9\x32\xe8\xec\x0a\x28\x0f\x56\x33\x9e\x44\xa1\xb5\x8d\x4d\xdc\xff\x2f\x31\x2e\x5f\x34\xdc\xfe\x39\xe8\x9c\x6a\x94\xdc\xee\x86\xdb\xbd\xae\x5b\x79\xba\x4e\x08\x19\xa9\xe7\xbf\xd9\xd9\x82\xe7\xee\x6c\x86\xee\x68\x39\x6e\x8b\x3a\x14\xc9\xc8\xf3\x4b\x17\x8e\xb7\x41\xf9\xd3\xf1\x21\x10\x9b\xf5\xc8\x17\x2f\xad\xa2\xe7\x68\xf9\xea\x14\x33\x03\x2c\x00\x4a\x8a\xa0\x7e\xb9\x90\x00\x0a\x48\xdc\x94\xc8\xba\xc8\xaa\xbe\x2b\x09\xb1\xaa\x46\xc0\xa2\xaa\x0e\x12\xf6\x3f\xbb\xa7\x75\xba\x7e" - } - , -- Example 8.3 - VectorPSS - { message = - "\xe3\x5c\x6e\xd9\x8f\x64\xa6\xd5\xa6\x48\xfc\xab\x8a\xdb\x16\x33\x1d\xb3\x2e\x5d\x15\xc7\x4a\x40\xed\xf9\x4c\x3d\xc4\xa4\xde\x79\x2d\x19\x08\x89\xf2\x0f\x1e\x24\xed\x12\x05\x4a\x6b\x28\x79\x8f\xcb\x42\xd1\xc5\x48\x76\x9b\x73\x4c\x96\x37\x31\x42\x09\x2a\xed\x27\x76\x03\xf4\x73\x8d\xf4\xdc\x14\x46\x58\x6d\x0e\xc6\x4d\xa4\xfb\x60\x53\x6d\xb2\xae\x17\xfc\x7e\x3c\x04\xbb\xfb\xbb\xd9\x07\xbf\x11\x7c\x08\x63\x6f\xa1\x6f\x95\xf5\x1a\x62\x16\x93\x4d\x3e\x34\xf8\x50\x30\xf1\x7b\xbb\xc5\xba\x69\x14\x40\x58\xaf\xf0\x81\xe0\xb1\x9c\xf0\x3c\x17\x19\x5c\x5e\x88\x8b\xa5\x8f\x6f\xe0\xa0\x2e\x5c\x3b\xda\x97\x19\xa7" - , salt = - "\xc6\xeb\xbe\x76\xdf\x0c\x4a\xea\x32\xc4\x74\x17\x5b\x2f\x13\x68\x62\xd0\x45\x29" - , signature = - "\x2a\xd2\x05\x09\xd7\x8c\xf2\x6d\x1b\x6c\x40\x61\x46\x08\x6e\x4b\x0c\x91\xa9\x1c\x2b\xd1\x64\xc8\x7b\x96\x6b\x8f\xaa\x42\xaa\x0c\xa4\x46\x02\x23\x23\xba\x4b\x1a\x1b\x89\x70\x6d\x7f\x4c\x3b\xe5\x7d\x7b\x69\x70\x2d\x16\x8a\xb5\x95\x5e\xe2\x90\x35\x6b\x8c\x4a\x29\xed\x46\x7d\x54\x7e\xc2\x3c\xba\xdf\x28\x6c\xcb\x58\x63\xc6\x67\x9d\xa4\x67\xfc\x93\x24\xa1\x51\xc7\xec\x55\xaa\xc6\xdb\x40\x84\xf8\x27\x26\x82\x5c\xfe\x1a\xa4\x21\xbc\x64\x04\x9f\xb4\x2f\x23\x14\x8f\x9c\x25\xb2\xdc\x30\x04\x37\xc3\x8d\x42\x8a\xa7\x5f\x96" - } - , -- Example 8.4 - VectorPSS - { message = - "\xdb\xc5\xf7\x50\xa7\xa1\x4b\xe2\xb9\x3e\x83\x8d\x18\xd1\x4a\x86\x95\xe5\x2e\x8a\xdd\x9c\x0a\xc7\x33\xb8\xf5\x6d\x27\x47\xe5\x29\xa0\xcc\xa5\x32\xdd\x49\xb9\x02\xae\xfe\xd5\x14\x44\x7f\x9e\x81\xd1\x61\x95\xc2\x85\x38\x68\xcb\x9b\x30\xf7\xd0\xd4\x95\xc6\x9d\x01\xb5\xc5\xd5\x0b\x27\x04\x5d\xb3\x86\x6c\x23\x24\xa4\x4a\x11\x0b\x17\x17\x74\x6d\xe4\x57\xd1\xc8\xc4\x5c\x3c\xd2\xa9\x29\x70\xc3\xd5\x96\x32\x05\x5d\x4c\x98\xa4\x1d\x6e\x99\xe2\xa3\xdd\xd5\xf7\xf9\x97\x9a\xb3\xcd\x18\xf3\x75\x05\xd2\x51\x41\xde\x2a\x1b\xff\x17\xb3\xa7\xdc\xe9\x41\x9e\xcc\x38\x5c\xf1\x1d\x72\x84\x0f\x19\x95\x3f\xd0\x50\x92\x51\xf6\xca\xfd\xe2\x89\x3d\x0e\x75\xc7\x81\xba\x7a\x50\x12\xca\x40\x1a\x4f\xa9\x9e\x04\xb3\xc3\x24\x9f\x92\x6d\x5a\xfe\x82\xcc\x87\xda\xb2\x2c\x3c\x1b\x10\x5d\xe4\x8e\x34\xac\xe9\xc9\x12\x4e\x59\x59\x7a\xc7\xeb\xf8" - , salt = - "\x02\x1f\xdc\xc6\xeb\xb5\xe1\x9b\x1c\xb1\x6e\x9c\x67\xf2\x76\x81\x65\x7f\xe2\x0a" - , signature = - "\x1e\x24\xe6\xe5\x86\x28\xe5\x17\x50\x44\xa9\xeb\x6d\x83\x7d\x48\xaf\x12\x60\xb0\x52\x0e\x87\x32\x7d\xe7\x89\x7e\xe4\xd5\xb9\xf0\xdf\x0b\xe3\xe0\x9e\xd4\xde\xa8\xc1\x45\x4f\xf3\x42\x3b\xb0\x8e\x17\x93\x24\x5a\x9d\xf8\xbf\x6a\xb3\x96\x8c\x8e\xdd\xc3\xb5\x32\x85\x71\xc7\x7f\x09\x1c\xc5\x78\x57\x69\x12\xdf\xeb\xd1\x64\xb9\xde\x54\x54\xfe\x0b\xe1\xc1\xf6\x38\x5b\x32\x83\x60\xce\x67\xec\x7a\x05\xf6\xe3\x0e\xb4\x5c\x17\xc4\x8a\xc7\x00\x41\xd2\xca\xb6\x7f\x0a\x2a\xe7\xaa\xfd\xcc\x8d\x24\x5e\xa3\x44\x2a\x63\x00\xcc\xc7" - } - , -- Example 8.5 - VectorPSS - { message = - "\x04\xdc\x25\x1b\xe7\x2e\x88\xe5\x72\x34\x85\xb6\x38\x3a\x63\x7e\x2f\xef\xe0\x76\x60\xc5\x19\xa5\x60\xb8\xbc\x18\xbd\xed\xb8\x6e\xae\x23\x64\xea\x53\xba\x9d\xca\x6e\xb3\xd2\xe7\xd6\xb8\x06\xaf\x42\xb3\xe8\x7f\x29\x1b\x4a\x88\x81\xd5\xbf\x57\x2c\xc9\xa8\x5e\x19\xc8\x6a\xcb\x28\xf0\x98\xf9\xda\x03\x83\xc5\x66\xd3\xc0\xf5\x8c\xfd\x8f\x39\x5d\xcf\x60\x2e\x5c\xd4\x0e\x8c\x71\x83\xf7\x14\x99\x6e\x22\x97\xef" - , salt = - "\xc5\x58\xd7\x16\x7c\xbb\x45\x08\xad\xa0\x42\x97\x1e\x71\xb1\x37\x7e\xea\x42\x69" - , signature = - "\x33\x34\x1b\xa3\x57\x6a\x13\x0a\x50\xe2\xa5\xcf\x86\x79\x22\x43\x88\xd5\x69\x3f\x5a\xcc\xc2\x35\xac\x95\xad\xd6\x8e\x5e\xb1\xee\xc3\x16\x66\xd0\xca\x7a\x1c\xda\x6f\x70\xa1\xaa\x76\x2c\x05\x75\x2a\x51\x95\x0c\xdb\x8a\xf3\xc5\x37\x9f\x18\xcf\xe6\xb5\xbc\x55\xa4\x64\x82\x26\xa1\x5e\x91\x2e\xf1\x9a\xd7\x7a\xde\xea\x91\x1d\x67\xcf\xef\xd6\x9b\xa4\x3f\xa4\x11\x91\x35\xff\x64\x21\x17\xba\x98\x5a\x7e\x01\x00\x32\x5e\x95\x19\xf1\xca\x6a\x92\x16\xbd\xa0\x55\xb5\x78\x50\x15\x29\x11\x25\xe9\x0d\xcd\x07\xa2\xca\x96\x73\xee" - } - , -- Example 8.6 - VectorPSS - { message = - "\x0e\xa3\x7d\xf9\xa6\xfe\xa4\xa8\xb6\x10\x37\x3c\x24\xcf\x39\x0c\x20\xfa\x6e\x21\x35\xc4\x00\xc8\xa3\x4f\x5c\x18\x3a\x7e\x8e\xa4\xc9\xae\x09\x0e\xd3\x17\x59\xf4\x2d\xc7\x77\x19\xcc\xa4\x00\xec\xdc\xc5\x17\xac\xfc\x7a\xc6\x90\x26\x75\xb2\xef\x30\xc5\x09\x66\x5f\x33\x21\x48\x2f\xc6\x9a\x9f\xb5\x70\xd1\x5e\x01\xc8\x45\xd0\xd8\xe5\x0d\x2a\x24\xcb\xf1\xcf\x0e\x71\x49\x75\xa5\xdb\x7b\x18\xd9\xe9\xe9\xcb\x91\xb5\xcb\x16\x86\x90\x60\xed\x18\xb7\xb5\x62\x45\x50\x3f\x0c\xaf\x90\x35\x2b\x8d\xe8\x1c\xb5\xa1\xd9\xc6\x33\x60\x92\xf0\xcd" - , salt = - "\x76\xfd\x4e\x64\xfd\xc9\x8e\xb9\x27\xa0\x40\x3e\x35\xa0\x84\xe7\x6b\xa9\xf9\x2a" - , signature = - "\x1e\xd1\xd8\x48\xfb\x1e\xdb\x44\x12\x9b\xd9\xb3\x54\x79\x5a\xf9\x7a\x06\x9a\x7a\x00\xd0\x15\x10\x48\x59\x3e\x0c\x72\xc3\x51\x7f\xf9\xff\x2a\x41\xd0\xcb\x5a\x0a\xc8\x60\xd7\x36\xa1\x99\x70\x4f\x7c\xb6\xa5\x39\x86\xa8\x8b\xbd\x8a\xbc\xc0\x07\x6a\x2c\xe8\x47\x88\x00\x31\x52\x5d\x44\x9d\xa2\xac\x78\x35\x63\x74\xc5\x36\xe3\x43\xfa\xa7\xcb\xa4\x2a\x5a\xaa\x65\x06\x08\x77\x91\xc0\x6a\x8e\x98\x93\x35\xae\xd1\x9b\xfa\xb2\xd5\xe6\x7e\x27\xfb\x0c\x28\x75\xaf\x89\x6c\x21\xb6\xe8\xe7\x30\x9d\x04\xe4\xf6\x72\x7e\x69\x46\x3e" - } - ] - -doSignTest key i vector = testCase (show i) (Right (signature vector) @=? actual) - where - actual = - PSS.signWithSalt - (salt vector) - Nothing - PSS.defaultPSSParamsSHA1 - key - (message vector) - -doVerifyTest key i vector = testCase (show i) (True @=? actual) - where - actual = - PSS.verify - PSS.defaultPSSParamsSHA1 - (private_pub key) - (message vector) - (signature vector) - -pssTests = - testGroup - "RSA-PSS" - [ testGroup - "signature internal" - [doSignTest rsaKeyInt katZero vectorInt] - , testGroup - "verify internal" - [doVerifyTest rsaKeyInt katZero vectorInt] - , testGroup "signature key 1024" $ - zipWith (doSignTest rsaKey1) [katZero ..] vectorsKey1 - , testGroup "verify key 1024" $ - zipWith (doVerifyTest rsaKey1) [katZero ..] vectorsKey1 - , testGroup "signature key 1025" $ - zipWith (doSignTest rsaKey2) [katZero ..] vectorsKey2 - , testGroup "verify key 1025" $ - zipWith (doVerifyTest rsaKey2) [katZero ..] vectorsKey2 - , testGroup "signature key 1026" $ - zipWith (doSignTest rsaKey3) [katZero ..] vectorsKey3 - , testGroup "verify key 1026" $ - zipWith (doVerifyTest rsaKey3) [katZero ..] vectorsKey3 - , testGroup "signature key 1031" $ - zipWith (doSignTest rsaKey8) [katZero ..] vectorsKey8 - , testGroup "verify key 1031" $ - zipWith (doVerifyTest rsaKey8) [katZero ..] vectorsKey8 - ] diff --git a/tests/KAT_PubKey/RSA.hs b/tests/KAT_PubKey/RSA.hs deleted file mode 100644 --- a/tests/KAT_PubKey/RSA.hs +++ /dev/null @@ -1,130 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.RSA (rsaTests) where - -import Crypto.Hash -import qualified Crypto.PubKey.RSA as RSA -import qualified Crypto.PubKey.RSA.PKCS15 as RSA -import Data.Either - -import Imports - -data VectorRSA = VectorRSA - { size :: Int - , msg :: ByteString - , n :: Integer - , e :: Integer - , d :: Integer - , p :: Integer - , q :: Integer - , dP :: Integer - , dQ :: Integer - , qinv :: Integer - , sig :: Either RSA.Error ByteString - } - -vectorsSHA1 :: [VectorRSA] -vectorsSHA1 = - [ VectorRSA - { size = 2048 `div` 8 - , msg = "The quick brown fox jumps over the lazy dog" - , n = - 0x00c896c245fcca81775346c5f4f958229cab1aee08196dab4ee5959018b856aab93e4486f37a32da1a6804403c88473ecf9f1b9266fc682400d45329b6ec195710c98d9ba728bc09d767e7e9d9b8b102c3b7e7529b87f649a2a5ebe165da21863ec7842de600a834a8be2227bc989145b52f84ba685d45484a3d530745598a5d8a9e7551b3278bf139a770929f776aed5d43559205fe937df93eeb8ff3fb3f2ce22d4b8a5c17aeafd19758ac5e6251df09ef6a2858e8558a7f476dde4efe859ff2fcb97767614563033fd1d2d300196b1abf256f7badb16c17def3804946d1bf9cd51760176b41bbd506b44ff2bfe5bcd5052180da3cfbbc6cd6f662c06a8baed9 - , e = 0x10001 - , d = - 0x58aa533bae8f310536d95cdd796e5cf655a7f4b9bdcbbd62859743f7b95c0de10e462a44ebaa18c07d640ba4f6344fee648d427ca56bbf2662b45407187be70173a655bc6104257182eb7f720ef2a79f2de6619c804ffca299a7179df6fac4a57179daf4052c550295f0f111ab7ae38e406ff219f9c88b38cdbcaac51bdc4e961361b87e100d168fc08b298626a806b3bfeaa9579f400bbe6e3e6e4ae9b27446e1c5ce8c10c848b9ad7b6ed3a6b3871ad6a1a88af24e581da054845c197e8bae1582858410087c1180c4f0cc61689abfd0f61b8031910f3b3779e11a7fbe823d9a704c63c313f78c994975de834ee9ead5faf6c18b3e4248c51ba307776bf845 - , p = - 0x00f85bfcfe55af59445f21f67ab1d8617d1f84360556eeb660d5c466f29e4d2228f9cc3fde4c594ea97069a19c666b68b6d905b65738ae63de6c11f9181ee9262313e5165591651bb3abec192abbc8c3694550bcffa451a2e2d1976bf3ecbc4480354f8d8646133298156aaa626b8807c5295850f93686400835466b6a5ccec61b - , q = - 0x00cec28b22b1d37c6c60d25e9747cb1bebd1270f0306db56ed8533f392d6a0cfe6b3dde13789758cf89febac214ba96667e46599f89ca210dced550ca6092a854ff95dff80ea48ff1a83455f4bb93f2ececa782da03b85a789239e8be5264130628724ceab57c8f76e4c7e822bf4fbf334c7d32610bec65047433e0e3b636afe1b - , dP = - 0x52fe0a50c339514f33ab19be6e67ac4c2f97f2a55e236ef674f8a89e329ffbe64d731f749d76ca7e7c7e0fef3f9a6ce78d260784a600408736fdda8b60e8f0419088612a3ee7d695f7c171b78200d8abf8e9bdfe7f5e785beb45fa610c9eed151abb76c383ef2e5cfbeb24fcb68a426e741e7b108c53d859e5d39e5970a1f839 - , dQ = - 0x39ef91853b47038a6ae707d2642fa9b73e782f60adbf307085eeb4c5e496532b56234a4481a40ac870275da846c74506bf9d28b3dd501c618baf5548013185018fe2a301c0a48bb726297e367dc6129ba7685d8094ad32f0dea64295074f24fbb6dabd7e8daea686a5b09d512be89d91a09cae01eb332eb389480e3cddf2d119 - , qinv = - 0x09ce1fa29008ef4b9798e5b8ec213dbdfec4fab4403ebf4b8786ad401ef33bc880c40a990b0826f72415192a206a504b27d2ba45ca555706200ea8e7a9b42d4077e9e6e0d80d4144966c53a36d23d30d987322dcc0013efe8df3b6b5914a2ceefc22cc5de6d569731794e9894f18f11d36a79558dc4c3ae5db1ce9bd05e7bf2e - , sig = - Right - "\x56\x66\x99\x0f\xd4\xea\x2b\xe0\x6d\x46\x3b\x10\x99\x5b\x06\x32\x5e\xec\x29\xfe\xa4\x63\x4d\x54\xf6\x31\x74\x5d\x01\x5a\x67\x09\x2e\xa7\x02\x8a\x48\x00\x3c\x0d\xef\x04\xe7\x52\x46\xe0\xfa\xb1\x42\x26\x89\xe7\xec\x25\x44\x76\xa0\x86\x33\xb0\xbe\x22\x17\x88\x9b\x18\x4d\x3e\xc2\x9b\xd4\x61\x2b\x9e\xde\x08\x56\xf8\xd5\xee\xb8\x38\xf4\x3d\xda\x9a\xbb\x34\x58\x87\x71\x1d\x1a\x7e\xc7\x3d\x46\x39\x01\x79\x29\x8b\xa4\xcd\xce\xd7\xab\xcb\x2e\x94\x5c\xfd\x54\xcc\xef\x80\x31\xfc\x5e\x8f\xc2\x4d\x76\x1e\x4c\xbc\x50\x7a\x9b\x08\xae\x85\xeb\x6a\xe0\x80\xdc\xff\x60\x13\xb0\x31\x94\x14\x9d\x8f\x9f\x48\x38\xcf\x4c\x82\x9d\x3b\x68\xc6\xe4\xe9\x5d\x94\x74\xa2\xac\x1f\xb9\x84\x41\x86\x11\xeb\x2c\x50\x64\xd7\x00\xe0\x85\x21\x5a\xd7\xae\x9b\x4c\x8e\x6a\x92\x97\xac\xcc\xb8\x38\x4f\x41\xb9\x3d\xa9\xfe\x69\x8b\x04\x81\xad\xfb\x0f\x49\x74\xfe\x26\x9c\x86\x0c\xf3\xd1\x8e\xa1\xb5\xaf\xef\x85\x3d\xfe\xd0\x7c\xcf\x18\xe4\x0f\x14\x99\xea\x93\x61\x79\x16\xbf\x38\xac\xa2\xa2\xac\xac\x2d\xae\x21\x85\x71\x94\xda\x5d\xa1\x82\xa8\x76\x82\xe5\x2f" - } - , VectorRSA - { size = 360 `div` 8 - , msg = "The quick brown fox jumps over the lazy dog" - , n = - 0x00bc2d7481c83c8be55da4caeaf1a30dbf9a1226ba7443c0a66213180d3eb8e29c3162401b7be067dff8f571a8eb - , e = 0x10001 - , d = - 0x726fb62d82c707507a2d5055a6934136270d28ce350c3a36d89066e26fb54f5b33da0bc9a05c2084f2b39be4e1 - , p = 0x0e3ff89e1f95a461c9f5ee480fd7b13529a225f3ee07fb - , q = 0x0d349ebc89329b493c03451ad20155de9775df55c55fd1 - , dP = 0x00943adef9fb93a561967bab33f198c2c7414e777df997 - , dQ = 0x078de99ceb5392f7f327dfb97717a27ae2e4606dddaa71 - , qinv = 0x0c54d59eaa029844fb3fe33a180161590b1cb103cc668e - , sig = Left RSA.SignatureTooLong - } - , VectorRSA - { size = 368 `div` 8 - , msg = "The quick brown fox jumps over the lazy dog" - , n = - 0x009cff2fd20246e390d6860b48a3926e83086d1386f7147e9f195623cf8f18546ceb20d428b77e0748864c8f611cb7 - , e = 0x10001 - , d = - 0x0097706cbf6624dd448c3a36ce35c27d49762a4948ca33804178d2ff826f8d336aaed622801c8d76d442be371da841 - , p = 0x00d12519f81441069ab1a86c38e0065e9578a46e655d5a17 - , q = 0x00c02b485ac3ee241d57b6b282f830d7d5bf6f4de75c1661 - , dP = 0x00a1af4611444f34f4d88d7504cf23fd711e70382c42ec07 - , dQ = 0x04226a4219a90bf9dda33e9ff6bb0649c0fea20c723cc1 - , qinv = 0x5dd87bf3c1e295dcc8602859a7cd74f05a2fe91a9d5877 - , sig = - Right - "\x51\xe4\xdd\x98\xee\xd5\x06\xef\x7a\xa5\x3c\xaf\x29\x33\xa4\x91\xfa\x8b\xb8\x09\xcf\x3e\xa1\x64\x92\x71\xad\x7b\x3a\x83\xb2\xa0\x77\x94\x4e\x59\xdf\x69\x58\x2e\xc8\x8d\xa0\x70\xfe\x7d" - } - ] - -vectorToPrivate :: VectorRSA -> RSA.PrivateKey -vectorToPrivate vector = - RSA.PrivateKey - { RSA.private_pub = vectorToPublic vector - , RSA.private_d = d vector - , RSA.private_p = p vector - , RSA.private_q = q vector - , RSA.private_dP = dP vector - , RSA.private_dQ = dQ vector - , RSA.private_qinv = qinv vector - } - -vectorToPublic :: VectorRSA -> RSA.PublicKey -vectorToPublic vector = - RSA.PublicKey - { RSA.public_size = size vector - , RSA.public_n = n vector - , RSA.public_e = e vector - } - -vectorHasSignature :: VectorRSA -> Bool -vectorHasSignature = isRight . sig - -doSignatureTest :: Show a => a -> VectorRSA -> TestTree -doSignatureTest i vector = testCase (show i) (expected @=? actual) - where - expected = sig vector - actual = RSA.sign Nothing (Just SHA1) (vectorToPrivate vector) (msg vector) - -doVerifyTest :: Show a => a -> VectorRSA -> TestTree -doVerifyTest i vector = testCase (show i) (True @=? actual) - where - actual = RSA.verify (Just SHA1) (vectorToPublic vector) (msg vector) bs - bs = fromRight (error "doVerifyTest") $ sig vector - -rsaTests :: TestTree -rsaTests = - testGroup - "RSA" - [ testGroup - "SHA1" - [ testGroup "signature" $ zipWith doSignatureTest [katZero ..] vectorsSHA1 - , testGroup "verify" $ - zipWith doVerifyTest [katZero ..] $ - filter vectorHasSignature vectorsSHA1 - ] - ] diff --git a/tests/KAT_PubKey/Rabin.hs b/tests/KAT_PubKey/Rabin.hs deleted file mode 100644 --- a/tests/KAT_PubKey/Rabin.hs +++ /dev/null @@ -1,242 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_PubKey.Rabin (rabinTests) where - -import qualified Data.ByteString as B - -import Crypto.Hash -import Crypto.Number.Serialize (os2ip) -import qualified Crypto.PubKey.Rabin.Basic as BRabin -import qualified Crypto.PubKey.Rabin.Modified as MRabin -import qualified Crypto.PubKey.Rabin.OAEP as OAEP -import qualified Crypto.PubKey.Rabin.RW as RW - -import Imports - -basicRabinKey = - BRabin.PrivateKey - { BRabin.private_pub = - BRabin.PublicKey - { BRabin.public_n = - 0xc9c4b0df9db989d93df4137fc2de2a9cee2610523f7a450ecbbf252babe98fba2f8e389c3e420c081e18f584c5746ca43f77f6af1fc79161f8bf8fbcb9564779986ecbe656dd16740cb8e399c33ff1dcc679e73c9c98a58c65a8673b7de57290a2d3191cb27e29d627f7ec6e874b1406051ffe9181e4d90d1b487b100ad30685 - , BRabin.public_size = 128 - } - , BRabin.private_p = - 0xe071f231ab5912285a1f8db199795f5efdea4c32f646a3436eaec091ba853a3092216f26b539bbac1fe2ab2e4fbb20aad272a434a1e909bf6d3028aecae2a7b7 - , BRabin.private_q = - 0xe6229470dc7da58bfcd962f1b3ddcf52304efbfb91d31c8ed84dbae2380c1ad2e338a523b4250863a689b3f262f949bd7a9f1a603c36634bb932dd71bf5daba3 - , BRabin.private_a = - 0x65956653f711a63b776ce45862d4cd78f1ad7b1f8ed118bb8b5ea5fffd59762da5dc7c5298e236a8e45d5c93477cbc51f214b1cd1a4980eda859c1cb05e55666 - , BRabin.private_b = - -0x63126dd9c5d6b5215f62012885570e1306b6a47ec1c46553f3b13ceae869149d14544438dbb976800cd62fbb52266f9a6405bc91f192a462c974bc8a6f832e03 - } - -modifiedRabinKey = - MRabin.PrivateKey - { MRabin.private_pub = - MRabin.PublicKey - { MRabin.public_n = - 0x9461a6e7c55cb610f20fd9af5d642404a63332a8d7c4fe7aa559cbcaec691e7216eed5d9322cb6a8619c220a0241b44e0d0a7cefda01fb84e59722b4e842ab5e190d214424bbdfed6d523426fc57a28045dfbb6e8159123077c542c0278ee2daf2d8993e286bf709a10a948da6b13008441581a22233f0ad3d5ebc5858ff7be5 - , MRabin.public_size = 128 - } - , MRabin.private_p = - 0xc401e0ddbe565a8797292389bebb561c35eb019116ba25cc6c865a8d3d7bc599626ddf0bc4f575c22f89144fe99fc3300dd497ec2b7acc0221e729a61756b3f3 - , MRabin.private_q = - 0xc1cc0e35f23f5086691a18c755881e3fe6937581948b109f47605b45d055e7b352e19ff729dfb33fbecb1d28b115e590449e5e4e228ab1876d889d3d41d87ec7 - , MRabin.private_d = - 0x128c34dcf8ab96c21e41fb35ebac848094c666551af89fcf54ab39795d8d23ce42dddabb264596d50c33844140483689c1a14f9dfb403f709cb2e4569d08556b9267e6460e84c69beda1defabd0285c4852c288b7ac27b78987bd19da337a6b1c7b123476732d9c0f656cc62a17f70e8fe34516cfa85ce6475bddeae9ffa0926 - } - -rwKey = - RW.PrivateKey - { RW.private_pub = - RW.PublicKey - { RW.public_n = - 0x992db4c84564c68d4ee2fe0903d938b41e83bcac48dfe8f2219ccee2ccbdefda4cbeea9f1c98a515c5f39a458f5ea11bca97102aaa3d9ac69e000093024e7b968359287cdf57bdacff5df1893df3539c7e358f037d49b5c6ae7110ab8117220c73b6265987039c2c97078fccacdd3f5a560aff5076fdc3958c532db28ab9a855 - , RW.public_size = 128 - } - , RW.private_p = - 0xc144dd739c45397d61868ca944a9729a7ad34cf90466c8f5c98a88f5ab5e3288bcfd31d4af1d441d23a756a60abd4cf05c3e0b0053eb150166a327ae31e9347b - , RW.private_q = - 0xcae5a381f25a27ae2c359068753118fc384471cd6027e88b8b910306fb940781261089259a3c569546677aebd268704c767a071dbd4f50cb9f15fe448788856f - , RW.private_d = - 0x1325b69908ac98d1a9dc5fc1207b271683d07795891bfd1e443399dc5997bdfb4997dd53e39314a2b8be7348b1ebd4237952e2055547b358d3c000126049cf729ee5d4f0ea170b902e343a8ef0831900b963ba07a3176088ab2ab095db449d0052150d6be7b5402f459f17c759f6f043b06a5da64cb86bb910d340f7fa28fdce - } - -data EncryptionVector = EncryptionVector - { seed :: ByteString - , plainText :: ByteString - , cipherText :: ByteString - } - -data SignatureVector = SignatureVector - { message :: ByteString - , padding :: ByteString - , signature :: Integer - } - -basicRabinEncryptionVectors = - [ EncryptionVector - { plainText = - "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" - , seed = - "\x0c\xc7\x42\xce\x4a\x9b\x7f\x32\xf9\x51\xbc\xb2\x51\xef\xd9\x25\xfe\x4f\xe3\x5f" - , cipherText = - "\xaf\xc7\x03\xe3\x9d\x2f\x81\xc6\x3a\x80\x2a\xd1\x44\x26\x3f\x17\x0c\x0a\xe6\x48\x68\x98\x23\x14\x8f\x95\xd2\xce\xbb\xe7\x3f\x49\x34\x76\x1d\x99\x30\x7b\xeb\x84\xe5\x2a\x10\xd2\x1e\x11\x7e\x65\xe8\x88\x24\xc1\x12\xeb\x19\x0d\x97\xcd\x12\x25\x6b\x1f\x9b\x0c\x40\x40\xa3\x47\x00\xb7\x11\xf8\x50\x08\x51\x79\xe8\x1b\xd1\x77\xe0\x99\xa7\xe1\x5c\x63\xda\x29\xc7\xde\x28\x5d\x60\xed\x8e\xb2\x12\xd4\xfe\xb8\x1a\x5d\x17\x65\x80\x62\x6e\x65\x5c\x37\x07\x1c\xfa\xff\xe6\x21\xa5\x9f\xcd\x6a\x6a\xce\xa6\x96\xb2\xc5\x08\xe6" - } - ] - -basicRabinSignatureVectors = - [ SignatureVector - { message = - "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" - , padding = "\xe9\x87\x17\x15\xa2\xe4\x30\x15" - , signature = - 0xac95807bdd03ca975690151d39d23d75e5db2731c4ba30b83c3f3ea74709e4d4e340d7dab952356a76c9b8705b214e28d59f5bdc7c7fdff4e104569e30359b5c65c2dcd5b94db58505cd8b188267121700beebd7edbee492e374514646471b5c3fa252a2580dc7343f455683815d6d7c590dd3bcaa7df41d8b08197ccb183408 - } - ] - -modifiedRabinSignatureVectors = - [ SignatureVector - { message = - "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" - , padding = B.empty -- not used - , signature = - 0x278c7c269119218ab7f501ea53a97ab15a3a5a263c6daed8980abec78291e9729e0e3457731cdea8ec31a7566e93d10fc9b2615fe3e54f4533a5506ac24a3bd286e270324e538066f0ddf503f9b5e0c18e18379659834906ebd99c0d31588c66e70fc653bc8865b9239999cbd35704917d8647d1199286c533233e3e03582dd - } - ] - -rwEncryptionVectors = - [ EncryptionVector - { plainText = - "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" - , seed = - "\x0c\xc7\x42\xce\x4a\x9b\x7f\x32\xf9\x51\xbc\xb2\x51\xef\xd9\x25\xfe\x4f\xe3\x5f" - , cipherText = - "\x40\xc2\xe3\x36\xac\x46\x72\x8a\xaf\x33\x75\xe1\x27\xd0\x38\x40\xe2\x24\x4e\x20\xa7\x5d\x85\xd3\x74\x81\x21\xfd\xc9\x40\x90\x80\x8c\xed\x2d\xd3\x5b\xc4\xb7\xc9\x7c\x80\xa5\x2f\x63\x86\x34\x4e\x8c\x92\x07\x86\x9e\xda\xfd\xf8\x11\x83\x8a\x5a\x23\xc1\xe6\x77\x37\x5d\xf9\x5c\x60\xd1\x6d\xfd\x0c\x54\xd1\x00\xe9\xab\x97\x6d\x8e\x83\x8b\x6e\x1a\x38\x73\x43\xe2\x24\xc2\xe2\x4e\x74\x3f\xe4\x4d\xdd\x27\xed\xc7\x72\x88\xd3\x0f\x93\xb3\xdb\xa2\xb7\xaf\x6d\xe9\xab\x76\x53\x63\xf9\x62\xd7\x52\x44\x61\x60\x5d\x2e\x9b\xf7" - } - ] - -rwSignatureVectors = - [ SignatureVector - { message = - "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" - , padding = B.empty -- not used - , signature = - 0x1e57b554a8e83aacd9d4067f9535991e7db47803250cded5cc8af5458a6bb11fea852139e0afe143f9339dd94a518e354e702134d1ae222460127829d92e8bf6441336f5ae7044ec7b6c3ad8b9aeeb1ea02a49798e020cb5b558120bbb51f060eb1608ba68f90cac7edb1051c177d3bdbb99d1ad92e8d75d6f72f1d06f1d25be - } - ] - -doBasicRabinEncryptTest key i vector = testCase (show i) (Right (cipherText vector) @=? actual) - where - actual = - BRabin.encryptWithSeed - (seed vector) - (OAEP.defaultOAEPParams SHA1) - key - (plainText vector) - -doBasicRabinDecryptTest key i vector = testCase (show i) (Just (plainText vector) @=? actual) - where - actual = BRabin.decrypt (OAEP.defaultOAEPParams SHA1) key (cipherText vector) - -doBasicRabinSignTest key i vector = - testCase - (show i) - ( Right (BRabin.Signature ((os2ip $ padding vector), (signature vector))) - @=? actual - ) - where - actual = BRabin.signWith (padding vector) key SHA1 (message vector) - -doBasicRabinVerifyTest key i vector = testCase (show i) (True @=? actual) - where - actual = - BRabin.verify - key - SHA1 - (message vector) - (BRabin.Signature ((os2ip $ padding vector), (signature vector))) - -doModifiedRabinSignTest key i vector = testCase (show i) (Right (signature vector) @=? actual) - where - actual = MRabin.sign key SHA1 (message vector) - -doModifiedRabinVerifyTest key i vector = testCase (show i) (True @=? actual) - where - actual = MRabin.verify key SHA1 (message vector) (signature vector) - -doRwEncryptTest key i vector = testCase (show i) (Right (cipherText vector) @=? actual) - where - actual = - RW.encryptWithSeed - (seed vector) - (OAEP.defaultOAEPParams SHA1) - key - (plainText vector) - -doRwDecryptTest key i vector = testCase (show i) (Just (plainText vector) @=? actual) - where - actual = RW.decrypt (OAEP.defaultOAEPParams SHA1) key (cipherText vector) - -doRwSignTest key i vector = testCase (show i) (Right (signature vector) @=? actual) - where - actual = RW.sign key SHA1 (message vector) - -doRwVerifyTest key i vector = testCase (show i) (True @=? actual) - where - actual = RW.verify key SHA1 (message vector) (signature vector) - -rabinTests = - testGroup - "Rabin" - [ testGroup - "Basic" - [ testGroup "encrypt" $ - zipWith - (doBasicRabinEncryptTest $ BRabin.private_pub basicRabinKey) - [katZero ..] - basicRabinEncryptionVectors - , testGroup "decrypt" $ - zipWith - (doBasicRabinDecryptTest basicRabinKey) - [katZero ..] - basicRabinEncryptionVectors - , testGroup "sign" $ - zipWith - (doBasicRabinSignTest basicRabinKey) - [katZero ..] - basicRabinSignatureVectors - , testGroup "verify" $ - zipWith - (doBasicRabinVerifyTest $ BRabin.private_pub basicRabinKey) - [katZero ..] - basicRabinSignatureVectors - ] - , testGroup - "Modified" - [ testGroup "sign" $ - zipWith - (doModifiedRabinSignTest modifiedRabinKey) - [katZero ..] - modifiedRabinSignatureVectors - , testGroup "verify" $ - zipWith - (doModifiedRabinVerifyTest $ MRabin.private_pub modifiedRabinKey) - [katZero ..] - modifiedRabinSignatureVectors - ] - , testGroup - "RW" - [ testGroup "encrypt" $ - zipWith - (doRwEncryptTest $ RW.private_pub rwKey) - [katZero ..] - rwEncryptionVectors - , testGroup "decrypt" $ - zipWith (doRwDecryptTest rwKey) [katZero ..] rwEncryptionVectors - , testGroup "sign" $ zipWith (doRwSignTest rwKey) [katZero ..] rwSignatureVectors - , testGroup "verify" $ - zipWith (doRwVerifyTest $ RW.private_pub rwKey) [katZero ..] rwSignatureVectors - ] - ] diff --git a/tests/KAT_RC4.hs b/tests/KAT_RC4.hs deleted file mode 100644 --- a/tests/KAT_RC4.hs +++ /dev/null @@ -1,42 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE ViewPatterns #-} - -module KAT_RC4 where - -import Test.Tasty -import Test.Tasty.HUnit - -import qualified Crypto.Cipher.RC4 as RC4 -import Data.ByteString (ByteString) -import Data.ByteString.Char8 () - --- taken from wikipedia pages -vectors :: [(ByteString, ByteString, ByteString)] -vectors = - [ - ( "Key" - , "Plaintext" - , "\xBB\xF3\x16\xE8\xD9\x40\xAF\x0A\xD3" - ) - , - ( "Wiki" - , "pedia" - , "\x10\x21\xBF\x04\x20" - ) - , - ( "Secret" - , "Attack at dawn" - , "\x45\xA0\x1F\x64\x5F\xC3\x5B\x38\x35\x52\x54\x4B\x9B\xF5" - ) - ] - -tests = - testGroup "RC4" $ - zipWith toKatTest is vectors - where - toKatTest i (key, plainText, cipherText) = - testCase - (show i) - (cipherText @=? snd (RC4.combine (RC4.initialize key) plainText)) - is :: [Int] - is = [1 ..] diff --git a/tests/KAT_Scrypt.hs b/tests/KAT_Scrypt.hs deleted file mode 100644 --- a/tests/KAT_Scrypt.hs +++ /dev/null @@ -1,41 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module KAT_Scrypt (tests) where - -import Data.ByteString (ByteString) -import Data.ByteString.Char8 () - -import Data.Word -import Test.Tasty -import Test.Tasty.HUnit - -import qualified Crypto.KDF.Scrypt as Scrypt - -vectors :: [((ByteString, ByteString, Word64, Int, Int, Int), ByteString)] -vectors = - [ - ( ("", "", 16, 1, 1, 64) - , "\x77\xd6\x57\x62\x38\x65\x7b\x20\x3b\x19\xca\x42\xc1\x8a\x04\x97\xf1\x6b\x48\x44\xe3\x07\x4a\xe8\xdf\xdf\xfa\x3f\xed\xe2\x14\x42\xfc\xd0\x06\x9d\xed\x09\x48\xf8\x32\x6a\x75\x3a\x0f\xc8\x1f\x17\xe8\xd3\xe0\xfb\x2e\x0d\x36\x28\xcf\x35\xe2\x0c\x38\xd1\x89\x06" - ) - , - ( ("password", "NaCl", 1024, 8, 16, 64) - , "\xfd\xba\xbe\x1c\x9d\x34\x72\x00\x78\x56\xe7\x19\x0d\x01\xe9\xfe\x7c\x6a\xd7\xcb\xc8\x23\x78\x30\xe7\x73\x76\x63\x4b\x37\x31\x62\x2e\xaf\x30\xd9\x2e\x22\xa3\x88\x6f\xf1\x09\x27\x9d\x98\x30\xda\xc7\x27\xaf\xb9\x4a\x83\xee\x6d\x83\x60\xcb\xdf\xa2\xcc\x06\x40" - ) - , - ( ("pleaseletmein", "SodiumChloride", 16384, 8, 1, 64) - , "\x70\x23\xbd\xcb\x3a\xfd\x73\x48\x46\x1c\x06\xcd\x81\xfd\x38\xeb\xfd\xa8\xfb\xba\x90\x4f\x8e\x3e\xa9\xb5\x43\xf6\x54\x5d\xa1\xf2\xd5\x43\x29\x55\x61\x3f\x0f\xcf\x62\xd4\x97\x05\x24\x2a\x9a\xf9\xe6\x1e\x85\xdc\x0d\x65\x1e\x40\xdf\xcf\x01\x7b\x45\x57\x58\x87" - ) - , - ( ("pleaseletmein", "SodiumChloride", 1048576, 8, 1, 64) - , "\x21\x01\xcb\x9b\x6a\x51\x1a\xae\xad\xdb\xbe\x09\xcf\x70\xf8\x81\xec\x56\x8d\x57\x4a\x2f\xfd\x4d\xab\xe5\xee\x98\x20\xad\xaa\x47\x8e\x56\xfd\x8f\x4b\xa5\xd0\x9f\xfa\x1c\x6d\x92\x7c\x40\xf4\xc3\x37\x30\x40\x49\xe8\xa9\x52\xfb\xcb\xf4\x5c\x6f\xa7\x7a\x41\xa4" - ) - ] - -tests = - testGroup "Scrypt" $ - zipWith toCase [(1 :: Int) ..] vectors - where - toCase i ((pass, salt, n, r, p, dklen), output) = - testCase - (show i) - (output @=? Scrypt.generate (Scrypt.Parameters n r p dklen) pass salt) diff --git a/tests/KAT_TripleDES.hs b/tests/KAT_TripleDES.hs deleted file mode 100644 --- a/tests/KAT_TripleDES.hs +++ /dev/null @@ -1,14 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} -{-# LANGUAGE ViewPatterns #-} - -module KAT_TripleDES (tests) where - -import BlockCipher -import qualified Crypto.Cipher.TripleDES as TripleDES -import Imports - -kats = defaultKATs - -tests = - localOption (QuickCheckTests 5) $ - testBlockCipher kats (undefined :: TripleDES.DES_EEE3) diff --git a/tests/KAT_Twofish.hs b/tests/KAT_Twofish.hs deleted file mode 100644 --- a/tests/KAT_Twofish.hs +++ /dev/null @@ -1,388 +0,0 @@ -module KAT_Twofish (tests) where - -import BlockCipher -import Imports - -import Crypto.Cipher.Twofish -import qualified Data.ByteString as B - -vectors_twofish128 = - [ KAT_ECB - (B.replicate 16 0x00) - (B.replicate 16 0x00) - ( B.pack - [ 0x9F - , 0x58 - , 0x9F - , 0x5C - , 0xF6 - , 0x12 - , 0x2C - , 0x32 - , 0xB6 - , 0xBF - , 0xEC - , 0x2F - , 0x2A - , 0xE8 - , 0xC3 - , 0x5A - ] - ) - , KAT_ECB - ( B.pack - [ 0x9F - , 0x58 - , 0x9F - , 0x5C - , 0xF6 - , 0x12 - , 0x2C - , 0x32 - , 0xB6 - , 0xBF - , 0xEC - , 0x2F - , 0x2A - , 0xE8 - , 0xC3 - , 0x5A - ] - ) - ( B.pack - [ 0xD4 - , 0x91 - , 0xDB - , 0x16 - , 0xE7 - , 0xB1 - , 0xC3 - , 0x9E - , 0x86 - , 0xCB - , 0x08 - , 0x6B - , 0x78 - , 0x9F - , 0x54 - , 0x19 - ] - ) - ( B.pack - [ 0x01 - , 0x9F - , 0x98 - , 0x09 - , 0xDE - , 0x17 - , 0x11 - , 0x85 - , 0x8F - , 0xAA - , 0xC3 - , 0xA3 - , 0xBA - , 0x20 - , 0xFB - , 0xC3 - ] - ) - ] - -vectors_twofish192 = - [ KAT_ECB - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xAB - , 0xCD - , 0xEF - , 0xFE - , 0xDC - , 0xBA - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - , 0x00 - , 0x11 - , 0x22 - , 0x33 - , 0x44 - , 0x55 - , 0x66 - , 0x77 - ] - ) - ( B.pack - [ 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - ] - ) - ( B.pack - [ 0xCF - , 0xD1 - , 0xD2 - , 0xE5 - , 0xA9 - , 0xBE - , 0x9C - , 0xDF - , 0x50 - , 0x1F - , 0x13 - , 0xB8 - , 0x92 - , 0xBD - , 0x22 - , 0x48 - ] - ) - , KAT_ECB - ( B.pack - [ 0x88 - , 0xB2 - , 0xB2 - , 0x70 - , 0x6B - , 0x10 - , 0x5E - , 0x36 - , 0xB4 - , 0x46 - , 0xBB - , 0x6D - , 0x73 - , 0x1A - , 0x1E - , 0x88 - , 0xEF - , 0xA7 - , 0x1F - , 0x78 - , 0x89 - , 0x65 - , 0xBD - , 0x44 - ] - ) - ( B.pack - [ 0x39 - , 0xDA - , 0x69 - , 0xD6 - , 0xBA - , 0x49 - , 0x97 - , 0xD5 - , 0x85 - , 0xB6 - , 0xDC - , 0x07 - , 0x3C - , 0xA3 - , 0x41 - , 0xB2 - ] - ) - ( B.pack - [ 0x18 - , 0x2B - , 0x02 - , 0xD8 - , 0x14 - , 0x97 - , 0xEA - , 0x45 - , 0xF9 - , 0xDA - , 0xAC - , 0xDC - , 0x29 - , 0x19 - , 0x3A - , 0x65 - ] - ) - ] - -vectors_twofish256 = - [ KAT_ECB - ( B.pack - [ 0x01 - , 0x23 - , 0x45 - , 0x67 - , 0x89 - , 0xAB - , 0xCD - , 0xEF - , 0xFE - , 0xDC - , 0xBA - , 0x98 - , 0x76 - , 0x54 - , 0x32 - , 0x10 - , 0x00 - , 0x11 - , 0x22 - , 0x33 - , 0x44 - , 0x55 - , 0x66 - , 0x77 - , 0x88 - , 0x99 - , 0xAA - , 0xBB - , 0xCC - , 0xDD - , 0xEE - , 0xFF - ] - ) - ( B.pack - [ 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - , 0x00 - ] - ) - ( B.pack - [ 0x37 - , 0x52 - , 0x7B - , 0xE0 - , 0x05 - , 0x23 - , 0x34 - , 0xB8 - , 0x9F - , 0x0C - , 0xFC - , 0xCA - , 0xE8 - , 0x7C - , 0xFA - , 0x20 - ] - ) - , KAT_ECB - ( B.pack - [ 0xD4 - , 0x3B - , 0xB7 - , 0x55 - , 0x6E - , 0xA3 - , 0x2E - , 0x46 - , 0xF2 - , 0xA2 - , 0x82 - , 0xB7 - , 0xD4 - , 0x5B - , 0x4E - , 0x0D - , 0x57 - , 0xFF - , 0x73 - , 0x9D - , 0x4D - , 0xC9 - , 0x2C - , 0x1B - , 0xD7 - , 0xFC - , 0x01 - , 0x70 - , 0x0C - , 0xC8 - , 0x21 - , 0x6F - ] - ) - ( B.pack - [ 0x90 - , 0xAF - , 0xE9 - , 0x1B - , 0xB2 - , 0x88 - , 0x54 - , 0x4F - , 0x2C - , 0x32 - , 0xDC - , 0x23 - , 0x9B - , 0x26 - , 0x35 - , 0xE6 - ] - ) - ( B.pack - [ 0x6C - , 0xB4 - , 0x56 - , 0x1C - , 0x40 - , 0xBF - , 0x0A - , 0x97 - , 0x05 - , 0x93 - , 0x1C - , 0xB6 - , 0xD4 - , 0x08 - , 0xE7 - , 0xFA - ] - ) - ] - -kats128 = defaultKATs{kat_ECB = vectors_twofish128} -kats192 = defaultKATs{kat_ECB = vectors_twofish192} -kats256 = defaultKATs{kat_ECB = vectors_twofish256} - -tests = - testGroup - "Twofish" - [ testBlockCipher kats128 (undefined :: Twofish128) - , testBlockCipher kats192 (undefined :: Twofish192) - , testBlockCipher kats256 (undefined :: Twofish256) - ] diff --git a/tests/KDF/Argon2Spec.hs b/tests/KDF/Argon2Spec.hs new file mode 100644 --- /dev/null +++ b/tests/KDF/Argon2Spec.hs @@ -0,0 +1,91 @@ +{-# LANGUAGE OverloadedStrings #-} + +module KDF.Argon2Spec (spec) where + +import Control.Exception (evaluate) +import Crypto.Error +import qualified Crypto.KDF.Argon2 as Argon2 +import qualified Data.ByteString as B +import Imports + +data KDFVector = KDFVector + { kdfPass :: ByteString + , kdfSalt :: ByteString + , kdfOptions :: Argon2.Options + , kdfResult :: ByteString + } + +argon2i_13 :: Argon2.TimeCost -> Argon2.MemoryCost -> Argon2.Options +argon2i_13 iters memory = + Argon2.Options + { Argon2.iterations = iters + , Argon2.memory = memory + , Argon2.parallelism = 1 + , Argon2.variant = Argon2.Argon2i + , Argon2.version = Argon2.Version13 + } + +vectors = + [ KDFVector + "password" + "somesalt" + (argon2i_13 2 65536) + "\xc1\x62\x88\x32\x14\x7d\x97\x20\xc5\xbd\x1c\xfd\x61\x36\x70\x78\x72\x9f\x6d\xfb\x6f\x8f\xea\x9f\xf9\x81\x58\xe0\xd7\x81\x6e\xd0" + ] + +kdfTests :: [Spec] +kdfTests = zipWith toKDFTest is vectors + where + toKDFTest i v = + it + (show i) + ( Argon2.hash (kdfOptions v) (kdfPass v) (kdfSalt v) (B.length $ kdfResult v) + `shouldBe` CryptoPassed (kdfResult v) + ) + + is :: [Int] + is = [1 ..] + +-- | 'Argon2.hash' returns a 'CryptoFailable', but the bounds on iterations, +-- memory and parallelism are only enforced by the C implementation, whose +-- return code was turned into an 'error' raised from inside the allocation. +-- Invalid options have to come back through the failure the type already +-- offers. +-- +-- The bytes are forced, because 'CryptoPassed' holds them lazily; a raise +-- rather than a 'CryptoFailed' therefore fails the example. +outcome :: CryptoFailable ByteString -> IO (Either CryptoError Int) +outcome (CryptoFailed err) = return (Left err) +outcome (CryptoPassed bs) = Right <$> evaluate (B.length bs) + +refuses :: String -> Argon2.Options -> Spec +refuses name options = + it name $ + outcome (Argon2.hash options pass salt outLen) + `shouldReturn` Left CryptoError_ParameterInvalid + +pass :: ByteString +pass = "password" + +salt :: ByteString +salt = "somesalt" + +outLen :: Int +outLen = 32 + +optionTests :: [Spec] +optionTests = + [ it "valid options hash" $ + outcome (Argon2.hash (argon2i_13 2 65536) pass salt outLen) + `shouldReturn` Right outLen + , refuses + "parallelism of 0 is refused" + (argon2i_13 2 65536){Argon2.parallelism = 0} + , refuses "iterations of 0 is refused" (argon2i_13 0 65536) + , refuses "memory below the minimum is refused" (argon2i_13 2 1) + ] + +spec :: Spec +spec = do + describe "KATs" $ sequence_ kdfTests + describe "options" $ sequence_ optionTests diff --git a/tests/KDF/BCryptPBKDFSpec.hs b/tests/KDF/BCryptPBKDFSpec.hs new file mode 100644 --- /dev/null +++ b/tests/KDF/BCryptPBKDFSpec.hs @@ -0,0 +1,235 @@ +{-# LANGUAGE OverloadedStrings #-} + +module KDF.BCryptPBKDFSpec (spec) where + +import qualified Data.ByteString as B + +import Control.Exception (evaluate) +import Test.Hspec + +import Crypto.Error +import Crypto.KDF.BCryptPBKDF ( + Parameters (..), + generate, + tryGenerate, + hashInternal, + tryHashInternal, + ) + +spec :: Spec +spec = do + describe "generate" $ do + it "1" generate1 + it "2" generate2 + it "3" generate3 + describe "hashInternal" $ do + it "1" hashInternal1 + describe "invalid parameters" $ do + it "rejects an iteration count below one" $ + evaluate (run (Parameters 0 32)) `shouldThrow` cryptoError + it "rejects an output length of zero" $ + evaluate (run (Parameters 1 0)) `shouldThrow` cryptoError + it "rejects an output length above 1024" $ + evaluate (run (Parameters 1 1025)) `shouldThrow` cryptoError + it "reports them without raising" $ do + run' (Parameters 0 32) `shouldBe` refused + run' (Parameters 1 0) `shouldBe` refused + run' (Parameters 1 1025) `shouldBe` refused + it "rejects a hashInternal input that is not 512 bits" $ do + evaluate + (hashInternal (B.replicate 63 0x61) (B.replicate 64 0x61) :: B.ByteString) + `shouldThrow` cryptoError + ( tryHashInternal (B.replicate 64 0x61) (B.replicate 63 0x61) + :: CryptoFailable B.ByteString + ) + `shouldBe` refused + where + run params = + generate params ("password" :: B.ByteString) ("salt" :: B.ByteString) + :: B.ByteString + run' params = + tryGenerate params ("password" :: B.ByteString) ("salt" :: B.ByteString) + :: CryptoFailable B.ByteString + refused = CryptoFailed CryptoError_ParameterInvalid + cryptoError e = e == CryptoError_ParameterInvalid + -- test vector taken from the go implementation by @dchest + generate1 = generate params pass salt `shouldBe` expected + where + params = Parameters 12 32 + pass = "password" :: B.ByteString + salt = "salt" :: B.ByteString + expected = + B.pack + [ 0x1a + , 0xe4 + , 0x2c + , 0x05 + , 0xd4 + , 0x87 + , 0xbc + , 0x02 + , 0xf6 + , 0x49 + , 0x21 + , 0xa4 + , 0xeb + , 0xe4 + , 0xea + , 0x93 + , 0xbc + , 0xac + , 0xfe + , 0x13 + , 0x5f + , 0xda + , 0x99 + , 0x97 + , 0x4c + , 0x06 + , 0xb7 + , 0xb0 + , 0x1f + , 0xae + , 0x14 + , 0x9a + ] + :: B.ByteString + + -- test vector generated with the go implemenation by @dchest + generate2 = generate params pass salt `shouldBe` expected + where + params = Parameters 7 71 + pass = "DieWuerdeDesMenschenIstUnantastbar" :: B.ByteString + salt = "Tafelsalz" :: B.ByteString + expected = + B.pack + [ 0x17 + , 0xb4 + , 0x76 + , 0xaa + , 0xd7 + , 0x42 + , 0x33 + , 0x49 + , 0x5c + , 0xe8 + , 0x79 + , 0x49 + , 0x15 + , 0x74 + , 0x4c + , 0x71 + , 0xf9 + , 0x99 + , 0x66 + , 0x89 + , 0x7a + , 0x60 + , 0xc3 + , 0x70 + , 0xb4 + , 0x3c + , 0xa8 + , 0x83 + , 0x80 + , 0x5a + , 0x56 + , 0xde + , 0x38 + , 0xbc + , 0x51 + , 0x8c + , 0xd4 + , 0xeb + , 0xd1 + , 0xcf + , 0x46 + , 0x0a + , 0x68 + , 0x3d + , 0xc8 + , 0x12 + , 0xcf + , 0xf8 + , 0x43 + , 0xce + , 0x21 + , 0x9d + , 0x98 + , 0x81 + , 0x20 + , 0x26 + , 0x6e + , 0x42 + , 0x0f + , 0xaa + , 0x75 + , 0x5d + , 0x09 + , 0x8d + , 0x45 + , 0xda + , 0xd5 + , 0x15 + , 0x6e + , 0x65 + , 0x1d + ] + :: B.ByteString + + -- test vector generated with the go implemenation by @dchest + generate3 = generate params pass salt `shouldBe` expected + where + params = Parameters 5 5 + pass = "ABC" :: B.ByteString + salt = "DEF" :: B.ByteString + expected = + B.pack + [ 0xdd + , 0x6e + , 0xa0 + , 0x69 + , 0x29 + ] + :: B.ByteString + + hashInternal1 = hashInternal passHash saltHash `shouldBe` expected + where + passHash = B.pack [0 .. 63] :: B.ByteString + saltHash = B.pack [64 .. 127] :: B.ByteString + expected = + B.pack + [ 0x87 + , 0x90 + , 0x48 + , 0x70 + , 0xee + , 0xf9 + , 0xde + , 0xdd + , 0xf8 + , 0xe7 + , 0x61 + , 0x1a + , 0x14 + , 0x01 + , 0x06 + , 0xe6 + , 0xaa + , 0xf1 + , 0xa3 + , 0x63 + , 0xd9 + , 0xa2 + , 0xc5 + , 0x04 + , 0xdb + , 0x35 + , 0x64 + , 0x43 + , 0x72 + , 0x1e + , 0xb5 + , 0x55 + ] + :: B.ByteString diff --git a/tests/KDF/BCryptSpec.hs b/tests/KDF/BCryptSpec.hs new file mode 100644 --- /dev/null +++ b/tests/KDF/BCryptSpec.hs @@ -0,0 +1,175 @@ +{-# LANGUAGE OverloadedStrings #-} + +module KDF.BCryptSpec ( + spec, +) +where + +import Control.Exception (evaluate) +import Crypto.Error +import Crypto.KDF.BCrypt +import qualified Data.ByteString as B +import Imports + +-- Openwall bcrypt spec, with 2x versions and 0xFF special cases removed. +expected :: [(ByteString, ByteString)] +expected = + [ ("$2a$05$CCCCCCCCCCCCCCCCCCCCC.E5YPO9kmyuRGyh0XouQYb4YMJKvyOeW", "U*U") + , ("$2a$05$CCCCCCCCCCCCCCCCCCCCC.VGOzA784oUp/Z0DY336zx7pLYAy0lwK", "U*U*") + , ("$2a$05$XXXXXXXXXXXXXXXXXXXXXOAcXxm9kjPGEMsLznoKqmqw7tc8WCx4a", "U*U*U") + , + ( "$2a$05$abcdefghijklmnopqrstuu5s2v8.iXieOjg/.AySBTTZIIVFJeBui" + , "0123456789abcdefghijklmnopqrstuvwxyz\ + \ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789\ + \chars after 72 are ignored" + ) + , ("$2y$05$/OK.fbVrR/bpIqNJ5ianF.CE5elHaaO4EbggVDjb8P19RukzXSM3e", "\xff\xff\xa3") + , ("$2b$05$/OK.fbVrR/bpIqNJ5ianF.CE5elHaaO4EbggVDjb8P19RukzXSM3e", "\xff\xff\xa3") + , ("$2y$05$/OK.fbVrR/bpIqNJ5ianF.Sa7shbm4.OzKpvFnX1pQLmQW96oUlCq", "\xa3") + , ("$2a$05$/OK.fbVrR/bpIqNJ5ianF.Sa7shbm4.OzKpvFnX1pQLmQW96oUlCq", "\xa3") + , ("$2b$05$/OK.fbVrR/bpIqNJ5ianF.Sa7shbm4.OzKpvFnX1pQLmQW96oUlCq", "\xa3") + , + ( "$2a$05$/OK.fbVrR/bpIqNJ5ianF.swQOIzjOiJ9GHEPuhEkvqrUyvWhEMx6" + , "\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ + \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ + \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ + \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ + \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ + \\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\xaa\ + \chars after 72 are ignored as usual" + ) + , + ( "$2a$05$/OK.fbVrR/bpIqNJ5ianF.R9xrDjiycxMbQE2bp.vgqlYpW5wx2yy" + , "\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ + \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ + \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ + \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ + \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\ + \\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55\xaa\x55" + ) + , + ( "$2a$05$/OK.fbVrR/bpIqNJ5ianF.9tQZzcJfm3uj2NvJ/n5xkhpqLrMpWCe" + , "\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ + \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ + \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ + \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ + \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\ + \\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff\x55\xaa\xff" + ) + , ("$2a$05$CCCCCCCCCCCCCCCCCCCCC.7uG0VCzI2bS7j6ymqJi9CdcdxiRTWNy", "") + , ("$2a$06$DCq7YPn5Rq63x1Lad4cll.TV4S6ytwfsfvkgY8jIucDrjc8deX1s.", "") + , ("$2a$08$HqWuK6/Ng6sg9gQzbLrgb.Tl.ZHfXLhvt/SgVyWhQqgqcZ7ZuUtye", "") + , ("$2a$10$k1wbIrmNyFAPwPVPSVa/zecw2BCEnBwVS2GbrmgzxFUOqW9dk4TCW", "") + , ("$2a$12$k42ZFHFWqBp3vWli.nIn8uYyIkbvYRvodzbfbK18SSsY.CsIQPlxO", "") + , ("$2a$06$m0CrhHm10qJ3lXRY.5zDGO3rS2KdeeWLuGmsfGlMfOxih58VYVfxe", "a") + , ("$2a$08$cfcvVd2aQ8CMvoMpP2EBfeodLEkkFJ9umNEfPD18.hUF62qqlC/V.", "a") + , ("$2a$12$8NJH3LsPrANStV6XtBakCez0cKHXVxmvxIlcz785vxAIZrihHZpeS", "a") + , ("$2a$06$If6bvum7DFjUnE9p2uDeDu0YHzrHM6tf.iqN8.yx.jNN1ILEf7h0i", "abc") + , ("$2a$08$Ro0CUfOqk6cXEKf3dyaM7OhSCvnwM9s4wIX9JeLapehKK5YdLxKcm", "abc") + , ("$2a$10$WvvTPHKwdBJ3uk0Z37EMR.hLA2W6N9AEBhEgrAOljy2Ae5MtaSIUi", "abc") + , + ( "$2a$06$.rCVZVOThsIa97pEDOxvGuRRgzG64bvtJ0938xuqzv18d3ZpQhstC" + , "abcdefghijklmnopqrstuvwxyz" + ) + ] + +makeKATs = concatMap maketest (zip3 is passwords hashes) + where + is :: [Int] + is = [1 ..] + + passwords = map snd expected + hashes = map fst expected + + maketest (i, password, hash) = + [ it (show i) (assertBool "" (validatePassword password hash)) + ] + +spec :: Spec +spec = do + describe "KATs" $ sequence_ makeKATs + it + "Invalid hash length" + ( assertEqual + "" + (Left "Invalid hash format") + ( validatePasswordEither + B.empty + ("$2a$06$DCq7YPn5Rq63x1Lad4cll.TV4S6ytwfsfvkgY8jIucDrjc8deX1s" :: B.ByteString) + ) + ) + it + "Hash and validate" + ( assertBool + "Hashed password should validate" + (validatePassword somePassword (bcrypt 5 aSalt somePassword :: B.ByteString)) + ) + describe "salt length" $ do + it "rejects a salt shorter than 16 bytes" $ + evaluate (bcrypt (5 :: Int) (B.replicate 15 0x61) somePassword :: B.ByteString) + `shouldThrow` (== CryptoError_ParameterInvalid) + it "rejects a salt longer than 16 bytes" $ + evaluate (bcrypt (5 :: Int) (B.replicate 17 0x61) somePassword :: B.ByteString) + `shouldThrow` (== CryptoError_ParameterInvalid) + it "reports a wrong salt length without raising" $ + ( tryBcrypt (5 :: Int) (B.replicate 15 0x61) somePassword + :: CryptoFailable B.ByteString + ) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + describe "cost" $ do + -- What made the old behaviour wrong was not the floor but that it was + -- silent: a request for cost 3 came back as a cost-10 hash and a + -- request for cost 50 as a cost-31 one, with nothing said either way. + it "refuses a cost below the floor rather than substituting one" $ + [ c + | c <- [minBound, -1, 0, 1, 2, 3] + , tryBcrypt c aSalt somePassword + /= (CryptoFailed CryptoError_ParameterInvalid :: CryptoFailable B.ByteString) + ] + `shouldBe` [] + it "refuses a cost above the ceiling rather than substituting one" $ + [ c + | c <- [32, 33, 64, maxBound] + , tryBcrypt c aSalt somePassword + /= (CryptoFailed CryptoError_ParameterInvalid :: CryptoFailable B.ByteString) + ] + `shouldBe` [] + it "raises the same thing through bcrypt" $ + evaluate (bcrypt (3 :: Int) aSalt somePassword :: B.ByteString) + `shouldThrow` (== CryptoError_ParameterInvalid) + it "takes the bottom of the range" $ + validatePassword somePassword (bcrypt (4 :: Int) aSalt somePassword :: B.ByteString) + `shouldBe` True + it "writes the cost it was given, not another one" $ + B.take 7 (bcrypt (4 :: Int) aSalt somePassword :: B.ByteString) + `shouldBe` "$2b$04$" + it "reports it from hashPassword too, without raising" $ do + r <- tryHashPassword (3 :: Int) somePassword + (r :: CryptoFailable B.ByteString) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + it "hashes through hashPassword when the cost is one bcrypt takes" $ do + h <- hashPassword (4 :: Int) somePassword + validatePassword somePassword (h :: B.ByteString) `shouldBe` True + describe "password length limit" $ do + -- bcrypt keys Blowfish with at most the first 72 bytes of the + -- password, so everything after that is ignored. The Openwall + -- vectors above cover the hash value; these cover what it means for + -- a caller, which is what the haddock now documents. + it "ignores everything after the first 72 bytes" $ + bcrypt 5 aSalt longer `shouldBe` (bcrypt 5 aSalt otherTail :: B.ByteString) + it "accepts a password differing only past the 72nd byte" $ + validatePassword otherTail (bcrypt 5 aSalt longer :: B.ByteString) + `shouldBe` True + it "still separates passwords differing within the first 72 bytes" $ + validatePassword + (B.snoc (B.take 71 prefix72) 0x21) + (bcrypt 5 aSalt longer :: B.ByteString) + `shouldBe` False + where + prefix72 = B.replicate 72 0x61 + longer = prefix72 `B.append` "aaaaaaaaaaaaaaaaaaaa" + otherTail = prefix72 `B.append` "something else entirely" + somePassword = "some password" :: B.ByteString + aSalt = + "\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f" + :: B.ByteString diff --git a/tests/KDF/HKDFSpec.hs b/tests/KDF/HKDFSpec.hs new file mode 100644 --- /dev/null +++ b/tests/KDF/HKDFSpec.hs @@ -0,0 +1,411 @@ +{-# LANGUAGE OverloadedStrings #-} + +module KDF.HKDFSpec (spec) where + +import Control.Exception (evaluate) +import Crypto.Error (CryptoError (..), CryptoFailable (..)) +import Crypto.Hash (HashAlgorithm, SHA1, SHA256, SHA384, SHA512) +import qualified Crypto.KDF.HKDF as HKDF +import qualified Data.ByteString as B + +import Imports + +data KDFVector hash = KDFVector + { kdfIKM :: ByteString + , kdfSalt :: ByteString + , kdfInfo :: ByteString + , kdfResult :: ByteString + } + +sha256KDFVectors :: [KDFVector SHA256] +sha256KDFVectors = + [ KDFVector + "\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b\x0b" + ( B.pack + [0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07, 0x08, 0x09, 0x0a, 0x0b, 0x0c] + ) + "\xf0\xf1\xf2\xf3\xf4\xf5\xf6\xf7\xf8\xf9" + "\x3c\xb2\x5f\x25\xfa\xac\xd5\x7a\x90\x43\x4f\x64\xd0\x36\x2f\x2a\x2d\x2d\x0a\x90\xcf\x1a\x5a\x4c\x5d\xb0\x2d\x56\xec\xc4\xc5\xbf\x34\x00\x72\x08\xd5\xb8\x87\x18\x58\x65" + , KDFVector + ( B.pack + [ 0x00 + , 0x01 + , 0x02 + , 0x03 + , 0x04 + , 0x05 + , 0x06 + , 0x07 + , 0x08 + , 0x09 + , 0x0a + , 0x0b + , 0x0c + , 0x0d + , 0x0e + , 0x0f + , 0x10 + , 0x11 + , 0x12 + , 0x13 + , 0x14 + , 0x15 + , 0x16 + , 0x17 + , 0x18 + , 0x19 + , 0x1a + , 0x1b + , 0x1c + , 0x1d + , 0x1e + , 0x1f + , 0x20 + , 0x21 + , 0x22 + , 0x23 + , 0x24 + , 0x25 + , 0x26 + , 0x27 + , 0x28 + , 0x29 + , 0x2a + , 0x2b + , 0x2c + , 0x2d + , 0x2e + , 0x2f + , 0x30 + , 0x31 + , 0x32 + , 0x33 + , 0x34 + , 0x35 + , 0x36 + , 0x37 + , 0x38 + , 0x39 + , 0x3a + , 0x3b + , 0x3c + , 0x3d + , 0x3e + , 0x3f + , 0x40 + , 0x41 + , 0x42 + , 0x43 + , 0x44 + , 0x45 + , 0x46 + , 0x47 + , 0x48 + , 0x49 + , 0x4a + , 0x4b + , 0x4c + , 0x4d + , 0x4e + , 0x4f + ] + ) + ( B.pack + [ 0x60 + , 0x61 + , 0x62 + , 0x63 + , 0x64 + , 0x65 + , 0x66 + , 0x67 + , 0x68 + , 0x69 + , 0x6a + , 0x6b + , 0x6c + , 0x6d + , 0x6e + , 0x6f + , 0x70 + , 0x71 + , 0x72 + , 0x73 + , 0x74 + , 0x75 + , 0x76 + , 0x77 + , 0x78 + , 0x79 + , 0x7a + , 0x7b + , 0x7c + , 0x7d + , 0x7e + , 0x7f + , 0x80 + , 0x81 + , 0x82 + , 0x83 + , 0x84 + , 0x85 + , 0x86 + , 0x87 + , 0x88 + , 0x89 + , 0x8a + , 0x8b + , 0x8c + , 0x8d + , 0x8e + , 0x8f + , 0x90 + , 0x91 + , 0x92 + , 0x93 + , 0x94 + , 0x95 + , 0x96 + , 0x97 + , 0x98 + , 0x99 + , 0x9a + , 0x9b + , 0x9c + , 0x9d + , 0x9e + , 0x9f + , 0xa0 + , 0xa1 + , 0xa2 + , 0xa3 + , 0xa4 + , 0xa5 + , 0xa6 + , 0xa7 + , 0xa8 + , 0xa9 + , 0xaa + , 0xab + , 0xac + , 0xad + , 0xae + , 0xaf + ] + ) + ( B.pack + [ 0xb0 + , 0xb1 + , 0xb2 + , 0xb3 + , 0xb4 + , 0xb5 + , 0xb6 + , 0xb7 + , 0xb8 + , 0xb9 + , 0xba + , 0xbb + , 0xbc + , 0xbd + , 0xbe + , 0xbf + , 0xc0 + , 0xc1 + , 0xc2 + , 0xc3 + , 0xc4 + , 0xc5 + , 0xc6 + , 0xc7 + , 0xc8 + , 0xc9 + , 0xca + , 0xcb + , 0xcc + , 0xcd + , 0xce + , 0xcf + , 0xd0 + , 0xd1 + , 0xd2 + , 0xd3 + , 0xd4 + , 0xd5 + , 0xd6 + , 0xd7 + , 0xd8 + , 0xd9 + , 0xda + , 0xdb + , 0xdc + , 0xdd + , 0xde + , 0xdf + , 0xe0 + , 0xe1 + , 0xe2 + , 0xe3 + , 0xe4 + , 0xe5 + , 0xe6 + , 0xe7 + , 0xe8 + , 0xe9 + , 0xea + , 0xeb + , 0xec + , 0xed + , 0xee + , 0xef + , 0xf0 + , 0xf1 + , 0xf2 + , 0xf3 + , 0xf4 + , 0xf5 + , 0xf6 + , 0xf7 + , 0xf8 + , 0xf9 + , 0xfa + , 0xfb + , 0xfc + , 0xfd + , 0xfe + , 0xff + ] + ) + ( B.pack + [ 0xb1 + , 0x1e + , 0x39 + , 0x8d + , 0xc8 + , 0x03 + , 0x27 + , 0xa1 + , 0xc8 + , 0xe7 + , 0xf7 + , 0x8c + , 0x59 + , 0x6a + , 0x49 + , 0x34 + , 0x4f + , 0x01 + , 0x2e + , 0xda + , 0x2d + , 0x4e + , 0xfa + , 0xd8 + , 0xa0 + , 0x50 + , 0xcc + , 0x4c + , 0x19 + , 0xaf + , 0xa9 + , 0x7c + , 0x59 + , 0x04 + , 0x5a + , 0x99 + , 0xca + , 0xc7 + , 0x82 + , 0x72 + , 0x71 + , 0xcb + , 0x41 + , 0xc6 + , 0x5e + , 0x59 + , 0x0e + , 0x09 + , 0xda + , 0x32 + , 0x75 + , 0x60 + , 0x0c + , 0x2f + , 0x09 + , 0xb8 + , 0x36 + , 0x77 + , 0x93 + , 0xa9 + , 0xac + , 0xa3 + , 0xdb + , 0x71 + , 0xcc + , 0x30 + , 0xc5 + , 0x81 + , 0x79 + , 0xec + , 0x3e + , 0x87 + , 0xc1 + , 0x4c + , 0x01 + , 0xd5 + , 0xc1 + , 0xf3 + , 0x43 + , 0x4f + , 0x1d + , 0x87 + ] + ) + ] + +kdfTests :: [Spec] +kdfTests = + [ describe "sha256" $ mapM_ toKDFTest $ zip is sha256KDFVectors + ] + where + toKDFTest (i, kdfVector) = do + it (show i) (t HKDF.extract kdfVector) + + t + :: HashAlgorithm a + => (ByteString -> ByteString -> HKDF.PRK a) -> KDFVector a -> Expectation + t ext v = + let prk = ext (kdfSalt v) (kdfIKM v) + in HKDF.expand prk (kdfInfo v) (B.length $ kdfResult v) `shouldBe` kdfResult v + + is :: [Int] + is = [1 ..] + +boundTests :: [Spec] +boundTests = + [ boundTest "SHA-1" (HKDF.extract salt ikm :: HKDF.PRK SHA1) 20 + , boundTest "SHA-256" (HKDF.extract salt ikm :: HKDF.PRK SHA256) 32 + , boundTest "SHA-384" (HKDF.extract salt ikm :: HKDF.PRK SHA384) 48 + , boundTest "SHA-512" (HKDF.extract salt ikm :: HKDF.PRK SHA512) 64 + ] + where + salt = "salt" :: ByteString + ikm = "input key material" :: ByteString + info = "info" :: ByteString + boundTest name prk hashLen = + describe name $ do + it "maximum length" $ + B.length (HKDF.expand prk info maxLen :: ByteString) `shouldBe` maxLen + it "one byte past the maximum" $ + evaluate (B.length (HKDF.expand prk info (maxLen + 1) :: ByteString)) + `shouldThrow` (== CryptoError_OutputLengthTooBig) + it "reports one byte past the maximum without raising" $ + (HKDF.tryExpand prk info (maxLen + 1) :: CryptoFailable ByteString) + `shouldBe` CryptoFailed CryptoError_OutputLengthTooBig + where + maxLen = 255 * hashLen + +spec :: Spec +spec = do + describe "KATs" $ sequence_ kdfTests + describe "output bound" $ sequence_ boundTests diff --git a/tests/KDF/PBKDF2Spec.hs b/tests/KDF/PBKDF2Spec.hs new file mode 100644 --- /dev/null +++ b/tests/KDF/PBKDF2Spec.hs @@ -0,0 +1,156 @@ +{-# LANGUAGE OverloadedStrings #-} + +-- from +module KDF.PBKDF2Spec (spec) where + +import Control.Exception (evaluate) +import Crypto.Error +import Crypto.Hash (SHA1 (..), SHA256 (..), SHA512 (..)) +import qualified Crypto.KDF.PBKDF2 as PBKDF2 + +import Data.ByteString (ByteString) +import Data.ByteString.Char8 () + +import Test.Hspec + +type VectParams = (ByteString, ByteString, Int, Int) + +vectors_hmac_sha1 :: [(VectParams, ByteString)] +vectors_hmac_sha1 = + [ + ( ("password", "salt", 2, 20) + , "\xea\x6c\x01\x4d\xc7\x2d\x6f\x8c\xcd\x1e\xd9\x2a\xce\x1d\x41\xf0\xd8\xde\x89\x57" + ) + , + ( ("password", "salt", 4096, 20) + , "\x4b\x00\x79\x01\xb7\x65\x48\x9a\xbe\xad\x49\xd9\x26\xf7\x21\xd0\x65\xa4\x29\xc1" + ) + , + ( ("passwordPASSWORDpassword", "saltSALTsaltSALTsaltSALTsaltSALTsalt", 4096, 25) + , "\x3d\x2e\xec\x4f\xe4\x1c\x84\x9b\x80\xc8\xd8\x36\x62\xc0\xe4\x4a\x8b\x29\x1a\x96\x4c\xf2\xf0\x70\x38" + ) + , + ( ("pass\0word", "sa\0lt", 4096, 16) + , "\x56\xfa\x6a\xa7\x55\x48\x09\x9d\xcc\x37\xd7\xf0\x34\x25\xe0\xc3" + ) + ] + +vectors_hmac_sha256 :: [(VectParams, ByteString)] +vectors_hmac_sha256 = + [ + ( ("password", "salt", 2, 32) + , "\xae\x4d\x0c\x95\xaf\x6b\x46\xd3\x2d\x0a\xdf\xf9\x28\xf0\x6d\xd0\x2a\x30\x3f\x8e\xf3\xc2\x51\xdf\xd6\xe2\xd8\x5a\x95\x47\x4c\x43" + ) + , + ( ("passwordPASSWORDpassword", "saltSALTsaltSALTsaltSALTsaltSALTsalt", 4096, 40) + , "\x34\x8c\x89\xdb\xcb\xd3\x2b\x2f\x32\xd8\x14\xb8\x11\x6e\x84\xcf\x2b\x17\x34\x7e\xbc\x18\x00\x18\x1c\x4e\x2a\x1f\xb8\xdd\x53\xe1\xc6\x35\x51\x8c\x7d\xac\x47\xe9" + ) + ] + +vectors_hmac_sha512 :: [(VectParams, ByteString)] +vectors_hmac_sha512 = + [ + ( ("password", "salt", 1, 32) + , "\x86\x7f\x70\xcf\x1a\xde\x02\xcf\xf3\x75\x25\x99\xa3\xa5\x3d\xc4\xaf\x34\xc7\xa6\x69\x81\x5a\xe5\xd5\x13\x55\x4e\x1c\x8c\xf2\x52" + ) + , + ( ("password", "salt", 2, 32) + , "\xe1\xd9\xc1\x6a\xa6\x81\x70\x8a\x45\xf5\xc7\xc4\xe2\x15\xce\xb6\x6e\x01\x1a\x2e\x9f\x00\x40\x71\x3f\x18\xae\xfd\xb8\x66\xd5\x3c" + ) + , + ( ("password", "salt", 4096, 32) + , "\xd1\x97\xb1\xb3\x3d\xb0\x14\x3e\x01\x8b\x12\xf3\xd1\xd1\x47\x9e\x6c\xde\xbd\xcc\x97\xc5\xc0\xf8\x7f\x69\x02\xe0\x72\xf4\x57\xb5" + ) + , + ( ("passwordPASSWORDpassword", "saltSALTsaltSALTsaltSALTsaltSALTsalt", 1, 72) + , "n\x23\xf2\x76\x38\x08\x4b\x0f\x7e\xa1\x73\x4e\x0d\x98\x41\xf5\x5d\xd2\x9e\xa6\x0a\x83\x44\x66\xf3\x39\x6b\xac\x80\x1f\xac\x1e\xeb\x63\x80\x2f\x03\xa0\xb4\xac\xd7\x60\x3e\x36\x99\xc8\xb7\x44\x37\xbe\x83\xff\x01\xad\x7f\x55\xda\xc1\xef\x60\xf4\xd5\x64\x80\xc3\x5e\xe6\x8f\xd5\x2c\x69\x36" + ) + ] + +spec :: Spec +spec = do + describe "KATs-HMAC-SHA1" $ + sequence_ (katTests (PBKDF2.prfHMAC SHA1) vectors_hmac_sha1) + describe "KATs-HMAC-SHA1 (fast)" $ + sequence_ (katTestFastPBKDF2_SHA1 vectors_hmac_sha1) + describe "KATs-HMAC-SHA256" $ + sequence_ $ + (katTests (PBKDF2.prfHMAC SHA256) vectors_hmac_sha256) + describe "KATs-HMAC-SHA256 (fast)" $ + sequence_ $ + (katTestFastPBKDF2_SHA256 vectors_hmac_sha256) + describe "KATs-HMAC-SHA512" $ + sequence_ $ + (katTests (PBKDF2.prfHMAC SHA512) vectors_hmac_sha512) + describe "KATs-HMAC-SHA512 (fast)" $ + sequence_ $ + (katTestFastPBKDF2_SHA512 vectors_hmac_sha512) + describe "invalid parameters" $ do + -- A zero iteration count derives the zero key rather than a key, and + -- a negative output length used to ask memSet for a buffer of -1 + -- bytes, which took the process down with it. + it "rejects an iteration count below one" $ + evaluate (slow (PBKDF2.Parameters 0 32)) `shouldThrow` cryptoError + it "rejects a negative output length" $ + evaluate (slow (PBKDF2.Parameters 1 (-1))) `shouldThrow` cryptoError + it "rejects an iteration count below one in the fast path" $ do + evaluate (fast1 (PBKDF2.Parameters 0 32)) `shouldThrow` cryptoError + evaluate (fast256 (PBKDF2.Parameters 0 32)) `shouldThrow` cryptoError + evaluate (fast512 (PBKDF2.Parameters 0 32)) `shouldThrow` cryptoError + it "rejects a negative output length in the fast path" $ do + evaluate (fast1 (PBKDF2.Parameters 1 (-1))) `shouldThrow` cryptoError + evaluate (fast256 (PBKDF2.Parameters 1 (-1))) `shouldThrow` cryptoError + evaluate (fast512 (PBKDF2.Parameters 1 (-1))) `shouldThrow` cryptoError + it "reports them without raising" $ do + PBKDF2.tryGenerate badPrf (PBKDF2.Parameters 0 32) badPass badSalt + `shouldBe` refused + PBKDF2.tryFastPBKDF2_SHA1 (PBKDF2.Parameters 1 (-1)) badPass badSalt + `shouldBe` refused + PBKDF2.tryFastPBKDF2_SHA256 (PBKDF2.Parameters 0 32) badPass badSalt + `shouldBe` refused + PBKDF2.tryFastPBKDF2_SHA512 (PBKDF2.Parameters 1 (-1)) badPass badSalt + `shouldBe` refused + where + badPrf = PBKDF2.prfHMAC SHA256 + badPass = "password" :: ByteString + badSalt = "salt" :: ByteString + refused = CryptoFailed CryptoError_ParameterInvalid :: CryptoFailable ByteString + cryptoError e = e == CryptoError_ParameterInvalid + slow params = PBKDF2.generate badPrf params badPass badSalt :: ByteString + fast1 params = PBKDF2.fastPBKDF2_SHA1 params badPass badSalt :: ByteString + fast256 params = PBKDF2.fastPBKDF2_SHA256 params badPass badSalt :: ByteString + fast512 params = PBKDF2.fastPBKDF2_SHA512 params badPass badSalt :: ByteString + + katTests prf = zipWith (toKatTest prf) is + + toKatTest prf i ((pass, salt, iter, dkLen), output) = + it + (show i) + (PBKDF2.generate prf (PBKDF2.Parameters iter dkLen) pass salt `shouldBe` output) + + katTestFastPBKDF2_SHA1 = zipWith toKatTestFastPBKDF2_SHA1 is + toKatTestFastPBKDF2_SHA1 i ((pass, salt, iter, dkLen), output) = + it + (show i) + ( PBKDF2.fastPBKDF2_SHA1 (PBKDF2.Parameters iter dkLen) pass salt + `shouldBe` output + ) + + katTestFastPBKDF2_SHA256 = zipWith toKatTestFastPBKDF2_SHA256 is + toKatTestFastPBKDF2_SHA256 i ((pass, salt, iter, dkLen), output) = + it + (show i) + ( PBKDF2.fastPBKDF2_SHA256 (PBKDF2.Parameters iter dkLen) pass salt + `shouldBe` output + ) + + katTestFastPBKDF2_SHA512 = zipWith toKatTestFastPBKDF2_SHA512 is + toKatTestFastPBKDF2_SHA512 i ((pass, salt, iter, dkLen), output) = + it + (show i) + ( PBKDF2.fastPBKDF2_SHA512 (PBKDF2.Parameters iter dkLen) pass salt + `shouldBe` output + ) + + is :: [Int] + is = [1 ..] diff --git a/tests/KDF/ScryptSpec.hs b/tests/KDF/ScryptSpec.hs new file mode 100644 --- /dev/null +++ b/tests/KDF/ScryptSpec.hs @@ -0,0 +1,59 @@ +{-# LANGUAGE OverloadedStrings #-} + +module KDF.ScryptSpec (spec) where + +import Data.ByteString (ByteString) +import Data.ByteString.Char8 () + +import Control.Exception (evaluate) +import Data.Word +import Test.Hspec + +import Crypto.Error +import qualified Crypto.KDF.Scrypt as Scrypt + +vectors :: [((ByteString, ByteString, Word64, Int, Int, Int), ByteString)] +vectors = + [ + ( ("", "", 16, 1, 1, 64) + , "\x77\xd6\x57\x62\x38\x65\x7b\x20\x3b\x19\xca\x42\xc1\x8a\x04\x97\xf1\x6b\x48\x44\xe3\x07\x4a\xe8\xdf\xdf\xfa\x3f\xed\xe2\x14\x42\xfc\xd0\x06\x9d\xed\x09\x48\xf8\x32\x6a\x75\x3a\x0f\xc8\x1f\x17\xe8\xd3\xe0\xfb\x2e\x0d\x36\x28\xcf\x35\xe2\x0c\x38\xd1\x89\x06" + ) + , + ( ("password", "NaCl", 1024, 8, 16, 64) + , "\xfd\xba\xbe\x1c\x9d\x34\x72\x00\x78\x56\xe7\x19\x0d\x01\xe9\xfe\x7c\x6a\xd7\xcb\xc8\x23\x78\x30\xe7\x73\x76\x63\x4b\x37\x31\x62\x2e\xaf\x30\xd9\x2e\x22\xa3\x88\x6f\xf1\x09\x27\x9d\x98\x30\xda\xc7\x27\xaf\xb9\x4a\x83\xee\x6d\x83\x60\xcb\xdf\xa2\xcc\x06\x40" + ) + , + ( ("pleaseletmein", "SodiumChloride", 16384, 8, 1, 64) + , "\x70\x23\xbd\xcb\x3a\xfd\x73\x48\x46\x1c\x06\xcd\x81\xfd\x38\xeb\xfd\xa8\xfb\xba\x90\x4f\x8e\x3e\xa9\xb5\x43\xf6\x54\x5d\xa1\xf2\xd5\x43\x29\x55\x61\x3f\x0f\xcf\x62\xd4\x97\x05\x24\x2a\x9a\xf9\xe6\x1e\x85\xdc\x0d\x65\x1e\x40\xdf\xcf\x01\x7b\x45\x57\x58\x87" + ) + , + ( ("pleaseletmein", "SodiumChloride", 1048576, 8, 1, 64) + , "\x21\x01\xcb\x9b\x6a\x51\x1a\xae\xad\xdb\xbe\x09\xcf\x70\xf8\x81\xec\x56\x8d\x57\x4a\x2f\xfd\x4d\xab\xe5\xee\x98\x20\xad\xaa\x47\x8e\x56\xfd\x8f\x4b\xa5\xd0\x9f\xfa\x1c\x6d\x92\x7c\x40\xf4\xc3\x37\x30\x40\x49\xe8\xa9\x52\xfb\xcb\xf4\x5c\x6f\xa7\x7a\x41\xa4" + ) + ] + +spec :: Spec +spec = do + sequence_ $ zipWith toCase [(1 :: Int) ..] vectors + describe "invalid parameters" $ do + it "rejects an n that is not a power of two" $ + evaluate (run (Scrypt.Parameters 3 8 1 32)) `shouldThrow` cryptoError + it "rejects an r and p that overflow" $ + evaluate (run (Scrypt.Parameters 16 1073741824 1 32)) + `shouldThrow` cryptoError + it "reports them without raising" $ do + run' (Scrypt.Parameters 3 8 1 32) `shouldBe` refused + run' (Scrypt.Parameters 16 1073741824 1 32) `shouldBe` refused + where + run params = + Scrypt.generate params ("password" :: ByteString) ("salt" :: ByteString) + :: ByteString + run' params = + Scrypt.tryGenerate params ("password" :: ByteString) ("salt" :: ByteString) + :: CryptoFailable ByteString + refused = CryptoFailed CryptoError_ParameterInvalid + cryptoError e = e == CryptoError_ParameterInvalid + toCase i ((pass, salt, n, r, p, dklen), output) = + it + (show i) + (Scrypt.generate (Scrypt.Parameters n r p dklen) pass salt `shouldBe` output) diff --git a/tests/MAC/Blake2Spec.hs b/tests/MAC/Blake2Spec.hs new file mode 100644 --- /dev/null +++ b/tests/MAC/Blake2Spec.hs @@ -0,0 +1,198 @@ +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE RecordWildCards #-} + +module MAC.Blake2Spec (spec) where + +import Crypto.Hash (digestFromByteString) +import Crypto.Hash.Algorithms +import qualified Crypto.MAC.KeyedBlake2 as KB + +import qualified Data.ByteString as B + +import Imports + +data MACVector hash = MACVector + { macMessage :: ByteString + , macKey :: ByteString + , macResult :: KB.KeyedBlake2 hash + } + +instance Show (KB.KeyedBlake2 hash) where + show (KB.KeyedBlake2 d) = show d + +digest :: KB.HashBlake2 hash => ByteString -> KB.KeyedBlake2 hash +digest = maybe (error "cannot get digest") KB.KeyedBlake2 . digestFromByteString + +-- From: https://github.com/BLAKE2/BLAKE2/blob/master/testvectors/ +vectorsBlake2bKAT :: [MACVector (Blake2b 512)] +vectorsBlake2bKAT = + [ MACVector + { macMessage = "" + , macKey = fixedKey + , macResult = + digest + "\x10\xeb\xb6\x77\x00\xb1\x86\x8e\xfb\x44\x17\x98\x7a\xcf\x46\x90\xae\x9d\x97\x2f\xb7\xa5\x90\xc2\xf0\x28\x71\x79\x9a\xaa\x47\x86\xb5\xe9\x96\xe8\xf0\xf4\xeb\x98\x1f\xc2\x14\xb0\x05\xf4\x2d\x2f\xf4\x23\x34\x99\x39\x16\x53\xdf\x7a\xef\xcb\xc1\x3f\xc5\x15\x68" + } + , MACVector + { macMessage = "\x00" + , macKey = fixedKey + , macResult = + digest + "\x96\x1f\x6d\xd1\xe4\xdd\x30\xf6\x39\x01\x69\x0c\x51\x2e\x78\xe4\xb4\x5e\x47\x42\xed\x19\x7c\x3c\x5e\x45\xc5\x49\xfd\x25\xf2\xe4\x18\x7b\x0b\xc9\xfe\x30\x49\x2b\x16\xb0\xd0\xbc\x4e\xf9\xb0\xf3\x4c\x70\x03\xfa\xc0\x9a\x5e\xf1\x53\x2e\x69\x43\x02\x34\xce\xbd" + } + , MACVector + { macMessage = B.pack [0x00 .. 0xfe] + , macKey = fixedKey + , macResult = + digest + "\x14\x27\x09\xd6\x2e\x28\xfc\xcc\xd0\xaf\x97\xfa\xd0\xf8\x46\x5b\x97\x1e\x82\x20\x1d\xc5\x10\x70\xfa\xa0\x37\x2a\xa4\x3e\x92\x48\x4b\xe1\xc1\xe7\x3b\xa1\x09\x06\xd5\xd1\x85\x3d\xb6\xa4\x10\x6e\x0a\x7b\xf9\x80\x0d\x37\x3d\x6d\xee\x2d\x46\xd6\x2e\xf2\xa4\x61" + } + ] + where + fixedKey = B.pack [0x00 .. 0x3f] + +vectorsBlake2bpKAT :: [MACVector (Blake2bp 512)] +vectorsBlake2bpKAT = + [ MACVector + { macMessage = "" + , macKey = fixedKey + , macResult = + digest + "\x9d\x94\x61\x07\x3e\x4e\xb6\x40\xa2\x55\x35\x7b\x83\x9f\x39\x4b\x83\x8c\x6f\xf5\x7c\x9b\x68\x6a\x3f\x76\x10\x7c\x10\x66\x72\x8f\x3c\x99\x56\xbd\x78\x5c\xbc\x3b\xf7\x9d\xc2\xab\x57\x8c\x5a\x0c\x06\x3b\x9d\x9c\x40\x58\x48\xde\x1d\xbe\x82\x1c\xd0\x5c\x94\x0a" + } + , MACVector + { macMessage = "\x00" + , macKey = fixedKey + , macResult = + digest + "\xff\x8e\x90\xa3\x7b\x94\x62\x39\x32\xc5\x9f\x75\x59\xf2\x60\x35\x02\x9c\x37\x67\x32\xcb\x14\xd4\x16\x02\x00\x1c\xbb\x73\xad\xb7\x92\x93\xa2\xdb\xda\x5f\x60\x70\x30\x25\x14\x4d\x15\x8e\x27\x35\x52\x95\x96\x25\x1c\x73\xc0\x34\x5c\xa6\xfc\xcb\x1f\xb1\xe9\x7e" + } + , MACVector + { macMessage = B.pack [0x00 .. 0xfe] + , macKey = fixedKey + , macResult = + digest + "\x96\xfb\xcb\xb6\x0b\xd3\x13\xb8\x84\x50\x33\xe5\xbc\x05\x8a\x38\x02\x74\x38\x57\x2d\x7e\x79\x57\xf3\x68\x4f\x62\x68\xaa\xdd\x3a\xd0\x8d\x21\x76\x7e\xd6\x87\x86\x85\x33\x1b\xa9\x85\x71\x48\x7e\x12\x47\x0a\xad\x66\x93\x26\x71\x6e\x46\x66\x7f\x69\xf8\xd7\xe8" + } + ] + where + fixedKey = B.pack [0x00 .. 0x3f] + +vectorsBlake2sKAT :: [MACVector (Blake2s 256)] +vectorsBlake2sKAT = + [ MACVector + { macMessage = "" + , macKey = fixedKey + , macResult = + digest + "\x48\xa8\x99\x7d\xa4\x07\x87\x6b\x3d\x79\xc0\xd9\x23\x25\xad\x3b\x89\xcb\xb7\x54\xd8\x6a\xb7\x1a\xee\x04\x7a\xd3\x45\xfd\x2c\x49" + } + , MACVector + { macMessage = "\x00" + , macKey = fixedKey + , macResult = + digest + "\x40\xd1\x5f\xee\x7c\x32\x88\x30\x16\x6a\xc3\xf9\x18\x65\x0f\x80\x7e\x7e\x01\xe1\x77\x25\x8c\xdc\x0a\x39\xb1\x1f\x59\x80\x66\xf1" + } + , MACVector + { macMessage = B.pack [0x00 .. 0xfe] + , macKey = fixedKey + , macResult = + digest + "\x3f\xb7\x35\x06\x1a\xbc\x51\x9d\xfe\x97\x9e\x54\xc1\xee\x5b\xfa\xd0\xa9\xd8\x58\xb3\x31\x5b\xad\x34\xbd\xe9\x99\xef\xd7\x24\xdd" + } + ] + where + fixedKey = B.pack [0x00 .. 0x1f] + +vectorsBlake2spKAT :: [MACVector (Blake2sp 256)] +vectorsBlake2spKAT = + [ MACVector + { macMessage = "" + , macKey = fixedKey + , macResult = + digest + "\x71\x5c\xb1\x38\x95\xae\xb6\x78\xf6\x12\x41\x60\xbf\xf2\x14\x65\xb3\x0f\x4f\x68\x74\x19\x3f\xc8\x51\xb4\x62\x10\x43\xf0\x9c\xc6" + } + , MACVector + { macMessage = "\x00" + , macKey = fixedKey + , macResult = + digest + "\x40\x57\x8f\xfa\x52\xbf\x51\xae\x18\x66\xf4\x28\x4d\x3a\x15\x7f\xc1\xbc\xd3\x6a\xc1\x3c\xbd\xcb\x03\x77\xe4\xd0\xcd\x0b\x66\x03" + } + , MACVector + { macMessage = B.pack [0x00 .. 0xfe] + , macKey = fixedKey + , macResult = + digest + "\x0c\x8a\x36\x59\x7d\x74\x61\xc6\x3a\x94\x73\x28\x21\xc9\x41\x85\x6c\x66\x83\x76\x60\x6c\x86\xa5\x2d\xe0\xee\x41\x04\xc6\x15\xdb" + } + ] + where + fixedKey = B.pack [0x00 .. 0x1f] + +macTests :: [Spec] +macTests = + [ describe "Blake2b_512" $ mapM_ toMACTest $ zip is vectorsBlake2bKAT + , describe "Blake2bp_512" $ mapM_ toMACTest $ zip is vectorsBlake2bpKAT + , describe "Blake2s_512" $ mapM_ toMACTest $ zip is vectorsBlake2sKAT + , describe "Blake2sp_512" $ mapM_ toMACTest $ zip is vectorsBlake2spKAT + ] + where + toMACTest (i, MACVector{..}) = do + it (show i) (KB.keyedBlake2 macKey macMessage `shouldBe` macResult) + it + ("incr-" ++ show i) + ( KB.finalize (KB.update (KB.initialize macKey) macMessage) + `shouldBe` macResult + ) + is :: [Int] + is = [1 ..] + +data MacIncremental a = MacIncremental ByteString ByteString (KB.KeyedBlake2 a) + deriving (Show, Eq) + +instance KB.HashBlake2 a => Arbitrary (MacIncremental a) where + arbitrary = do + key <- arbitraryBSof 32 64 + msg <- arbitraryBSof 1 99 + return $ MacIncremental key msg (KB.keyedBlake2 key msg) + +data MacIncrementalList a + = MacIncrementalList ByteString [ByteString] (KB.KeyedBlake2 a) + deriving (Show, Eq) + +instance KB.HashBlake2 a => Arbitrary (MacIncrementalList a) where + arbitrary = do + key <- arbitraryBSof 32 64 + msgs <- choose (1, 20) >>= \n -> replicateM n (arbitraryBSof 1 99) + return $ MacIncrementalList key msgs (KB.keyedBlake2 key (B.concat msgs)) + +macIncrementalTests :: [Spec] +macIncrementalTests = + [ testIncrProperties "Blake2b_512" (Blake2b :: Blake2b 512) + , testIncrProperties "Blake2bp_512" (Blake2bp :: Blake2bp 512) + , testIncrProperties "Blake2s_256" (Blake2s :: Blake2s 256) + , testIncrProperties "Blake2sp_256" (Blake2sp :: Blake2sp 256) + ] + where + testIncrProperties :: KB.HashBlake2 a => String -> a -> Spec + testIncrProperties name a = + describe name $ do + prop "list-one" (prop_inc0 a) + prop "list-multi" (prop_inc1 a) + + prop_inc0 :: KB.HashBlake2 a => a -> MacIncremental a -> Bool + prop_inc0 _ (MacIncremental secret msg result) = + result `assertEq` KB.finalize (KB.update (KB.initialize secret) msg) + + prop_inc1 :: KB.HashBlake2 a => a -> MacIncrementalList a -> Bool + prop_inc1 _ (MacIncrementalList secret msgs result) = + result `assertEq` KB.finalize (foldl' KB.update (KB.initialize secret) msgs) + +spec :: Spec +spec = do + describe "KATs" $ sequence_ macTests + describe "properties" $ sequence_ macIncrementalTests diff --git a/tests/MAC/CMACSpec.hs b/tests/MAC/CMACSpec.hs new file mode 100644 --- /dev/null +++ b/tests/MAC/CMACSpec.hs @@ -0,0 +1,258 @@ +module MAC.CMACSpec (spec) where + +import Crypto.Cipher.AES (AES128, AES192, AES256) +import Crypto.Cipher.TripleDES (DES_EDE2, DES_EDE3) +import Crypto.Cipher.Types ( + BlockCipher, + Cipher, + blockSize, + cipherInit, + ecbEncrypt, + ) +import Crypto.Error (eitherCryptoError) +import qualified Crypto.MAC.CMAC as CMAC + +import Imports + +import Data.Bits (xor) +import qualified Data.ByteArray as B +import qualified Data.ByteString as BS +import Data.Char (digitToInt) + +hxs :: String -> ByteString +hxs = BS.pack . rec' + where + dtoW8 = fromIntegral . digitToInt + rec' (' ' : xs) = rec' xs + rec' (x : y : xs) = dtoW8 x * 16 + dtoW8 y : rec' xs + rec' [_] = error "hxs: invalid hex pattern." + rec' [] = [] + +unsafeCipher :: Cipher k => ByteString -> k +unsafeCipher = either (error . show) id . eitherCryptoError . cipherInit + +ecb0 :: BlockCipher k => k -> ByteString +ecb0 k = ecbEncrypt k $ BS.replicate (blockSize k) 0 + +{- Test vectors from NIST data-sheet + (AES128-CMAC, AES192-CMAC, AES256-CMAC, Three Key TDEA, Two Key TDEA) + http://csrc.nist.gov/publications/nistpubs/800-38B/Updated_CMAC_Examples.pdf + The data of AES128-CMAC is same as them in RFC4493. + -} + +msg512 :: ByteString +msg512 = + hxs $ + "6bc1bee2 2e409f96 e93d7e11 7393172a" + ++ "ae2d8a57 1e03ac9c 9eb76fac 45af8e51" + ++ "30c81c46 a35ce411 e5fbc119 1a0a52ef" + ++ "f69f2445 df4f9b17 ad2b417b e66c3710" + +msg320 :: ByteString +msg320 = BS.take 40 msg512 + +msg256 :: ByteString +msg256 = BS.take 32 msg512 + +msg160 :: ByteString +msg160 = BS.take 20 msg512 + +msg128 :: ByteString +msg128 = BS.take 16 msg512 + +msg64 :: ByteString +msg64 = BS.take 8 msg512 + +msg0 :: ByteString +msg0 = BS.empty + +bsCMAC :: BlockCipher k => k -> ByteString -> ByteString +bsCMAC k = B.convert . CMAC.cmac k + +-- | CMAC as RFC 4493 section 2.4 states it, written out here so that the +-- implementation has something to be compared against at lengths the NIST +-- vectors do not cover: the message is split into blocks, the last one is +-- exclusive-ored with the first subkey when it is full and padded and +-- exclusive-ored with the second when it is not, and the blocks are chained +-- through the cipher from a block of zeroes. +refCMAC :: BlockCipher k => k -> ByteString -> ByteString +refCMAC k msg = foldl step (BS.replicate bsz 0) (blocks msg) + where + bsz = blockSize k + (k1, k2) = CMAC.subKeys k + step c m = ecbEncrypt k (bxor c m) + blocks m + | BS.length m <= bsz = [lastBlock m] + | otherwise = BS.take bsz m : blocks (BS.drop bsz m) + lastBlock m + | BS.length m == bsz = bxor k1 m + | otherwise = + bxor k2 $ + BS.concat + [m, BS.singleton 0x80, BS.replicate (bsz - BS.length m - 1) 0] + bxor a b = BS.pack (BS.zipWith xor a b) + +-- | The lengths around a block boundary, and one message long enough that a +-- decision made once per block is repeated thousands of times. +lengthTests :: Spec +lengthTests = + describe "message lengths" $ do + it "agrees with the definition at every length from 0 to 80" $ + [ n + | n <- [0 .. 80] + , let m = BS.take n (BS.concat [msg512, msg512]) + , bsCMAC key m /= refCMAC key m + ] + `shouldBe` [] + it "agrees with the definition on a message of 256 KiB" $ + bsCMAC key big `shouldBe` refCMAC key big + where + key :: AES128 + key = unsafeCipher $ hxs "2b7e1516 28aed2a6 abf71588 09cf4f3c" + big = BS.concat (replicate 4096 msg512) + +gAES128 :: Spec +gAES128 = + igroup + "aes128" + [ ecb0 aes128key `shouldBe` hxs "7df76b0c 1ab899b3 3e42f047 b91b546f" + , aes128k1 `shouldBe` hxs "fbeed618 35713366 7c85e08f 7236a8de" + , aes128k2 `shouldBe` hxs "f7ddac30 6ae266cc f90bc11e e46d513b" + , bsCMAC aes128key msg0 + `shouldBe` hxs "bb1d6929 e9593728 7fa37d12 9b756746" + , bsCMAC aes128key msg128 + `shouldBe` hxs "070a16b4 6b4d4144 f79bdd9d d04a287c" + , bsCMAC aes128key msg320 + `shouldBe` hxs "dfa66747 de9ae630 30ca3261 1497c827" + , bsCMAC aes128key msg512 + `shouldBe` hxs "51f0bebf 7e3b9d92 fc497417 79363cfe" + ] + where + aes128key :: AES128 + aes128key = + unsafeCipher $ + hxs + "2b7e1516 28aed2a6 abf71588 09cf4f3c" + + aes128k1, aes128k2 :: ByteString + (aes128k1, aes128k2) = CMAC.subKeys aes128key + +gAES192 :: Spec +gAES192 = + igroup + "aes192" + [ ecb0 aes192key `shouldBe` hxs "22452d8e 49a8a593 9f7321ce ea6d514b" + , aes192k1 `shouldBe` hxs "448a5b1c 93514b27 3ee6439d d4daa296" + , aes192k2 `shouldBe` hxs "8914b639 26a2964e 7dcc873b a9b5452c" + , bsCMAC aes192key msg0 + `shouldBe` hxs "d17ddf46 adaacde5 31cac483 de7a9367" + , bsCMAC aes192key msg128 + `shouldBe` hxs "9e99a7bf 31e71090 0662f65e 617c5184" + , bsCMAC aes192key msg320 + `shouldBe` hxs "8a1de5be 2eb31aad 089a82e6 ee908b0e" + , bsCMAC aes192key msg512 + `shouldBe` hxs "a1d5df0e ed790f79 4d775896 59f39a11" + ] + where + aes192key :: AES192 + aes192key = + unsafeCipher . hxs $ + "8e73b0f7 da0e6452 c810f32b 809079e5" + ++ "62f8ead2 522c6b7b" + + aes192k1, aes192k2 :: ByteString + (aes192k1, aes192k2) = CMAC.subKeys aes192key + +gAES256 :: Spec +gAES256 = + igroup + "aes256" + [ ecb0 aes256key `shouldBe` hxs "e568f681 94cf76d6 174d4cc0 4310a854" + , aes256k1 `shouldBe` hxs "cad1ed03 299eedac 2e9a9980 8621502f" + , aes256k2 `shouldBe` hxs "95a3da06 533ddb58 5d353301 0c42a0d9" + , bsCMAC aes256key msg0 + `shouldBe` hxs "028962f6 1b7bf89e fc6b551f 4667d983" + , bsCMAC aes256key msg128 + `shouldBe` hxs "28a7023f 452e8f82 bd4bf28d 8c37c35c" + , bsCMAC aes256key msg320 + `shouldBe` hxs "aaf3d8f1 de5640c2 32f5b169 b9c911e6" + , bsCMAC aes256key msg512 + `shouldBe` hxs "e1992190 549f6ed5 696a2c05 6c315410" + ] + where + aes256key :: AES256 + aes256key = + unsafeCipher . hxs $ + "603deb10 15ca71be 2b73aef0 857d7781" + ++ "1f352c07 3b6108d7 2d9810a3 0914dff4" + + aes256k1, aes256k2 :: ByteString + (aes256k1, aes256k2) = CMAC.subKeys aes256key + +gTDEA3 :: Spec +gTDEA3 = + igroup + "Three Key TDEA" + [ ecb0 tdea3key `shouldBe` hxs "c8cc74e9 8a7329a2" + , tdea3k1 `shouldBe` hxs "9198e9d3 14e6535f" + , tdea3k2 `shouldBe` hxs "2331d3a6 29cca6a5" + , bsCMAC tdea3key msg0 + `shouldBe` hxs "b7a688e1 22ffaf95" + , bsCMAC tdea3key msg64 + `shouldBe` hxs "8e8f2931 36283797" + , bsCMAC tdea3key msg160 + `shouldBe` hxs "743ddbe0 ce2dc2ed" + , bsCMAC tdea3key msg256 + `shouldBe` hxs "33e6b109 2400eae5" + ] + where + tdea3key :: DES_EDE3 + tdea3key = + unsafeCipher . hxs $ + "8aa83bf8 cbda1062" + ++ "0bc1bf19 fbb6cd58" + ++ "bc313d4a 371ca8b5" + + tdea3k1, tdea3k2 :: ByteString + (tdea3k1, tdea3k2) = CMAC.subKeys tdea3key + +gTDEA2 :: Spec +gTDEA2 = + igroup + "Two Key TDEA" + [ ecb0 tdea2key `shouldBe` hxs "c7679b9f 6b8d7d7a" + , tdea2k1 `shouldBe` hxs "8ecf373e d71afaef" + , tdea2k2 `shouldBe` hxs "1d9e6e7d ae35f5c5" + , bsCMAC tdea2key msg0 + `shouldBe` hxs "bd2ebf9a 3ba00361" + , bsCMAC tdea2key msg64 + `shouldBe` hxs "4ff2ab81 3c53ce83" + , bsCMAC tdea2key msg160 + `shouldBe` hxs "62dd1b47 1902bd4e" + , bsCMAC tdea2key msg256 + `shouldBe` hxs "31b1e431 dabc4eb8" + ] + where + tdea2key :: DES_EDE2 + tdea2key = + unsafeCipher . hxs $ + "4cf15134 a2850dd5" + ++ "8a3d10ba 80570d38" + + tdea2k1, tdea2k2 :: ByteString + (tdea2k1, tdea2k2) = CMAC.subKeys tdea2key + +igroup :: String -> [Expectation] -> Spec +igroup nm = describe nm . sequence_ . zipWith (flip ($)) [1 ..] . map icase + where + icase c i = it (show (i :: Int)) c + +nistVectors :: Spec +nistVectors = + describe "KAT - NIST test vectors" $ do + sequence_ [gAES128, gAES192, gAES256, gTDEA3, gTDEA2] + +spec :: Spec +spec = do + nistVectors + lengthTests diff --git a/tests/MAC/HMACSpec.hs b/tests/MAC/HMACSpec.hs new file mode 100644 --- /dev/null +++ b/tests/MAC/HMACSpec.hs @@ -0,0 +1,209 @@ +{-# LANGUAGE OverloadedStrings #-} + +module MAC.HMACSpec (spec) where + +import Crypto.Hash ( + HashAlgorithm, + Keccak_224 (..), + Keccak_256 (..), + Keccak_384 (..), + Keccak_512 (..), + MD5 (..), + SHA1 (..), + SHA256 (..), + SHA3_224 (..), + SHA3_256 (..), + SHA3_384 (..), + SHA3_512 (..), + digestFromByteString, + ) +import qualified Crypto.MAC.HMAC as HMAC +import qualified Data.ByteString as B + +import Imports + +data MACVector hash = MACVector + { macKey :: ByteString + , macSecret :: ByteString + , macResult :: HMAC.HMAC hash + } + +instance Show (HMAC.HMAC a) where + show (HMAC.HMAC d) = show d + +digest :: HashAlgorithm hash => ByteString -> HMAC.HMAC hash +digest = maybe (error "cannot get digest") HMAC.HMAC . digestFromByteString + +v1 :: ByteString +v1 = "The quick brown fox jumps over the lazy dog" + +md5MACVectors :: [MACVector MD5] +md5MACVectors = + [ MACVector B.empty B.empty $ + digest "\x74\xe6\xf7\x29\x8a\x9c\x2d\x16\x89\x35\xf5\x8c\x00\x1b\xad\x88" + , MACVector "key" v1 $ + digest "\x80\x07\x07\x13\x46\x3e\x77\x49\xb9\x0c\x2d\xc2\x49\x11\xe2\x75" + ] + +sha1MACVectors :: [MACVector SHA1] +sha1MACVectors = + [ MACVector B.empty B.empty $ + digest + "\xfb\xdb\x1d\x1b\x18\xaa\x6c\x08\x32\x4b\x7d\x64\xb7\x1f\xb7\x63\x70\x69\x0e\x1d" + , MACVector "key" v1 $ + digest + "\xde\x7c\x9b\x85\xb8\xb7\x8a\xa6\xbc\x8a\x7a\x36\xf7\x0a\x90\x70\x1c\x9d\xb4\xd9" + ] + +sha256MACVectors :: [MACVector SHA256] +sha256MACVectors = + [ MACVector B.empty B.empty $ + digest + "\xb6\x13\x67\x9a\x08\x14\xd9\xec\x77\x2f\x95\xd7\x78\xc3\x5f\xc5\xff\x16\x97\xc4\x93\x71\x56\x53\xc6\xc7\x12\x14\x42\x92\xc5\xad" + , MACVector "key" v1 $ + digest + "\xf7\xbc\x83\xf4\x30\x53\x84\x24\xb1\x32\x98\xe6\xaa\x6f\xb1\x43\xef\x4d\x59\xa1\x49\x46\x17\x59\x97\x47\x9d\xbc\x2d\x1a\x3c\xd8" + ] + +keccak_key1 = "\x4a\x65\x66\x65" +keccak_data1 = + "\x77\x68\x61\x74\x20\x64\x6f\x20\x79\x61\x20\x77\x61\x6e\x74\x20\x66\x6f\x72\x20\x6e\x6f\x74\x68\x69\x6e\x67\x3f" + +keccak_224_MAC_Vectors :: [MACVector Keccak_224] +keccak_224_MAC_Vectors = + [ MACVector keccak_key1 keccak_data1 $ + digest + "\xe8\x24\xfe\xc9\x6c\x07\x4f\x22\xf9\x92\x35\xbb\x94\x2d\xa1\x98\x26\x64\xab\x69\x2c\xa8\x50\x10\x53\xcb\xd4\x14" + ] + +keccak_256_MAC_Vectors :: [MACVector Keccak_256] +keccak_256_MAC_Vectors = + [ MACVector keccak_key1 keccak_data1 $ + digest + "\xaa\x9a\xed\x44\x8c\x7a\xbc\x8b\x5e\x32\x6f\xfa\x6a\x01\xcd\xed\xf7\xb4\xb8\x31\x88\x14\x68\xc0\x44\xba\x8d\xd4\x56\x63\x69\xa1" + ] + +keccak_384_MAC_Vectors :: [MACVector Keccak_384] +keccak_384_MAC_Vectors = + [ MACVector keccak_key1 keccak_data1 $ + digest + "\x5a\xf5\xc9\xa7\x7a\x23\xa6\xa9\x3d\x80\x64\x9e\x56\x2a\xb7\x7f\x4f\x35\x52\xe3\xc5\xca\xff\xd9\x3b\xdf\x8b\x3c\xfc\x69\x20\xe3\x02\x3f\xc2\x67\x75\xd9\xdf\x1f\x3c\x94\x61\x31\x46\xad\x2c\x9d" + ] + +keccak_512_MAC_Vectors :: [MACVector Keccak_512] +keccak_512_MAC_Vectors = + [ MACVector keccak_key1 keccak_data1 $ + digest + "\xc2\x96\x2e\x5b\xbe\x12\x38\x00\x78\x52\xf7\x9d\x81\x4d\xbb\xec\xd4\x68\x2e\x6f\x09\x7d\x37\xa3\x63\x58\x7c\x03\xbf\xa2\xeb\x08\x59\xd8\xd9\xc7\x01\xe0\x4c\xec\xec\xfd\x3d\xd7\xbf\xd4\x38\xf2\x0b\x8b\x64\x8e\x01\xbf\x8c\x11\xd2\x68\x24\xb9\x6c\xeb\xbd\xcb" + ] + +sha3_key1 = "\x4a\x65\x66\x65" +sha3_data1 = + "\x77\x68\x61\x74\x20\x64\x6f\x20\x79\x61\x20\x77\x61\x6e\x74\x20\x66\x6f\x72\x20\x6e\x6f\x74\x68\x69\x6e\x67\x3f" + +sha3_224_MAC_Vectors :: [MACVector SHA3_224] +sha3_224_MAC_Vectors = + [ MACVector sha3_key1 sha3_data1 $ + digest + "\x7f\xdb\x8d\xd8\x8b\xd2\xf6\x0d\x1b\x79\x86\x34\xad\x38\x68\x11\xc2\xcf\xc8\x5b\xfa\xf5\xd5\x2b\xba\xce\x5e\x66" + ] + +sha3_256_MAC_Vectors :: [MACVector SHA3_256] +sha3_256_MAC_Vectors = + [ MACVector sha3_key1 sha3_data1 $ + digest + "\xc7\xd4\x07\x2e\x78\x88\x77\xae\x35\x96\xbb\xb0\xda\x73\xb8\x87\xc9\x17\x1f\x93\x09\x5b\x29\x4a\xe8\x57\xfb\xe2\x64\x5e\x1b\xa5" + ] + +sha3_384_MAC_Vectors :: [MACVector SHA3_384] +sha3_384_MAC_Vectors = + [ MACVector sha3_key1 sha3_data1 $ + digest + "\xf1\x10\x1f\x8c\xbf\x97\x66\xfd\x67\x64\xd2\xed\x61\x90\x3f\x21\xca\x9b\x18\xf5\x7c\xf3\xe1\xa2\x3c\xa1\x35\x08\xa9\x32\x43\xce\x48\xc0\x45\xdc\x00\x7f\x26\xa2\x1b\x3f\x5e\x0e\x9d\xf4\xc2\x0a" + ] + +sha3_512_MAC_Vectors :: [MACVector SHA3_512] +sha3_512_MAC_Vectors = + [ MACVector sha3_key1 sha3_data1 $ + digest + "\x5a\x4b\xfe\xab\x61\x66\x42\x7c\x7a\x36\x47\xb7\x47\x29\x2b\x83\x84\x53\x7c\xdb\x89\xaf\xb3\xbf\x56\x65\xe4\xc5\xe7\x09\x35\x0b\x28\x7b\xae\xc9\x21\xfd\x7c\xa0\xee\x7a\x0c\x31\xd0\x22\xa9\x5e\x1f\xc9\x2b\xa9\xd7\x7d\xf8\x83\x96\x02\x75\xbe\xb4\xe6\x20\x24" + ] + +macTests :: [Spec] +macTests = + [ describe "md5" $ mapM_ toMACTest $ zip is md5MACVectors + , describe "sha1" $ mapM_ toMACTest $ zip is sha1MACVectors + , describe "sha256" $ mapM_ toMACTest $ zip is sha256MACVectors + , describe "keccak-224" $ mapM_ toMACTest $ zip is keccak_224_MAC_Vectors + , describe "keccak-256" $ mapM_ toMACTest $ zip is keccak_256_MAC_Vectors + , describe "keccak-384" $ mapM_ toMACTest $ zip is keccak_384_MAC_Vectors + , describe "keccak-512" $ mapM_ toMACTest $ zip is keccak_512_MAC_Vectors + , describe "sha3-224" $ mapM_ toMACTest $ zip is sha3_224_MAC_Vectors + , describe "sha3-256" $ mapM_ toMACTest $ zip is sha3_256_MAC_Vectors + , describe "sha3-384" $ mapM_ toMACTest $ zip is sha3_384_MAC_Vectors + , describe "sha3-512" $ mapM_ toMACTest $ zip is sha3_512_MAC_Vectors + ] + where + toMACTest (i, macVector) = do + it + (show i) + ( HMAC.hmac (macKey macVector) (macSecret macVector) + `shouldBe` macResult macVector + ) + it + ("incr-" ++ show i) + ( HMAC.finalize + (HMAC.update (HMAC.initialize (macKey macVector)) (macSecret macVector)) + `shouldBe` macResult macVector + ) + is :: [Int] + is = [1 ..] + +data MacIncremental a = MacIncremental ByteString ByteString (HMAC.HMAC a) + deriving (Show, Eq) + +instance HashAlgorithm a => Arbitrary (MacIncremental a) where + arbitrary = do + key <- arbitraryBSof 1 89 + msg <- arbitraryBSof 1 99 + return $ MacIncremental key msg (HMAC.hmac key msg) + +data MacIncrementalList a = MacIncrementalList ByteString [ByteString] (HMAC.HMAC a) + deriving (Show, Eq) + +instance HashAlgorithm a => Arbitrary (MacIncrementalList a) where + arbitrary = do + key <- arbitraryBSof 1 89 + msgs <- choose (1, 20) >>= \n -> replicateM n (arbitraryBSof 1 99) + return $ MacIncrementalList key msgs (HMAC.hmac key (B.concat msgs)) + +macIncrementalTests :: [Spec] +macIncrementalTests = + [ testIncrProperties MD5 + , testIncrProperties SHA1 + , testIncrProperties SHA256 + , testIncrProperties SHA3_224 + , testIncrProperties SHA3_256 + , testIncrProperties SHA3_384 + , testIncrProperties SHA3_512 + ] + where + -- testIncrProperties :: HashAlgorithm a => a -> [Property] + testIncrProperties a = + describe (show a) $ do + prop "list-one" (prop_inc0 a) + prop "list-multi" (prop_inc1 a) + + prop_inc0 :: HashAlgorithm a => a -> MacIncremental a -> Bool + prop_inc0 _ (MacIncremental secret msg result) = + result `assertEq` HMAC.finalize (HMAC.update (HMAC.initialize secret) msg) + + prop_inc1 :: HashAlgorithm a => a -> MacIncrementalList a -> Bool + prop_inc1 _ (MacIncrementalList secret msgs result) = + result + `assertEq` HMAC.finalize (foldl' HMAC.update (HMAC.initialize secret) msgs) + +spec :: Spec +spec = do + describe "KATs" $ sequence_ macTests + describe "properties" $ sequence_ macIncrementalTests diff --git a/tests/MAC/KMACSpec.hs b/tests/MAC/KMACSpec.hs new file mode 100644 --- /dev/null +++ b/tests/MAC/KMACSpec.hs @@ -0,0 +1,151 @@ +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE FlexibleContexts #-} +{-# LANGUAGE FlexibleInstances #-} +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE RecordWildCards #-} + +module MAC.KMACSpec (spec) where + +import Crypto.Hash ( + HashAlgorithm, + SHAKE128 (..), + SHAKE256 (..), + digestFromByteString, + ) +import qualified Crypto.MAC.KMAC as KMAC + +import qualified Data.ByteString as B + +import Imports + +data MACVector hash = MACVector + { macString :: ByteString + , macKey :: ByteString + , macSecret :: ByteString + , macResult :: KMAC.KMAC hash + } + +instance Show (KMAC.KMAC a) where + show (KMAC.KMAC d) = show d + +digest :: HashAlgorithm hash => ByteString -> KMAC.KMAC hash +digest = maybe (error "cannot get digest") KMAC.KMAC . digestFromByteString + +vectors128 :: [MACVector (SHAKE128 256)] +vectors128 = + [ MACVector + { macString = "" + , macKey = B.pack [0x40 .. 0x5f] + , macSecret = B.pack [0x00 .. 0x03] + , macResult = + digest + "\xe5\x78\x0b\x0d\x3e\xa6\xf7\xd3\xa4\x29\xc5\x70\x6a\xa4\x3a\x00\xfa\xdb\xd7\xd4\x96\x28\x83\x9e\x31\x87\x24\x3f\x45\x6e\xe1\x4e" + } + , MACVector + { macString = "My Tagged Application" + , macKey = B.pack [0x40 .. 0x5f] + , macSecret = B.pack [0x00 .. 0x03] + , macResult = + digest + "\x3b\x1f\xba\x96\x3c\xd8\xb0\xb5\x9e\x8c\x1a\x6d\x71\x88\x8b\x71\x43\x65\x1a\xf8\xba\x0a\x70\x70\xc0\x97\x9e\x28\x11\x32\x4a\xa5" + } + , MACVector + { macString = "My Tagged Application" + , macKey = B.pack [0x40 .. 0x5f] + , macSecret = B.pack [0x00 .. 0xc7] + , macResult = + digest + "\x1f\x5b\x4e\x6c\xca\x02\x20\x9e\x0d\xcb\x5c\xa6\x35\xb8\x9a\x15\xe2\x71\xec\xc7\x60\x07\x1d\xfd\x80\x5f\xaa\x38\xf9\x72\x92\x30" + } + ] + +vectors256 :: [MACVector (SHAKE256 512)] +vectors256 = + [ MACVector + { macString = "My Tagged Application" + , macKey = B.pack [0x40 .. 0x5f] + , macSecret = B.pack [0x00 .. 0x03] + , macResult = + digest + "\x20\xc5\x70\xc3\x13\x46\xf7\x03\xc9\xac\x36\xc6\x1c\x03\xcb\x64\xc3\x97\x0d\x0c\xfc\x78\x7e\x9b\x79\x59\x9d\x27\x3a\x68\xd2\xf7\xf6\x9d\x4c\xc3\xde\x9d\x10\x4a\x35\x16\x89\xf2\x7c\xf6\xf5\x95\x1f\x01\x03\xf3\x3f\x4f\x24\x87\x10\x24\xd9\xc2\x77\x73\xa8\xdd" + } + , MACVector + { macString = "" + , macKey = B.pack [0x40 .. 0x5f] + , macSecret = B.pack [0x00 .. 0xc7] + , macResult = + digest + "\x75\x35\x8c\xf3\x9e\x41\x49\x4e\x94\x97\x07\x92\x7c\xee\x0a\xf2\x0a\x3f\xf5\x53\x90\x4c\x86\xb0\x8f\x21\xcc\x41\x4b\xcf\xd6\x91\x58\x9d\x27\xcf\x5e\x15\x36\x9c\xbb\xff\x8b\x9a\x4c\x2e\xb1\x78\x00\x85\x5d\x02\x35\xff\x63\x5d\xa8\x25\x33\xec\x6b\x75\x9b\x69" + } + , MACVector + { macString = "My Tagged Application" + , macKey = B.pack [0x40 .. 0x5f] + , macSecret = B.pack [0x00 .. 0xc7] + , macResult = + digest + "\xb5\x86\x18\xf7\x1f\x92\xe1\xd5\x6c\x1b\x8c\x55\xdd\xd7\xcd\x18\x8b\x97\xb4\xca\x4d\x99\x83\x1e\xb2\x69\x9a\x83\x7d\xa2\xe4\xd9\x70\xfb\xac\xfd\xe5\x00\x33\xae\xa5\x85\xf1\xa2\x70\x85\x10\xc3\x2d\x07\x88\x08\x01\xbd\x18\x28\x98\xfe\x47\x68\x76\xfc\x89\x65" + } + ] + +macTests :: [Spec] +macTests = + [ describe "SHAKE128" $ mapM_ toMACTest $ zip is vectors128 + , describe "SHAKE256" $ mapM_ toMACTest $ zip is vectors256 + ] + where + toMACTest (i, MACVector{..}) = do + it (show i) (KMAC.kmac macString macKey macSecret `shouldBe` macResult) + it + ("incr-" ++ show i) + ( KMAC.finalize (KMAC.update (KMAC.initialize macString macKey) macSecret) + `shouldBe` macResult + ) + is :: [Int] + is = [1 ..] + +data MacIncremental a = MacIncremental ByteString ByteString ByteString (KMAC.KMAC a) + deriving (Show, Eq) + +instance KMAC.HashSHAKE a => Arbitrary (MacIncremental a) where + arbitrary = do + str <- arbitraryBSof 0 49 + key <- arbitraryBSof 1 89 + msg <- arbitraryBSof 1 99 + return $ MacIncremental str key msg (KMAC.kmac str key msg) + +data MacIncrementalList a + = MacIncrementalList ByteString ByteString [ByteString] (KMAC.KMAC a) + deriving (Show, Eq) + +instance KMAC.HashSHAKE a => Arbitrary (MacIncrementalList a) where + arbitrary = do + str <- arbitraryBSof 0 49 + key <- arbitraryBSof 1 89 + msgs <- choose (1, 20) >>= \n -> replicateM n (arbitraryBSof 1 99) + return $ MacIncrementalList str key msgs (KMAC.kmac str key (B.concat msgs)) + +macIncrementalTests :: [Spec] +macIncrementalTests = + [ testIncrProperties "SHAKE128_256" (SHAKE128 :: SHAKE128 256) + , testIncrProperties "SHAKE256_512" (SHAKE256 :: SHAKE256 512) + ] + where + testIncrProperties :: KMAC.HashSHAKE a => String -> a -> Spec + testIncrProperties name a = + describe name $ do + prop "list-one" (prop_inc0 a) + prop "list-multi" (prop_inc1 a) + + prop_inc0 :: KMAC.HashSHAKE a => a -> MacIncremental a -> Bool + prop_inc0 _ (MacIncremental str secret msg result) = + result `assertEq` KMAC.finalize (KMAC.update (KMAC.initialize str secret) msg) + + prop_inc1 :: KMAC.HashSHAKE a => a -> MacIncrementalList a -> Bool + prop_inc1 _ (MacIncrementalList str secret msgs result) = + result + `assertEq` KMAC.finalize (foldl' KMAC.update (KMAC.initialize str secret) msgs) + +spec :: Spec +spec = do + describe "KATs" $ sequence_ macTests + describe "properties" $ sequence_ macIncrementalTests diff --git a/tests/MAC/Poly1305Spec.hs b/tests/MAC/Poly1305Spec.hs new file mode 100644 --- /dev/null +++ b/tests/MAC/Poly1305Spec.hs @@ -0,0 +1,84 @@ +{-# LANGUAGE OverloadedStrings #-} + +module MAC.Poly1305Spec (spec) where + +import qualified Data.ByteString as B +import qualified Data.ByteString.Char8 as B () + +import Crypto.Error +import Imports + +import qualified Crypto.MAC.Poly1305 as Poly1305 +import qualified Data.ByteArray as B (convert) + +import qualified MAC.Poly1305Vectors as Vectors + +instance Show Poly1305.Auth where + show _ = "Auth" + +-- The key is part of this: with the all-zero key the property below held +-- whatever either side did, r being zero and the tag therefore the nonce -- +-- which is how it came to feed the chunks in the wrong order and pass. +data Chunking = Chunking Int Int ByteString + deriving (Show, Eq) + +instance Arbitrary Chunking where + arbitrary = + Chunking <$> choose (1, 34) <*> choose (1, 2048) <*> arbitraryBS 32 + +-- | The key is checked once, and then 'Poly1305.initialize' and +-- 'Poly1305.auth' cannot fail. +mkKey :: ByteString -> Poly1305.Key +mkKey = throwCryptoError . Poly1305.key + +spec :: Spec +spec = do + describe "key" $ do + it "takes thirty-two bytes" $ + isPassed (Poly1305.key (B.replicate 32 0x41)) `shouldBe` True + it "refuses any other length" $ + [n | n <- [0, 1, 16, 31, 33, 64], isPassed (Poly1305.key (B.replicate n 0x41))] + `shouldBe` [] + it "says which error" $ + -- Key has no Show, on purpose: it is key material + errorOf (Poly1305.key (B.replicate 31 0x41)) + `shouldBe` Just CryptoError_MacKeyInvalid + it "V0" $ + let k = + "\x85\xd6\xbe\x78\x57\x55\x6d\x33\x7f\x44\x52\xfe\x42\xd5\x06\xa8\x01\x03\x80\x8a\xfb\x0d\xb2\xfd\x4a\xbf\xf6\xaf\x41\x49\xf5\x1b" + :: ByteString + msg = "Cryptographic Forum Research Group" :: ByteString + tag = + "\xa8\x06\x1d\xc1\x30\x51\x36\xc6\xc2\x2b\x8b\xaf\x0c\x01\x27\xa9" :: ByteString + in B.convert (Poly1305.auth (mkKey k) msg) `shouldBe` tag + describe "vectors" $ mapM_ vectorTest Vectors.vectors + prop "Chunking" $ \(Chunking chunkLen totalLen k) -> + let msg = B.pack $ take totalLen $ concat (replicate 10 [1 .. 255]) + kk = mkKey k + in Poly1305.auth kk msg + == Poly1305.finalize + (foldl Poly1305.update (Poly1305.initialize kk) (chunks chunkLen msg)) + where + isPassed (CryptoPassed _) = True + isPassed (CryptoFailed _) = False + errorOf (CryptoFailed e) = Just e + errorOf (CryptoPassed _) = Nothing + vectorTest (ki, mi, len, expected) = + it + ( "key " + ++ show ki + ++ ", message " + ++ show mi + ++ ", " + ++ show len + ++ " bytes" + ) + $ B.convert + ( Poly1305.auth + (mkKey (Vectors.polyKey ki)) + (Vectors.polyMessage mi len) + ) + `shouldBe` expected + chunks i bs + | B.length bs < i = [bs] + | otherwise = let (b1, b2) = B.splitAt i bs in b1 : chunks i b2 diff --git a/tests/MAC/Poly1305Vectors.hs b/tests/MAC/Poly1305Vectors.hs new file mode 100644 --- /dev/null +++ b/tests/MAC/Poly1305Vectors.hs @@ -0,0 +1,375 @@ +{-# LANGUAGE OverloadedStrings #-} + +-- | Poly1305 tags from OpenSSL 3.6.4 through EVP_MAC, over patterned keys +-- and messages at lengths either side of the block size and of the groups +-- the bulk loops take. +-- +-- The keys are worth looking at: key 1 is every bit set, which the clamping +-- cuts down to the largest r the algorithm allows, and key 2 has r = 1. +-- Together with the all-ones messages those are what drive the accumulator +-- up against 2^130 - 5, which is where an implementation's carries either +-- work or do not. +module MAC.Poly1305Vectors ( + KATPoly1305, + polyKey, + polyMessage, + vectors, +) where + +import qualified Data.ByteString as B +import Data.ByteString.Char8 () +import Data.Word (Word8) + +-- (key, message pattern, message length, tag) +type KATPoly1305 = (Int, Int, Int, B.ByteString) + +polyKey :: Int -> B.ByteString +polyKey 0 = B.pack [fromIntegral (0x40 + i) :: Word8 | i <- [0 .. 31 :: Int]] +polyKey 1 = B.replicate 32 0xff +polyKey 2 = B.singleton 1 `B.append` B.replicate 31 0 +polyKey _ = B.pack [fromIntegral (i * 7 + 3) :: Word8 | i <- [0 .. 31 :: Int]] + +polyMessage :: Int -> Int -> B.ByteString +polyMessage 0 n = B.replicate n 0 +polyMessage 1 n = B.replicate n 0xff +polyMessage _ n = B.pack [fromIntegral (i * 7 + 3) :: Word8 | i <- [0 .. n - 1]] + +vectors :: [KATPoly1305] +vectors = + [ (0, 0, 0, "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f") + , (0, 0, 1, "\x5f\x91\x93\x95\x57\x99\x9b\x9d\x5f\xa1\xa3\xa5\x67\xa9\xab\xad") + , (0, 0, 15, "\x20\x64\x56\xe8\x2a\x6d\x5f\xf1\x33\x76\x68\xfa\x3c\x7f\x71\x9f") + , (0, 0, 16, "\xe0\x22\x65\x57\xe9\x2b\x6e\x60\xf2\x34\x77\x69\xfb\x3d\x80\x72") + , (0, 0, 17, "\x80\x32\xd6\x2b\xb6\x3a\x5d\xeb\x76\xb6\x52\x14\x9c\x91\xa2\x92") + , (0, 0, 31, "\x3c\x05\x99\x7e\x89\x0e\x21\x3f\x4b\x8b\x17\x69\x71\x67\x68\x84") + , (0, 0, 32, "\xfc\xc3\xa7\xed\x47\xcd\x2f\xae\x09\x4a\x26\xd8\x2f\x26\x77\x57") + , (0, 0, 33, "\x4e\x31\xba\x66\xd7\x64\xd4\x9a\x3b\x50\x48\xaf\x3d\x5e\x1c\x4b") + , (0, 0, 48, "\xca\xc2\x8b\x28\x69\xf7\xa6\x5d\xce\xe3\x1b\x73\xd1\xf2\xf0\x0f") + , (0, 0, 63, "\xa0\x75\x12\xc0\x68\x81\x66\xaf\xbd\x35\x7f\xdd\x1d\x85\x93\x78") + , (0, 0, 64, "\x5b\x34\x21\x2f\x27\x40\x75\x1e\x7c\xf4\x8d\x4c\xdc\x43\xa2\x4b") + , (0, 0, 65, "\xb0\x0a\xac\x16\x14\x0d\x22\xf8\xea\x1c\x63\x2a\xa4\xf4\xe6\xa4") + , (0, 0, 95, "\x57\x7c\x6c\x1e\x14\x3f\xce\x8d\xdd\xdd\xe8\x30\x83\x44\xbc\x9f") + , (0, 0, 96, "\x17\x3b\x7b\x8d\xd2\xfd\xdc\xfc\x9b\x9c\xf7\x9f\x41\x03\xcb\x72") + , (0, 0, 97, "\xb6\x00\xf4\xcb\xe5\xf0\xe6\x2c\x13\x77\xc0\x56\xe2\xc4\x3e\xfb") + , (0, 0, 127, "\x62\xe9\xab\x54\xe9\x73\x2c\xf0\x6a\x36\xd9\x85\x88\xa4\xc5\x1e") + , (0, 0, 128, "\x1d\xa8\xba\xc3\xa7\x32\x3b\x5f\x29\xf5\xe7\xf4\x46\x63\xd4\xf1") + , (0, 0, 129, "\x6b\x5a\xe6\x91\x98\x67\x6f\x9c\xd0\xe7\xce\x11\xf5\x51\x43\x56") + , (0, 0, 191, "\x34\xf2\xf5\xe5\x00\x93\x9f\x79\x9b\xe4\xc1\xd3\xa8\x31\xc2\x4d") + , (0, 0, 192, "\xf4\xb0\x04\x55\xbf\x51\xae\xe8\x59\xa3\xd0\x42\x67\xf0\xd0\x20") + , (0, 0, 255, "\xc8\xae\x35\x70\x15\x3c\xcb\x35\xa5\x49\x53\xda\x08\xb1\x0b\x39") + , (0, 0, 256, "\x88\x6d\x44\xdf\xd3\xfa\xd9\xa4\x63\x08\x62\x49\xc7\x6f\x1a\x0c") + , (0, 0, 257, "\x74\x1b\x9c\xd4\xc3\xc7\x45\xf2\x1b\xca\x71\x64\x6b\xe6\xb8\x1e") + , (0, 0, 1023, "\x5e\xb3\xd4\x07\x60\x46\xba\xf7\x3f\x5b\x43\x23\xf2\xa7\xa0\x99") + , (0, 0, 1024, "\x1e\x72\xe3\x76\x1e\x05\xc9\x66\xfe\x19\x52\x92\xb0\x66\xaf\x6c") + , (0, 0, 1025, "\x44\xef\xa3\x46\x95\x94\xac\x88\x45\xde\x91\xed\xf0\xb1\x67\x79") + , (0, 0, 4096, "\x67\x6f\xd3\x2b\x9b\x8b\x98\x8b\x9a\xc9\x4a\x43\xb4\xa3\xc3\x05") + , (0, 0, 8191, "\x9e\x5c\x85\x39\xbe\x9a\x49\x7c\xaa\x51\x5a\x34\x71\x73\x11\x2f") + , (0, 1, 0, "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f") + , (0, 1, 1, "\x33\x90\x92\xd4\x16\x98\x9a\xdc\x1e\xa0\xa2\xe4\x26\xa8\xaa\xec") + , (0, 1, 15, "\xb0\x35\x18\x7a\xbd\x3f\x22\x84\xc7\x49\x2c\x8e\xd1\x53\x36\xd0") + , (0, 1, 16, "\x30\xb3\x35\x58\x3a\xbd\x3f\x62\x44\xc7\x49\x6c\x4e\xd1\x53\x76") + , (0, 1, 17, "\xbd\x84\xe2\x88\xbb\x59\xe9\x39\xa1\xec\xa7\x9c\x32\x25\x06\x99") + , (0, 1, 31, "\x3a\x2a\x68\x2e\x62\x01\x71\xe1\x49\x96\x31\x46\xdd\xd0\x91\x7c") + , (0, 1, 32, "\xba\xa7\x85\x0c\xdf\x7e\x8e\xbf\xc6\x13\x4f\x24\x5a\x4e\xaf\x22") + , (0, 1, 33, "\x84\xb6\x8d\xcf\x49\x8c\xab\x3f\x8d\x0b\x35\xf0\x5a\xb4\xfe\x0f") + , (0, 1, 48, "\x7c\xd9\x30\x53\x6d\xb1\x50\xc5\xb2\x32\xdc\x77\x82\xdd\xa7\x99") + , (0, 1, 63, "\xae\x4a\x60\xe3\x3a\x8b\xf7\x34\xd3\x62\x14\x05\x79\xf4\x2b\x3b") + , (0, 1, 64, "\x2e\xc8\x7d\xc1\xb7\x08\x15\x13\x50\xe0\x31\xe3\xf5\x71\x49\xe1") + , (0, 1, 65, "\x60\xbb\x7c\x6f\x5f\x89\xd5\xe2\x92\x67\x88\x1d\x09\x9c\xbe\x7b") + , (0, 1, 95, "\xf4\x85\xff\xba\x3b\xd5\x40\xa6\x2c\xc6\x2c\x69\xbf\x73\xf6\x36") + , (0, 1, 96, "\x6f\x03\x1d\x99\xb8\x52\x5e\x84\xa9\x43\x4a\x47\x3c\xf1\x13\xdd") + , (0, 1, 97, "\x33\x7c\x39\x7c\xc1\x9a\x8e\x88\x29\x3a\xc5\x52\xed\x95\xc1\x49") + , (0, 1, 127, "\xcb\x08\xb2\x95\x3b\xae\x7e\x4f\xd6\x96\xb3\x68\x5f\x4d\x9b\x02") + , (0, 1, 128, "\x4b\x86\xcf\x73\xb8\x2b\x9c\x2d\x53\x14\xd1\x46\xdc\xca\xb8\xa8") + , (0, 1, 129, "\xd6\xe7\xf5\x9c\x31\x29\xff\xa7\xd9\x27\x70\x66\x9d\xa5\x60\x1d") + , (0, 1, 191, "\xcb\x79\xa4\xdd\x57\xa0\xd5\x5a\xdd\x67\x64\xf9\x52\x90\xfd\x3a") + , (0, 1, 192, "\x46\xf7\xc1\xbb\xd4\x1d\xf3\x38\x5a\xe5\x81\xd7\xcf\x0d\x1b\xe1") + , (0, 1, 255, "\xe4\x28\x24\x1d\x0a\x38\x3d\xd6\xb5\x7a\xac\x67\xf9\xf5\x88\x88") + , (0, 1, 256, "\x5f\xa6\x41\xfb\x86\xb5\x5a\xb4\x32\xf8\xc9\x45\x76\x73\xa6\x2e") + , (0, 1, 257, "\x15\x69\x36\x20\xff\x02\x9a\xa8\xcd\x9d\x33\x63\x2b\xbe\xed\x0d") + , (0, 1, 1023, "\x98\x61\x49\x06\xca\xaa\xf5\xbe\xa2\x5c\x99\x25\xde\xb7\x3b\xc9") + , (0, 1, 1024, "\x13\xdf\x66\xe4\x46\x28\x13\x9d\x1f\xda\xb6\x03\x5b\x35\x59\x6f") + , (0, 1, 1025, "\x75\x9a\x0c\xa9\x2d\x5f\x15\x5d\xcc\x82\xc0\x07\x32\x7f\xbf\x7a") + , (0, 1, 4096, "\xd6\xdb\x86\xbd\x0f\x30\x0c\x96\x74\x46\xe1\xa4\x92\x7e\x71\x8d") + , (0, 1, 8191, "\x15\x93\x50\xa8\xd4\xdc\xd4\x2a\xbc\xea\xb4\xd3\xa9\x45\xf6\xe2") + , (0, 2, 0, "\x50\x51\x52\x53\x54\x55\x56\x57\x58\x59\x5a\x5b\x5c\x5d\x5e\x5f") + , (0, 2, 1, "\x1f\x55\x5a\x9f\x23\x69\x6e\xb3\x37\x7d\x82\xc7\x4b\x91\x96\xdb") + , (0, 2, 15, "\x3b\xdf\xb2\xf6\x07\x93\x47\x65\x49\xa0\x19\xf5\x8f\x96\xb8\x01") + , (0, 2, 16, "\x60\x8e\x79\x43\x4d\x60\x2c\xd0\xac\x8b\x1c\x7e\x11\xa0\xd9\xdc") + , (0, 2, 17, "\xf4\x27\x1a\xd8\x3e\x68\xcd\x2c\x40\x26\x0d\x51\x8f\xf2\x64\xe7") + , (0, 2, 31, "\xdc\x4e\xd3\x5a\xda\x9c\x04\x97\x25\x00\x3e\x02\x02\x99\x9a\x5a") + , (0, 2, 32, "\xb0\x39\x62\xd9\x00\xd8\xe3\x65\x9c\x8b\x6d\x21\xc9\x74\x1a\x3e") + , (0, 2, 33, "\xf3\x10\xd2\x3a\xfc\xb6\x7f\x4d\x6f\x8d\x95\x06\x3e\xd3\xb1\x54") + , (0, 2, 48, "\x99\xe5\x5f\xe8\x87\x52\x38\xc3\x99\xc7\xd1\xdd\xa0\x15\xbf\xbe") + , (0, 2, 63, "\xa1\x38\xe8\xc8\x77\x9c\x80\xf3\xac\x50\x84\x2b\x7f\x01\x72\x4e") + , (0, 2, 64, "\x48\xc9\xf4\xa6\xcb\xdc\x3c\x81\xb0\x40\xf0\x68\x32\xa1\x8d\x2f") + , (0, 2, 65, "\x2f\x59\x9e\xf7\x24\xb2\x3a\xdf\x6c\x73\xb4\x78\x0c\xa2\x1d\xb6") + , (0, 2, 95, "\x10\x8d\x3f\x55\xbd\x4a\xb1\xbb\x78\xa6\xd4\x9d\x5f\x45\xd9\xba") + , (0, 2, 96, "\x85\xc3\xc9\x92\x3e\x90\x4a\x08\x09\xfb\x7c\xf9\xfe\xa8\x90\x99") + , (0, 2, 97, "\x7b\x20\x42\x0e\x3e\x16\xf5\xe1\x01\x17\xbd\x81\x67\xe4\x62\xcd") + , (0, 2, 127, "\x57\xa6\x86\x72\x39\x65\x64\xe9\x04\x82\x68\x57\xc0\xe0\x15\xb2") + , (0, 2, 128, "\x9a\x82\x8e\x0f\xe8\xaf\xda\xf4\x21\x3b\x4d\xd1\x4b\x08\x69\x8e") + , (0, 2, 129, "\x3a\x56\x1b\xff\x4d\xb4\x9a\x3c\xe1\x8e\x89\x6e\xbe\x65\x78\xe7") + , (0, 2, 191, "\xcd\xed\xba\x31\x2a\x1d\x7e\xf0\xb3\x62\xd2\x2d\x9a\xc1\x69\x31") + , (0, 2, 192, "\xa7\x15\xbe\x8d\x33\x72\xae\x79\xea\xe4\x2f\xe4\xfd\x70\xf4\x08") + , (0, 2, 255, "\xb5\x9d\xce\x3a\xe1\x77\x6b\xab\x85\x23\x73\x10\x5d\xbb\xd3\xf1") + , (0, 2, 256, "\xc0\xe2\xdf\x59\xda\xad\x6d\xbb\x6f\x4a\x66\x11\x38\xd3\xb7\xd7") + , (0, 2, 257, "\x7f\x48\x60\x7e\x67\x5a\x85\xf4\xbd\x2e\x9f\x1e\x35\xc4\xaa\x59") + , (0, 2, 1023, "\xfd\x6c\xaf\x92\x51\x28\x77\x4e\x0a\x2b\x69\x79\x82\xf6\x90\xcb") + , (0, 2, 1024, "\x08\xb2\xc0\xb1\x4a\x5e\x79\x5e\xf4\x51\x5c\x7a\x5d\x0e\x75\xb1") + , (0, 2, 1025, "\x87\xaa\x4c\xa2\x96\xf6\xfe\xb9\xc1\x5b\xdc\x22\x0f\x8a\xfc\x7b") + , (0, 2, 4096, "\x8a\x90\x94\x87\xcd\x36\x97\x1f\xfc\xc3\xd6\x98\xce\x27\x63\x2a") + , (0, 2, 8191, "\xa9\x28\x1b\xfb\x38\x40\xbf\x3b\xa0\x7a\x2f\x00\x3c\x0e\x71\xf3") + , (1, 0, 0, "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff") + , (1, 0, 1, "\x0e\xff\xff\xff\x0f\xfc\xff\xff\x0f\xfc\xff\xff\x0f\xfc\xff\xff") + , (1, 0, 15, "\xfa\xff\x13\xfb\xff\xff\x13\xfb\xff\xff\x13\xfb\xff\xff\x13\xff") + , (1, 0, 16, "\xfa\xff\xff\x13\xfb\xff\xff\x13\xfb\xff\xff\x13\xfb\xff\xff\x13") + , (1, 0, 17, "\x59\xff\x3f\x86\x49\xfc\xef\xad\x44\xfc\x9f\xd5\x3f\xfc\x4f\xfd") + , (1, 0, 31, "\x45\x00\x54\x81\x39\x00\x04\xa9\x34\x00\xb4\xd0\x2f\x00\x64\xfc") + , (1, 0, 32, "\x45\x00\x40\x9a\x34\x00\xf0\xc1\x2f\x00\xa0\xe9\x2a\x00\x50\x11") + , (1, 0, 33, "\x0c\x7c\x2e\x70\x03\x79\xc5\xd6\x23\xb9\x6d\x78\x3f\x39\x27\x55") + , (1, 0, 48, "\xf8\x7c\x2e\x84\xee\x7c\xc5\xea\x0e\xbd\x6d\x8c\x2a\x3d\x27\x69") + , (1, 0, 63, "\x4e\xe2\xfb\xe0\x25\xa5\x75\x45\xa2\x0c\x31\x7b\x2b\x13\x53\xd4") + , (1, 0, 64, "\x49\xe2\xe7\xf9\x20\xa5\x61\x5e\x9d\x0c\x1d\x94\x26\x13\x3f\xe9") + , (1, 0, 65, "\x57\x35\x14\x8c\x66\x80\xfb\x93\xe4\x3c\xd1\x53\x74\xef\x25\xa2") + , (1, 0, 95, "\xd8\xb9\x3a\x01\x74\xd7\x6b\x7a\xac\x5f\x1e\xe3\xb7\x25\xa3\x0a") + , (1, 0, 96, "\xd8\xb9\x26\x1a\x6f\xd7\x57\x93\xa7\x5f\x0a\xfc\xb2\x25\x8f\x1f") + , (1, 0, 97, "\x0a\x57\xd8\xf6\x4a\xe9\x5a\x39\x3e\x82\x2f\xd1\x94\xd0\x5c\x8f") + , (1, 0, 127, "\x23\x39\x46\xb0\xb9\x84\xfd\xec\x07\x6a\x75\xf4\x72\x6b\xc5\xd6") + , (1, 0, 128, "\x23\x39\x32\xc9\xb4\x84\xe9\x05\x03\x6a\x61\x0d\x6e\x6b\xb1\xeb") + , (1, 0, 129, "\xad\xa6\x83\xab\x24\x42\x6e\x94\xff\xb5\x1f\xac\x06\xa0\x8b\x7a") + , (1, 0, 191, "\xef\x04\x96\xb1\xd8\xf2\xd7\x28\x9b\xad\x3a\x00\x20\xe0\xa8\x26") + , (1, 0, 192, "\xef\x04\x82\xca\xd3\xf2\xc3\x41\x96\xad\x26\x19\x1b\xe0\x94\x3b") + , (1, 0, 255, "\xa4\x0a\xde\x3f\x60\xf5\x63\x8c\x03\xf6\x27\xed\x1d\x3a\x49\x19") + , (1, 0, 256, "\xa4\x0a\xca\x58\x5b\xf5\x4f\xa5\xfe\xf5\x13\x06\x19\x3a\x35\x2e") + , (1, 0, 257, "\xef\xf8\x27\xcb\x3b\x83\xa6\xe8\x4e\x3b\x71\x16\xdf\xf1\xf1\x58") + , (1, 0, 1023, "\x1d\x06\x39\x15\x75\xf1\x25\xf2\x8b\xcc\x58\xe3\x3f\x33\x2a\x6f") + , (1, 0, 1024, "\x1d\x06\x25\x2e\x70\xf1\x11\x0b\x87\xcc\x44\xfc\x3a\x33\x16\x84") + , (1, 0, 1025, "\x47\xaa\x39\x03\x90\xc6\x59\xe4\x95\xdc\x0a\x98\xba\x30\xa4\xc7") + , (1, 0, 4096, "\xa2\xa0\xa0\xd6\xbf\x6d\x5e\xd9\x7a\xcc\x51\x04\x15\x59\x10\xeb") + , (1, 0, 8191, "\xa4\x4b\xbf\xb2\x34\xd7\xb3\xb1\x69\x64\x34\xb5\x48\x52\x1f\x00") + , (1, 1, 0, "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff") + , (1, 1, 1, "\x23\xfe\xff\xef\x23\xf8\xff\xef\x23\xf8\xff\xef\x23\xf8\xff\xef") + , (1, 1, 15, "\xfb\xff\x27\xe6\x03\x00\x28\xe6\x03\x00\x28\xe6\x03\x00\x28\xee") + , (1, 1, 16, "\xfb\xff\xff\x17\xfa\xff\xff\x17\xfa\xff\xff\x17\xfa\xff\xff\x17") + , (1, 1, 17, "\x7c\xfe\x7f\xf7\x68\xf8\x1f\x27\x63\xf8\xbf\x56\x5d\xf8\x5f\x86") + , (1, 1, 31, "\x54\x00\xa8\xed\x48\x00\x48\x1d\x43\x00\xe8\x4c\x3d\x00\x88\x84") + , (1, 1, 32, "\x54\x00\x80\x1f\x3f\x00\x20\x4f\x39\x00\xc0\x7e\x33\x00\x60\xae") + , (1, 1, 33, "\x86\xfa\x6a\x43\x7b\xf4\xec\x24\xa2\x74\x50\x4d\xc3\x74\x95\xbc") + , (1, 1, 48, "\x5e\xfc\x6a\x6b\x51\xfc\xec\x4c\x78\x7c\x50\x75\x99\x7c\x95\xe4") + , (1, 1, 63, "\x90\x0f\x0b\xfa\xca\x5f\xd0\xa5\xc6\xa8\x17\xb3\xd1\xe3\xa6\x87") + , (1, 1, 64, "\x90\x0f\xe3\x2b\xc1\x5f\xa8\xd7\xbc\xa8\xef\xe4\xc7\xe3\x7e\xb1") + , (1, 1, 65, "\xe4\xd8\xb1\x31\xf2\x96\xfa\x07\x23\x79\x61\x21\x69\x4f\xfa\x18") + , (1, 1, 95, "\xa1\x78\x56\xba\x28\x9c\xc4\x4b\x39\xd9\x9a\xfc\x46\x2d\x3a\x2f") + , (1, 1, 96, "\x9c\x78\x2e\xec\x1e\x9c\x9c\x7d\x2f\xd9\x72\x2e\x3d\x2d\x12\x59") + , (1, 1, 97, "\xeb\x9a\x03\xe5\x9d\xe1\x39\x9b\x8e\xff\x38\x1e\x90\x5d\x6f\x02") + , (1, 1, 127, "\xfd\xaa\xca\xf2\x15\x6c\x73\x08\xa7\xe5\x9c\x44\xf4\x80\x96\x8a") + , (1, 1, 128, "\xf8\xaa\xa2\x24\x0c\x6c\x4b\x3a\x9d\xe5\x74\x76\xea\x80\x6e\xb4") + , (1, 1, 129, "\x14\x94\x37\x24\xa3\xb2\xea\x6e\x43\xd7\xf2\x24\x7f\x56\x74\xe9") + , (1, 1, 191, "\x21\x39\xf7\x8d\xa1\x56\x46\x50\xf1\x69\x56\xec\xc3\xd9\x0d\x84") + , (1, 1, 192, "\x1c\x39\xcf\xbf\x97\x56\x1e\x82\xe7\x69\x2e\x1e\xba\xd9\xe5\xad") + , (1, 1, 255, "\xc8\x0c\xb4\x38\x44\xf3\x87\x94\x6e\x5a\xa6\x08\x5b\xdf\x67\xda") + , (1, 1, 256, "\xc3\x0c\x8c\x6a\x3a\xf3\x5f\xc6\x64\x5a\x7e\x3a\x51\xdf\x3f\x04") + , (1, 1, 257, "\x2d\x5d\x96\xb0\x8b\xcd\xc7\xa0\xa2\xdd\x87\xa4\x4f\x52\x22\xc1") + , (1, 1, 1023, "\x25\xd4\xba\x38\x5d\xbb\x70\xdb\xab\x28\x14\x30\xea\xa3\x42\x0e") + , (1, 1, 1024, "\x25\xd4\x92\x6a\x53\xbb\x48\x0d\xa2\x28\xec\x61\xe0\xa3\x1a\x38") + , (1, 1, 1025, "\x04\x32\x45\x5a\x8a\x51\x05\x02\x2b\x9f\xa6\xd9\x23\x04\xc5\x12") + , (1, 1, 4096, "\x28\x27\x27\x9b\x4c\x1d\x3e\x6b\x93\x28\x62\x38\x19\x9e\x13\x1a") + , (1, 1, 8191, "\xc4\x5a\xc2\xf5\x0f\xcf\xe7\x5a\x1c\x12\x82\xc5\xf4\x95\x35\xbc") + , (1, 2, 0, "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff") + , (1, 2, 1, "\x10\xff\xff\x2f\x04\xfc\xff\x2f\x04\xfc\xff\x2f\x04\xfc\xff\x2f") + , (1, 2, 15, "\xd1\x94\xfa\x5a\x1e\xcb\x1b\x67\x15\xad\xe8\x1e\xb8\x3a\x61\xd6") + , (1, 2, 16, "\xcc\x94\x56\x60\x17\xcb\x77\x6c\x0e\xad\x44\x24\xb1\x3a\xbd\x87") + , (1, 2, 17, "\x3b\x31\xb9\x36\x76\x40\x13\x6a\x3d\xb4\x0c\x7c\x22\x85\xa2\x69") + , (1, 2, 31, "\x80\xe3\x0f\xe5\x15\xdc\x39\xd3\x82\xe0\xae\x4b\xb9\xed\x6b\xdf") + , (1, 2, 32, "\x80\xe3\x2b\xc3\x0c\xdc\x55\xb1\x79\xe0\xca\x29\xb0\xed\x87\x29") + , (1, 2, 33, "\xbe\x62\x25\xc1\x2e\x73\xec\x09\x7f\x09\x51\xb6\x50\xf3\x68\x1e") + , (1, 2, 48, "\x69\xff\x86\x7c\x2f\xa9\xc8\x2a\xd0\x7e\x5a\xef\xa4\x56\x4b\x1b") + , (1, 2, 63, "\x1a\x5e\x06\xc3\xf7\x2d\xd2\x55\x0b\xfc\x12\x19\xe5\x0c\xfb\xaa") + , (1, 2, 64, "\x1a\x5e\xa2\x3e\xef\x2d\x6e\xd1\x02\xfc\xae\x94\xdc\x0c\x97\x12") + , (1, 2, 65, "\xd8\x51\x2b\x80\x7c\x60\x02\x4d\x02\x0e\x2e\x20\x7c\x91\x2a\x3a") + , (1, 2, 95, "\xb9\xba\x11\xaf\xea\x25\x04\xad\xac\x81\xee\xba\x8e\x13\x32\xda") + , (1, 2, 96, "\xb4\xba\x2d\xc8\xe2\x25\x20\xc6\xa4\x81\x0a\xd4\x86\x13\x4e\x5f") + , (1, 2, 97, "\xad\x9d\x97\x3b\x35\x49\x6e\x70\x88\x5d\x1f\xdb\xae\xfc\xe7\xc0") + , (1, 2, 127, "\x81\x05\x0f\x62\xce\x71\x46\x9b\xbb\x74\xdb\xc3\x66\xa1\xc2\xda") + , (1, 2, 128, "\x81\x05\xab\x18\xc7\x71\xe2\x51\xb4\x74\x77\x7a\x5f\xa1\x5e\x7d") + , (1, 2, 129, "\xda\x18\x6a\x12\xc3\x1e\x90\xf3\x77\x12\x7f\x16\x06\x5c\xa9\x1e") + , (1, 2, 191, "\xfc\xdb\x96\x58\x66\x37\x62\x5e\xdb\x79\x9a\xed\x48\xa8\xf7\x7a") + , (1, 2, 192, "\xfc\xdb\x32\x4a\x60\x37\xfe\x4f\xd5\x79\x36\xdf\x42\xa8\x93\x58") + , (1, 2, 255, "\x66\x13\xde\xb0\x3a\x3a\xd9\x22\x8d\xf1\x03\xc1\xed\xa9\xa5\x53") + , (1, 2, 256, "\x61\x13\x7a\xf1\x30\x3a\x75\x63\x83\xf1\x9f\x01\xe4\xa9\x41\x80") + , (1, 2, 257, "\xe0\x11\x9c\x10\x1b\xd5\x3c\x06\xad\x17\xcf\xd4\xec\x6a\xef\x40") + , (1, 2, 1023, "\xac\xac\xb6\x9d\xcd\x83\x9d\x5e\xb1\x32\x9b\xdb\x81\x80\x20\x25") + , (1, 2, 1024, "\xa7\xac\x52\xde\xc3\x83\x39\x9f\xa7\x32\x37\x1c\x78\x80\xbc\x51") + , (1, 2, 1025, "\xba\x3d\x31\x86\x0a\xf9\xa5\x58\x4e\x52\x59\x4d\x6d\xdf\x34\x5f") + , (1, 2, 4096, "\x78\x43\x04\x8a\x27\xd6\xbb\x5c\xe4\x10\x0b\xf8\xb5\xcb\x8c\x90") + , (1, 2, 8191, "\xef\x45\x90\x25\x1c\x7b\x36\x2b\x18\xb4\xb5\xe0\x74\x84\x8d\xb7") + , (2, 0, 0, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 1, "\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 15, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 16, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 17, "\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 31, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 32, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 33, "\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 48, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 63, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 64, "\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 65, "\x05\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 95, "\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 96, "\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 97, "\x05\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 127, "\x05\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 128, "\x0a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 129, "\x0a\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 191, "\x0a\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 192, "\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 255, "\x0f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 256, "\x14\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 257, "\x14\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 1023, "\x4b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 0, 1024, "\x50\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 1025, "\x50\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 4096, "\x40\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 0, 8191, "\x7b\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01") + , (2, 1, 0, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 1, "\xff\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 15, "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x01") + , (2, 1, 16, "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff") + , (2, 1, 17, "\xfe\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 31, "\xfe\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x01") + , (2, 1, 32, "\x03\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 33, "\x02\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 48, "\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 63, "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 1, 64, "\x06\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 65, "\x05\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 95, "\x04\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 1, 96, "\x09\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 97, "\x08\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 127, "\x07\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 1, 128, "\x0c\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 129, "\x0b\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 191, "\x0d\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 1, 192, "\x12\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 255, "\x13\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 1, 256, "\x18\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 257, "\x17\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 1023, "\x5b\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 1, 1024, "\x60\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 1025, "\x5f\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 4096, "\x80\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 1, 8191, "\xfb\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02") + , (2, 2, 0, "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 2, 1, "\x03\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00") + , (2, 2, 15, "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x01") + , (2, 2, 16, "\x03\x0a\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c") + , (2, 2, 17, "\x76\x0b\x11\x18\x1f\x26\x2d\x34\x3b\x42\x49\x50\x57\x5e\x65\x6c") + , (2, 2, 31, "\x76\x84\x92\xa0\xae\xbc\xca\xd8\xe6\xf4\x02\x11\x1f\x2d\x3b\x6e") + , (2, 2, 32, "\x76\x84\x92\xa0\xae\xbc\xca\xd8\xe6\xf4\x02\x11\x1f\x2d\x3b\x49") + , (2, 2, 33, "\x59\x86\x92\xa0\xae\xbc\xca\xd8\xe6\xf4\x02\x11\x1f\x2d\x3b\x49") + , (2, 2, 48, "\x5e\x6f\x84\x99\xae\xc3\xd7\xec\x01\x17\x2c\x41\x56\x6b\x80\x95") + , (2, 2, 63, "\xb1\xc9\xe5\x01\x1e\x3a\x55\x71\x8d\xa9\xc5\xe1\xfd\x19\x36\x97") + , (2, 2, 64, "\xb1\xc9\xe5\x01\x1e\x3a\x55\x71\x8d\xa9\xc5\xe1\xfd\x19\x36\x52") + , (2, 2, 65, "\x74\xcb\xe5\x01\x1e\x3a\x55\x71\x8d\xa9\xc5\xe1\xfd\x19\x36\x52") + , (2, 2, 95, "\xa7\xce\xf8\x22\x4d\x77\xa0\xca\xf4\x1e\x48\x72\x9c\xc6\xf0\x7f") + , (2, 2, 96, "\xac\xce\xf8\x22\x4d\x77\xa0\xca\xf4\x1e\x48\x72\x9c\xc6\xf0\x1a") + , (2, 2, 97, "\x4f\xd0\xf8\x22\x4d\x77\xa0\xca\xf4\x1e\x48\x72\x9c\xc6\xf0\x1a") + , (2, 2, 127, "\x62\x93\xcb\x03\x3c\x74\xab\xe3\x1b\x54\x8b\xc3\xfb\x33\x6c\x28") + , (2, 2, 128, "\x62\x93\xcb\x03\x3c\x74\xab\xe3\x1b\x54\x8b\xc3\xfb\x33\x6c\xa3") + , (2, 2, 129, "\xe5\x94\xcb\x03\x3c\x74\xab\xe3\x1b\x54\x8b\xc3\xfb\x33\x6c\xa3") + , (2, 2, 191, "\x18\x5e\xb2\x05\x5a\xae\x01\x56\xa9\xfd\x50\xa5\xf9\x4d\xa2\xba") + , (2, 2, 192, "\x18\x5e\xb2\x05\x5a\xae\x01\x56\xa9\xfd\x50\xa5\xf9\x4d\xa2\xf5") + , (2, 2, 255, "\xc9\x27\x98\x07\x78\xe8\x57\xc8\x37\xa8\x17\x88\xf7\x67\xd8\x4c") + , (2, 2, 256, "\xce\x27\x98\x07\x78\xe8\x57\xc8\x37\xa8\x17\x88\xf7\x67\xd8\x47") + , (2, 2, 257, "\xd1\x28\x98\x07\x78\xe8\x57\xc8\x37\xa8\x17\x88\xf7\x67\xd8\x47") + , (2, 2, 1023, "\x33\x9f\x60\x1e\xe0\xa1\x5f\x21\xdf\xa0\x5e\x20\xde\x9f\x61\x24") + , (2, 2, 1024, "\x38\x9f\x60\x1e\xe0\xa1\x5f\x21\xdf\xa0\x5e\x20\xde\x9f\x61\x1f") + , (2, 2, 1025, "\x3b\xa0\x60\x1e\xe0\xa1\x5f\x21\xdf\xa0\x5e\x20\xde\x9f\x61\x1f") + , (2, 2, 4096, "\xe5\x7c\x82\x79\x80\x87\x7e\x85\x7c\x83\x7a\x81\x78\x7f\x86\x7d") + , (2, 2, 8191, "\xc5\xf9\x04\xf3\x00\x0f\xfd\x0a\xf9\x06\xf5\x02\xf1\xfe\x0c\x00") + , (3, 0, 0, "\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc") + , (3, 0, 1, "\x82\x7d\x8b\x99\x97\xb2\xc3\xd1\xaf\xea\xfb\x09\xc8\x22\x34\x42") + , (3, 0, 15, "\xbd\x8f\x8b\x2b\xff\xce\xa2\x6a\x3e\x0e\xba\xa9\x7d\x4d\xe5\xdf") + , (3, 0, 16, "\xf3\xc6\x96\x92\x32\x06\xd6\xa9\x71\x45\x15\xc1\xb0\x84\x54\xec") + , (3, 0, 17, "\xc5\xa4\xba\xba\xd8\x73\x8d\x31\xf4\xd7\x48\x11\x65\x28\x97\x4c") + , (3, 0, 31, "\x00\xb7\xba\x4c\x40\x90\x6c\xca\x82\xfb\x06\xb1\x1a\x53\x48\xea") + , (3, 0, 32, "\x3b\xee\xc5\xb3\x73\xc7\x9f\x09\xb6\x32\x62\xc8\x4d\x8a\xb7\xf6") + , (3, 0, 33, "\x9c\x1a\x1e\x2b\x12\x97\xaa\xfc\x50\xb8\x6e\x81\xd0\x00\xc0\x88") + , (3, 0, 48, "\x0d\x64\x29\x24\xad\xea\xbc\xd4\x12\x13\x88\x38\xb9\x62\xe0\x32") + , (3, 0, 63, "\xe7\x60\x81\x2c\x17\x5f\xa3\xd2\x5a\xb5\xf9\xa0\xbb\xa3\xa7\xdd") + , (3, 0, 64, "\x1d\x98\x8c\x93\x4a\x96\xd6\x11\x8e\xec\x54\xb8\xee\xda\x16\xea") + , (3, 0, 65, "\x7e\x5d\xcf\xbc\xde\x22\x8b\x75\xad\x1a\x88\x44\xb3\x62\xfb\xd3") + , (3, 0, 95, "\xd1\x24\xdf\xb8\x7b\xc3\x0c\x6d\xe9\x4a\x72\x37\xeb\x25\xc9\x95") + , (3, 0, 96, "\x07\x5c\xea\x1f\xaf\xfa\x3f\xac\x1c\x82\xcd\x4e\x1e\x5d\x38\xa2") + , (3, 0, 97, "\xaa\x75\x6f\x48\x7c\xc6\x03\x92\x25\xe7\x85\xc5\x00\xc6\x76\xbb") + , (3, 0, 127, "\xa9\x67\xa2\xea\xff\xaf\x32\x7a\xdf\x56\xf8\x44\xd2\x08\x3b\xf9") + , (3, 0, 128, "\xdf\x9e\xad\x51\x33\xe7\x65\xb9\x12\x8e\x53\x5c\x05\x40\xaa\x05") + , (3, 0, 129, "\x6f\xf4\xd0\x8f\xed\x62\xd2\x78\x66\x20\x24\xe4\x3b\x02\x27\x7b") + , (3, 0, 191, "\x12\x74\xd7\x6e\xa0\x0c\x65\x0c\xe4\x2c\xb9\xf8\x6e\x86\xc5\xa0") + , (3, 0, 192, "\x4d\xab\xe2\xd5\xd3\x43\x98\x4b\x17\x64\x14\x10\xa2\xbd\x34\xad") + , (3, 0, 255, "\x2e\x72\x79\xb4\x86\x0f\xb8\x4f\x8f\x2c\xcc\xfa\x16\x10\xeb\x2d") + , (3, 0, 256, "\x64\xa9\x84\x1b\xba\x46\xeb\x8e\xc2\x63\x27\x12\x4a\x47\x5a\x3a") + , (3, 0, 257, "\x1d\xce\xdc\xfc\x31\x6f\xc9\x27\xd6\x40\xde\xa3\xa5\x27\xb2\x39") + , (3, 0, 1023, "\xdf\xab\x40\xfd\xcb\x6d\x49\xa4\x68\xb2\x27\x10\xb3\x3f\x47\x48") + , (3, 0, 1024, "\x15\xe3\x4b\x64\xff\xa4\x7c\xe3\x9b\xe9\x82\x27\xe6\x76\xb6\x54") + , (3, 0, 1025, "\xb3\x73\xd8\xb5\xef\x8d\xd8\x56\x3d\x37\x0e\x1c\x9f\xbb\xe9\xf9") + , (3, 0, 4096, "\x0e\xa3\x46\xb2\xe1\x7e\xc9\x4f\xfc\xcb\x5d\xcf\xb8\x31\xdc\x5d") + , (3, 0, 8191, "\x3e\xaf\x23\x2b\x79\x15\x05\xe0\xb9\x02\x31\xa5\x76\x72\x3c\x48") + , (3, 1, 0, "\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc") + , (3, 1, 1, "\x8e\x76\x84\xa2\x83\xa8\xbc\xfa\x7b\xe0\xf4\x52\x74\x18\x2d\x9b") + , (3, 1, 15, "\x04\x9b\x84\xc6\x52\xe1\x7a\x2c\x99\x27\x71\x92\xdf\x6d\x8f\xd6") + , (3, 1, 16, "\x75\x09\x9b\x94\xb9\x4f\xe1\xaa\xff\x95\x27\xc1\x45\xdc\x6d\xef") + , (3, 1, 17, "\x78\x72\x23\x97\x9e\xc3\xae\x6d\x34\xfd\x1d\xc2\x30\x1f\xd7\xda") + , (3, 1, 31, "\xf3\x96\x23\xbb\x6d\xfc\x6c\x9f\x51\x44\x9a\x01\x9c\x74\x39\x16") + , (3, 1, 32, "\x5f\x05\x3a\x89\xd4\x6a\xd3\x1d\xb8\xb2\x50\x30\x02\xe3\x17\x2f") + , (3, 1, 33, "\x7b\x99\x67\x84\xe3\xed\x04\xfb\xd6\x70\x4b\xe2\xe4\x22\x08\x23") + , (3, 1, 48, "\x62\x2c\x7e\x76\x19\x95\x29\xab\x5a\x26\x7e\x50\xb6\xe6\x48\x77") + , (3, 1, 63, "\xd0\x62\x78\x94\x6f\xc1\xae\x0f\x88\x56\x8a\x21\x5d\xd5\xab\xa0") + , (3, 1, 64, "\x41\xd1\x8e\x62\xd6\x2f\x15\x8e\xee\xc4\x40\x50\xc3\x43\x8a\xb9") + , (3, 1, 65, "\x54\x83\x6f\x66\x3f\x2f\xdf\x58\xac\xb3\x36\x99\x8e\xcb\xb5\xe3") + , (3, 1, 95, "\x1f\x81\x82\xd6\x81\xd3\xc6\x95\x99\xa3\xb4\x6f\xc9\xa4\x6d\x4a") + , (3, 1, 96, "\x90\xef\x98\xa4\xe8\x41\x2d\x14\x00\x12\x6b\x9e\x2f\x13\x4c\x63") + , (3, 1, 97, "\xc1\x06\x96\xa7\x2f\x8d\x09\x7b\x3c\xdc\x00\x34\x85\x0f\x7d\xf9") + , (3, 1, 127, "\xef\x6a\x06\xdf\x86\x22\xc1\xd8\xc0\x7e\x55\xe6\x11\xb5\x5c\x5b") + , (3, 1, 128, "\x5b\xd9\x1c\xad\xed\x90\x27\x57\x27\xed\x0b\x15\x78\x23\x3b\x74") + , (3, 1, 129, "\x46\x05\x0b\xca\xea\x48\xce\x5c\x8a\xba\xbe\x58\xff\x8a\x50\xac") + , (3, 1, 191, "\x70\x13\x46\x17\xe1\xc4\x30\x88\xf9\xe5\x3c\xf1\xcd\x4b\x69\x24") + , (3, 1, 192, "\xdc\x81\x5c\xe5\x47\x33\x97\x06\x60\x54\xf3\x1f\x34\xba\x47\x3d") + , (3, 1, 255, "\xf7\xdd\x6e\x04\x8f\xfb\x2d\x0c\xc7\x18\xba\xf3\xfd\xf0\x2f\x34") + , (3, 1, 256, "\x63\x4c\x85\xd2\xf5\x69\x94\x8a\x2d\x87\x70\x22\x64\x5f\x0e\x4d") + , (3, 1, 257, "\x45\xd7\xb2\x19\x70\x24\x5d\xc8\x76\xe1\x6a\xd8\x17\xeb\x5d\xf7") + , (3, 1, 1023, "\x65\x56\xc4\x5b\x15\xd3\x42\x0b\xff\x52\x8e\x40\xb9\xc3\xd1\x53") + , (3, 1, 1024, "\xd1\xc4\xda\x29\x7c\x41\xa9\x89\x65\xc1\x44\x6f\x1f\x32\xb0\x6c") + , (3, 1, 1025, "\x01\x9e\xad\xf7\x86\x7c\xa2\xcd\x25\x09\x3e\x02\x10\x03\x07\xde") + , (3, 1, 4096, "\x31\xab\x07\x21\xf2\x46\xd2\x3e\xd9\x9d\x7e\x05\xe9\xab\x10\xde") + , (3, 1, 8191, "\xd7\xc0\xd4\x92\x4b\x9c\xbd\x1f\xfa\x4c\x66\xf3\xa3\x00\xf8\x86") + , (3, 2, 0, "\x73\x7a\x81\x88\x8f\x96\x9d\xa4\xab\xb2\xb9\xc0\xc7\xce\xd5\xdc") + , (3, 2, 1, "\x8b\x9b\xbe\xb1\xeb\x24\x4b\xde\x57\xb1\xd7\x0a\xc4\x3d\x64\x67") + , (3, 2, 15, "\x7f\xc5\xb4\xd6\xda\xf9\x4c\xda\x60\xd0\xfa\x66\xe3\x16\x30\x39") + , (3, 2, 16, "\x00\xf9\x00\x06\x27\x00\xb4\xa3\x7f\xa9\x7c\xca\xd4\xc2\x28\x90") + , (3, 2, 17, "\x2d\x5d\xb7\xae\xee\xb7\x09\xd6\xc7\xe3\xf4\x39\xee\xd7\x3b\xd0") + , (3, 2, 31, "\xd2\x3a\x59\x9f\x59\x9b\x7d\x62\xfe\xa1\x87\x22\x24\x18\xec\x88") + , (3, 2, 32, "\xcb\xbf\x0e\x23\x7d\x52\x2d\xce\x53\x8b\x31\x76\xab\x33\xac\x36") + , (3, 2, 33, "\x3f\x68\xd1\xc1\x88\x18\x77\x00\x31\xeb\x9a\xb9\xad\x1a\xed\xba") + , (3, 2, 48, "\x0e\x5f\x30\xa8\x47\xe3\x32\x37\x21\x00\xa4\xdc\x54\x6f\xa9\x3b") + , (3, 2, 63, "\x0f\x7c\x9f\x25\x29\xe6\x54\x3f\x25\x22\xbd\x2c\x6a\x20\x83\x2f") + , (3, 2, 64, "\x78\x57\x12\x48\x58\x8f\x5d\xea\x21\x99\x5b\x60\x34\x65\x53\x7b") + , (3, 2, 65, "\x25\xe3\xed\x15\xa5\x78\x8e\x53\x98\xec\x13\x2c\x8f\x5a\xe4\xf8") + , (3, 2, 95, "\xf3\xb9\x08\x15\x0c\x87\x05\x8a\xb8\xe8\x96\xbb\x0d\xae\x41\x36") + , (3, 2, 96, "\xcc\xeb\x38\xd6\x46\x22\x67\x74\x5c\xed\x29\xcf\x1a\x1c\x22\x20") + , (3, 2, 97, "\x5e\x4d\x61\xd7\x7e\x46\xc2\xac\x18\x88\x12\x02\x42\x35\x4f\x1c") + , (3, 2, 127, "\x4c\x4c\x4c\x1a\xe7\x66\x2f\xb2\x2d\xdd\x06\xf8\x78\xea\x39\x0a") + , (3, 2, 128, "\x95\xd4\x39\x7a\x2d\xf4\xe9\xdb\x78\x6f\x8e\xeb\xc8\x81\x2a\x92") + , (3, 2, 129, "\x39\xc5\xbd\x6b\x01\xc7\x0c\x76\x81\xc8\xbb\x75\xa4\xf7\x8f\x4c") + , (3, 2, 191, "\x18\xaf\xb1\xeb\xf6\xce\xd4\xe4\xe0\xd1\x39\x0c\xf8\x5c\x2b\x82") + , (3, 2, 192, "\x41\xe4\x19\x89\x54\x40\x41\x8d\x7a\x7f\xaa\xbf\xcd\x46\x3c\x46") + , (3, 2, 255, "\xc9\x40\x39\x72\x8f\x24\x2c\x6a\x1d\x32\xec\xf2\x51\x77\x3c\x73") + , (3, 2, 256, "\x52\x6f\x31\x57\xa7\xe9\x82\x96\xcb\x8d\xa1\x66\x96\x69\xec\x82") + , (3, 2, 257, "\x2b\xa9\xcc\xcf\x60\x2a\xf7\x42\xf5\x4d\x3f\xed\xcc\xa3\xf0\xee") + , (3, 2, 1023, "\x7b\x31\x59\xeb\x2d\x3c\x78\x6d\xb6\x98\xf3\x02\x43\x32\x57\xac") + , (3, 2, 1024, "\x04\x60\x51\xd0\x45\x01\xcf\x99\x64\xf4\xa8\x76\x87\x24\x07\xbc") + , (3, 2, 1025, "\x54\xf4\x5c\x58\x7e\x82\x4d\x01\x28\x48\xcb\xa7\x1a\x44\x06\xad") + , (3, 2, 4096, "\x63\x86\x57\x0e\xdd\xc2\xd7\xcf\x95\x72\xdb\xb5\x15\x17\xf7\x5c") + , (3, 2, 8191, "\xd2\x60\xb6\x4a\x6c\x0c\x2f\x78\x19\x14\xfe\x10\x91\xe7\xd3\xab") + ] diff --git a/tests/Number.hs b/tests/Number.hs deleted file mode 100644 --- a/tests/Number.hs +++ /dev/null @@ -1,117 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module Number (tests) where - -import Imports - -import Crypto.Number.Basic -import Crypto.Number.Generate -import Crypto.Number.ModArithmetic -import Crypto.Number.Prime -import qualified Crypto.Number.Serialize as BE -import qualified Crypto.Number.Serialize.LE as LE -import Data.Bits -import Data.ByteArray (Bytes) -import qualified Data.ByteArray as B - -serializationVectors :: [(Int, Integer, ByteString)] -serializationVectors = - [ - ( 128 - , 468189858948067662094510918729062682059955669513914188715630930503497261316361784677177564296207557978182700664806717692596876084916561811001371208806217360635705059859428069669992937334724312890015700331031248133952795914192719979937664050389500162437642525331653766885896869239678885404647468665996400635 - , "\x00\xaa\xae\x74\xc8\xec\x3c\x36\x06\x5e\x46\xca\x8e\x57\xab\x09\x87\xfd\xcd\x1f\xa4\xe7\xf9\xd2\x60\xd5\x4a\x1b\x74\xdc\xa8\x75\xd8\xdd\xff\x2b\x74\x28\x14\x59\x67\x6c\x82\xae\xa3\xa5\x1d\x3f\xb4\xb7\xfe\x5c\xd2\xf0\x7f\xd8\xd9\xa9\xb0\xce\x26\xc1\x26\x74\x96\xf5\xf6\x4c\x8f\x66\x7f\x5d\xf1\x68\x38\xd4\x03\x62\xe9\x30\xc8\xa1\xc1\x84\x97\x62\x20\xfd\xd7\x03\x35\xc1\x25\x45\x1b\x86\x81\x3d\xa4\x92\xc0\xd3\xdd\xfa\x86\x1d\xdf\x0a\xbb\xf4\xc0\x56\xf7\xa2\xb0\x3b\x52\xf7\xa5\x89\x4c\x69\x34\x91\x46\xd9\x57\xfb" - ) - , - ( 128 - , 40031303476923779996794876613623495515025748694978019540894726181695410095832601107261950025830235596060960914255795497479135806963313279476038687192202016132891881954743054164975707083302554941058329647014950354509055121290280892911153779672733723699997592027662953953692834215577119173225643193201177329 - , "\x00\x0e\x97\xf9\xd5\x79\xb9\x90\x7c\x85\x48\x49\x01\x19\x64\xfb\x76\x31\xcd\x51\xfb\x8a\x9d\x55\xe5\xd3\x7b\x87\x2d\xad\x63\x2d\x6b\x1c\x84\x3f\x65\x95\xb6\xf3\x1a\xa9\x43\x3f\x06\x46\x7b\xf8\xf3\x35\x45\x84\x11\x56\x91\x53\x43\xd7\xe1\x6d\x80\x64\x14\x45\x35\x4e\x93\x7d\x5e\x48\xec\xe0\x79\x7b\x44\x8e\xab\x0f\xc4\x5f\xc6\xa1\x71\xee\x37\xb1\x55\x51\x98\x44\x57\xe3\xc3\x56\x3a\x50\x27\xaf\xa5\x1d\x1a\x0a\x90\x19\x0d\x14\xed\x3d\x93\x40\x62\x76\xa3\xaa\x00\x23\x86\xca\x98\xb2\x6e\x02\x43\xa7\xbc\xb1\xb2\xf1" - ) - , - ( 128 - , 75152325976543603337003024341071663845101857195436434620947904288957274825323005869230041326941600298094896018190395352332646796347130114769768242670539699217743549573961461985255265474392937773768121046339453584830072421569334022498680626938734088755136253492360177084153487115846920446085149631919580041 - , "\x00\x1b\x65\xb1\x73\x74\xed\xd2\xcb\xb8\xf3\x6b\x3f\xc2\x05\xaa\x91\xab\x48\x5b\x03\x30\xae\x24\xa3\xec\x7a\x6a\xf0\x34\x73\x18\x04\xea\xe4\xd6\x19\x97\xc4\xc1\x13\x7d\x12\x0d\xd5\xcb\xbd\x18\x05\xc2\xce\x87\x66\x84\x12\xe8\x24\xa3\x31\x69\xfa\xf4\x2c\x21\x53\xa6\x04\x74\x78\xc4\x93\x0d\x38\x7f\x28\xfe\x80\x8e\xd2\x7b\x20\xc8\xf5\x1f\x0f\x73\x68\xb2\xe5\x08\xf1\x94\xa1\xe6\xcf\x3a\x2c\x12\x63\xda\x08\x3a\x78\x12\xb8\x11\x23\x3c\x38\x38\x10\x94\x2b\xac\x64\x5d\x67\x0c\xb6\x0d\xc3\x9a\x45\x39\x50\x8a\x63\x89" - ) - , - ( 128 - , 132094272981815297755209818914225029878347650582749561568514551350741192910991391836297682842650690115955454061006435646226436379226218676796260483719213285072886626400953065229934239690821114513313427305727000011361769875430428291375851099221794646192854831002408178061474948738788927399080262963320752452 - , "\x00\x30\x27\xe0\xbf\x46\xec\x77\x2d\xc6\x06\x77\xbc\x68\x87\x3c\x1b\x2e\xc7\xb7\x6c\x88\x25\xec\x8c\x95\xbf\x74\xe5\x37\x01\x25\x96\xe1\x70\x33\x5c\x7d\xab\x1f\xc2\x9c\xad\xf7\xca\x26\x85\x2d\xfc\x8f\xc7\xab\x49\x28\xa4\x47\xe6\xd5\x6e\xfa\x0a\xbb\x57\xe4\xa2\x51\xc7\xc6\x12\x0f\xa9\x98\x69\xb8\x05\x84\xc5\xe3\x28\x86\x0f\x54\x1d\xf9\x92\x42\x9f\xb1\x77\x2b\x58\x89\xe2\xfc\x22\xb0\x1e\x71\x78\xea\x39\xc1\x87\x4f\xd4\x83\x2c\x96\x1d\xea\xd5\xf9\xf9\xb9\x7b\x86\xfa\xf6\xad\x5b\xb1\x3c\xe7\x11\xd7\x96\x89\x44" - ) - , - ( 128 - , 577245873336454863811643140721674509319073059708446946821011267146688442860798353087462545395033001525475835015592425207995480357299993009193426638306801669333644226765032464458284920004140299209138389393494751627076239104390434285377314678827349631962212281858308570255468721491493027423799738158196939966 - , "\x00\xd2\x70\x41\xdb\x3d\xb5\xfe\x8c\xef\x79\xcf\x5b\x7b\x37\xb0\x05\xb8\x5a\x9b\x7d\x01\x28\xc7\xf5\x5a\x02\xba\xce\xbc\xf5\x8e\x91\x59\xd0\x42\x6f\x04\x82\x4b\x78\xb0\xdd\x91\x2e\x15\x9d\xea\x4f\x0c\x21\xc0\x67\x54\xa2\x39\xa8\xe1\x13\x8f\xa9\xff\x46\x2d\x11\x56\x04\xa0\xde\x64\xc8\x0f\xf4\x2c\xd2\x31\xdf\x2a\xfd\xac\xc7\x25\x58\xc8\xea\xfd\x47\x6e\xdd\x2a\x53\x02\x77\x49\xa7\x0d\x18\xfb\x05\x18\x4b\x28\xd3\xa2\x39\x8c\x83\x80\x90\xd1\xa8\x81\x56\x6f\xd1\x94\x9d\x65\x34\x95\x79\xc1\x27\xbc\x76\xc3\x5c\xbe" - ) - ] - -tests = - testGroup - "number" - [ testProperty "num-bits" $ \(Int1_2901 i) -> - and - [ (numBits (2 ^ i - 1) == i) - , (numBits (2 ^ i) == i + 1) - , (numBits (2 ^ i + (2 ^ i - 1)) == i + 1) - ] - , testProperty "num-bits2" $ \(Positive i) -> - not (i `testBit` numBits i) && (i `testBit` (numBits i - 1)) - , testProperty "generate-param" $ \testDRG (Int1_2901 bits) -> - let r = withTestDRG testDRG $ generateParams bits (Just SetHighest) False - in r >= 0 && numBits r == bits && testBit r (bits - 1) - , testProperty "generate-param2" $ \testDRG (Int1_2901 m1bits) -> - let bits = m1bits + 1 -- make sure minimum is 2 - r = withTestDRG testDRG $ generateParams bits (Just SetTwoHighest) False - in r >= 0 && numBits r == bits && testBit r (bits - 1) && testBit r (bits - 2) - , testProperty "generate-param-odd" $ \testDRG (Int1_2901 bits) -> - let r = withTestDRG testDRG $ generateParams bits Nothing True - in r >= 0 && odd r - , testProperty "generate-range" $ \testDRG (Positive range) -> - let r = withTestDRG testDRG $ generateMax range - in 0 <= r && r < range - , testProperty "generate-prime" $ \testDRG (Int0_2901 baseBits') -> - let baseBits = baseBits' `mod` 800 - bits = 5 + baseBits -- generating lower than 5 bits causes an error .. - prime = withTestDRG testDRG $ generatePrime bits - in bits == numBits prime - , testProperty "generate-safe-prime" $ \testDRG (Int0_2901 baseBits') -> - let baseBits = baseBits' `mod` 200 - bits = 6 + baseBits - prime = withTestDRG testDRG $ generateSafePrime bits - in bits == numBits prime - , testProperty "as-power-of-2-and-odd" $ \n -> - let (e, a1) = asPowerOf2AndOdd n - in n == (2 ^ e) * a1 - , testProperty "squareRoot" $ \testDRG (Int0_2901 baseBits') -> do - let baseBits = baseBits' `mod` 500 - bits = 5 + baseBits -- generating lower than 5 bits causes an error .. - p = withTestDRG testDRG $ generatePrime bits - g <- choose (1, p - 1) - let square x = (x * x) `mod` p - r = square <$> squareRoot p g - case jacobi g p of - Just 1 -> return $ Just g `assertEq` r - Just (-1) -> return $ Nothing `assertEq` r - _ -> error "invalid jacobi result" - , testProperty "marshalling-be" $ \qaInt -> - getQAInteger qaInt == BE.os2ip (BE.i2osp (getQAInteger qaInt) :: Bytes) - , testProperty "marshalling-le" $ \qaInt -> - getQAInteger qaInt == LE.os2ip (LE.i2osp (getQAInteger qaInt) :: Bytes) - , testProperty "be-rev-le" $ \qaInt -> - getQAInteger qaInt - == LE.os2ip (B.reverse (BE.i2osp (getQAInteger qaInt) :: Bytes)) - , testProperty "be-rev-le-40" $ \qaInt -> - getQAInteger qaInt - == LE.os2ip (B.reverse (BE.i2ospOf_ 40 (getQAInteger qaInt) :: Bytes)) - , testProperty "le-rev-be" $ \qaInt -> - getQAInteger qaInt - == BE.os2ip (B.reverse (LE.i2osp (getQAInteger qaInt) :: Bytes)) - , testProperty "le-rev-be-40" $ \qaInt -> - getQAInteger qaInt - == BE.os2ip (B.reverse (LE.i2ospOf_ 40 (getQAInteger qaInt) :: Bytes)) - , testGroup "marshalling-kat-to-bytearray" $ - zipWith toSerializationKat [katZero ..] serializationVectors - , testGroup "marshalling-kat-to-integer" $ - zipWith toSerializationKatInteger [katZero ..] serializationVectors - ] - where - toSerializationKat i (sz, n, ba) = testCase (show i) (ba @=? BE.i2ospOf_ sz n) - toSerializationKatInteger i (_, n, ba) = testCase (show i) (n @=? BE.os2ip ba) diff --git a/tests/Number/F2m.hs b/tests/Number/F2m.hs deleted file mode 100644 --- a/tests/Number/F2m.hs +++ /dev/null @@ -1,127 +0,0 @@ -module Number.F2m (tests) where - -import Crypto.Number.Basic (log2) -import Crypto.Number.F2m -import Data.Bits -import Data.Maybe -import Imports hiding ((.&.)) - -addTests = - testGroup - "addF2m" - [ testProperty "commutative" $ - \a b -> a `addF2m` b == b `addF2m` a - , testProperty "associative" $ - \a b c -> (a `addF2m` b) `addF2m` c == a `addF2m` (b `addF2m` c) - , testProperty "0 is neutral" $ - \a -> a `addF2m` 0 == a - , testProperty "nullable" $ - \a -> a `addF2m` a == 0 - , testProperty "works per bit" $ - \a b -> (a `addF2m` b) .&. b == (a .&. b) `addF2m` b - ] - -modTests = - testGroup - "modF2m" - [ testProperty "idempotent" $ - \(Positive m) (NonNegative a) -> modF2m m a == modF2m m (modF2m m a) - , testProperty "upper bound" $ - \(Positive m) (NonNegative a) -> modF2m m a < 2 ^ log2 m - , testProperty "reach upper" $ - \(Positive m) -> let a = 2 ^ log2 m - 1 in modF2m m (m `addF2m` a) == a - , testProperty "lower bound" $ - \(Positive m) (NonNegative a) -> modF2m m a >= 0 - , testProperty "reach lower" $ - \(Positive m) -> modF2m m m == 0 - , testProperty "additive" $ - \(Positive m) (NonNegative a) (NonNegative b) -> - modF2m m a `addF2m` modF2m m b == modF2m m (a `addF2m` b) - ] - -mulTests = - testGroup - "mulF2m" - [ testProperty "commutative" $ - \(Positive m) (NonNegative a) (NonNegative b) -> mulF2m m a b == mulF2m m b a - , testProperty "associative" $ - \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> - mulF2m m (mulF2m m a b) c == mulF2m m a (mulF2m m b c) - , testProperty "1 is neutral" $ - \(Positive m) (NonNegative a) -> mulF2m m a 1 == modF2m m a - , testProperty "0 is annihilator" $ - \(Positive m) (NonNegative a) -> mulF2m m a 0 == 0 - , testProperty "distributive" $ - \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> - mulF2m m a (b `addF2m` c) == mulF2m m a b `addF2m` mulF2m m a c - ] - -squareTests = - testGroup - "squareF2m" - [ testProperty "sqr(a) == a * a" $ - \(Positive m) (NonNegative a) -> mulF2m m a a == squareF2m m a - , -- disabled because we require @m@ to be a suitable modulus and there is no - -- way to guarantee this - -- , testProperty "sqrt(a) * sqrt(a) = a" - -- $ \(Positive m) (NonNegative aa) -> let a = sqrtF2m m aa in mulF2m m a a == modF2m m aa - testProperty "sqrt(a) * sqrt(a) = a in GF(2^16)" $ - let m = 65581 :: Integer -- x^16 + x^5 + x^3 + x^2 + 1 - nums = [0 .. 65535 :: Integer] - in nums == [let y = sqrtF2m m x in squareF2m m y | x <- nums] - ] - -powTests = - testGroup - "powF2m" - [ testProperty "2 is square" $ - \(Positive m) (NonNegative a) -> powF2m m a 2 == squareF2m m a - , testProperty "1 is identity" $ - \(Positive m) (NonNegative a) -> powF2m m a 1 == modF2m m a - , testProperty "0 is annihilator" $ - \(Positive m) (NonNegative a) -> powF2m m a 0 == modF2m m 1 - , testProperty "(a * b) ^ c == (a ^ c) * (b ^ c)" $ - \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> - powF2m m (mulF2m m a b) c == mulF2m m (powF2m m a c) (powF2m m b c) - , testProperty "a ^ (b + c) == (a ^ b) * (a ^ c)" $ - \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> - powF2m m a (b + c) == mulF2m m (powF2m m a b) (powF2m m a c) - , testProperty "a ^ (b * c) == (a ^ b) ^ c" $ - \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> - powF2m m a (b * c) == powF2m m (powF2m m a b) c - ] - -invTests = - testGroup - "invF2m" - [ testProperty "1 / a * a == 1" $ - \(Positive m) (NonNegative a) -> - maybe True (\c -> mulF2m m c a == modF2m m 1) (invF2m m a) - , testProperty "1 / a == a (mod a^2-1)" $ - \(NonNegative a) -> a < 2 || invF2m (squareF2m' a `addF2m` 1) a == Just a - ] - -divTests = - testGroup - "divF2m" - [ testProperty "1 / a == inv a" $ - \(Positive m) (NonNegative a) -> divF2m m 1 a == invF2m m a - , testProperty "a / b == a * inv b" $ - \(Positive m) (NonNegative a) (NonNegative b) -> - divF2m m a b == (mulF2m m a <$> invF2m m b) - , testProperty "a * b / b == a" $ - \(Positive m) (NonNegative a) (NonNegative b) -> - isNothing (invF2m m b) || divF2m m (mulF2m m a b) b == Just (modF2m m a) - ] - -tests = - testGroup - "number.F2m" - [ addTests - , modTests - , mulTests - , squareTests - , powTests - , invTests - , divTests - ] diff --git a/tests/Number/F2mSpec.hs b/tests/Number/F2mSpec.hs new file mode 100644 --- /dev/null +++ b/tests/Number/F2mSpec.hs @@ -0,0 +1,111 @@ +module Number.F2mSpec (spec) where + +import Crypto.Number.Basic (log2) +import Crypto.Number.F2m +import Data.Bits +import Data.Maybe +import Imports hiding ((.&.)) + +addTests = + describe "addF2m" $ do + prop "commutative" $ + \a b -> a `addF2m` b == b `addF2m` a + prop "associative" $ + \a b c -> (a `addF2m` b) `addF2m` c == a `addF2m` (b `addF2m` c) + prop "0 is neutral" $ + \a -> a `addF2m` 0 == a + prop "nullable" $ + \a -> a `addF2m` a == 0 + prop "works per bit" $ + \a b -> (a `addF2m` b) .&. b == (a .&. b) `addF2m` b + +modTests = + describe "modF2m" $ do + prop "idempotent" $ + \(Positive m) (NonNegative a) -> modF2m m a == modF2m m (modF2m m a) + prop "upper bound" $ + \(Positive m) (NonNegative a) -> modF2m m a < 2 ^ log2 m + prop "reach upper" $ + \(Positive m) -> let a = 2 ^ log2 m - 1 in modF2m m (m `addF2m` a) == a + prop "lower bound" $ + \(Positive m) (NonNegative a) -> modF2m m a >= 0 + prop "reach lower" $ + \(Positive m) -> modF2m m m == 0 + prop "additive" $ + \(Positive m) (NonNegative a) (NonNegative b) -> + modF2m m a `addF2m` modF2m m b == modF2m m (a `addF2m` b) + +mulTests = + describe "mulF2m" $ do + prop "commutative" $ + \(Positive m) (NonNegative a) (NonNegative b) -> mulF2m m a b == mulF2m m b a + prop "associative" $ + \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> + mulF2m m (mulF2m m a b) c == mulF2m m a (mulF2m m b c) + prop "1 is neutral" $ + \(Positive m) (NonNegative a) -> mulF2m m a 1 == modF2m m a + prop "0 is annihilator" $ + \(Positive m) (NonNegative a) -> mulF2m m a 0 == 0 + prop "distributive" $ + \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> + mulF2m m a (b `addF2m` c) == mulF2m m a b `addF2m` mulF2m m a c + +squareTests = + describe "squareF2m" $ do + prop "sqr(a) == a * a" $ + \(Positive m) (NonNegative a) -> mulF2m m a a == squareF2m m a + -- disabled because we require @m@ to be a suitable modulus and there is no + -- way to guarantee this + -- , prop "sqrt(a) * sqrt(a) = a" + -- $ \(Positive m) (NonNegative aa) -> let a = sqrtF2m m aa in mulF2m m a a == modF2m m aa + prop "sqrt(a) * sqrt(a) = a in GF(2^16)" $ + let m = 65581 :: Integer -- x^16 + x^5 + x^3 + x^2 + 1 + nums = [0 .. 65535 :: Integer] + in nums == [let y = sqrtF2m m x in squareF2m m y | x <- nums] + +powTests = + describe "powF2m" $ do + prop "2 is square" $ + \(Positive m) (NonNegative a) -> powF2m m a 2 == squareF2m m a + prop "1 is identity" $ + \(Positive m) (NonNegative a) -> powF2m m a 1 == modF2m m a + prop "0 is annihilator" $ + \(Positive m) (NonNegative a) -> powF2m m a 0 == modF2m m 1 + prop "(a * b) ^ c == (a ^ c) * (b ^ c)" $ + \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> + powF2m m (mulF2m m a b) c == mulF2m m (powF2m m a c) (powF2m m b c) + prop "a ^ (b + c) == (a ^ b) * (a ^ c)" $ + \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> + powF2m m a (b + c) == mulF2m m (powF2m m a b) (powF2m m a c) + prop "a ^ (b * c) == (a ^ b) ^ c" $ + \(Positive m) (NonNegative a) (NonNegative b) (NonNegative c) -> + powF2m m a (b * c) == powF2m m (powF2m m a b) c + +invTests = + describe "invF2m" $ do + prop "1 / a * a == 1" $ + \(Positive m) (NonNegative a) -> + maybe True (\c -> mulF2m m c a == modF2m m 1) (invF2m m a) + prop "1 / a == a (mod a^2-1)" $ + \(NonNegative a) -> a < 2 || invF2m (squareF2m' a `addF2m` 1) a == Just a + +divTests = + describe "divF2m" $ do + prop "1 / a == inv a" $ + \(Positive m) (NonNegative a) -> divF2m m 1 a == invF2m m a + prop "a / b == a * inv b" $ + \(Positive m) (NonNegative a) (NonNegative b) -> + divF2m m a b == (mulF2m m a <$> invF2m m b) + prop "a * b / b == a" $ + \(Positive m) (NonNegative a) (NonNegative b) -> + isNothing (invF2m m b) || divF2m m (mulF2m m a b) b == Just (modF2m m a) + +spec :: Spec +spec = do + addTests + modTests + mulTests + squareTests + powTests + invTests + divTests diff --git a/tests/NumberSpec.hs b/tests/NumberSpec.hs new file mode 100644 --- /dev/null +++ b/tests/NumberSpec.hs @@ -0,0 +1,319 @@ +{-# LANGUAGE OverloadedStrings #-} + +module NumberSpec (spec) where + +import Imports + +import Crypto.Number.Basic +import Crypto.Number.Generate +import Crypto.Number.ModArithmetic +import Crypto.Number.Prime +import qualified Crypto.Number.Serialize as BE +import qualified Crypto.Number.Serialize.LE as LE +import Data.Bits +import Data.ByteArray (Bytes) +import qualified Data.ByteArray as B + +serializationVectors :: [(Int, Integer, ByteString)] +serializationVectors = + [ + ( 128 + , 468189858948067662094510918729062682059955669513914188715630930503497261316361784677177564296207557978182700664806717692596876084916561811001371208806217360635705059859428069669992937334724312890015700331031248133952795914192719979937664050389500162437642525331653766885896869239678885404647468665996400635 + , "\x00\xaa\xae\x74\xc8\xec\x3c\x36\x06\x5e\x46\xca\x8e\x57\xab\x09\x87\xfd\xcd\x1f\xa4\xe7\xf9\xd2\x60\xd5\x4a\x1b\x74\xdc\xa8\x75\xd8\xdd\xff\x2b\x74\x28\x14\x59\x67\x6c\x82\xae\xa3\xa5\x1d\x3f\xb4\xb7\xfe\x5c\xd2\xf0\x7f\xd8\xd9\xa9\xb0\xce\x26\xc1\x26\x74\x96\xf5\xf6\x4c\x8f\x66\x7f\x5d\xf1\x68\x38\xd4\x03\x62\xe9\x30\xc8\xa1\xc1\x84\x97\x62\x20\xfd\xd7\x03\x35\xc1\x25\x45\x1b\x86\x81\x3d\xa4\x92\xc0\xd3\xdd\xfa\x86\x1d\xdf\x0a\xbb\xf4\xc0\x56\xf7\xa2\xb0\x3b\x52\xf7\xa5\x89\x4c\x69\x34\x91\x46\xd9\x57\xfb" + ) + , + ( 128 + , 40031303476923779996794876613623495515025748694978019540894726181695410095832601107261950025830235596060960914255795497479135806963313279476038687192202016132891881954743054164975707083302554941058329647014950354509055121290280892911153779672733723699997592027662953953692834215577119173225643193201177329 + , "\x00\x0e\x97\xf9\xd5\x79\xb9\x90\x7c\x85\x48\x49\x01\x19\x64\xfb\x76\x31\xcd\x51\xfb\x8a\x9d\x55\xe5\xd3\x7b\x87\x2d\xad\x63\x2d\x6b\x1c\x84\x3f\x65\x95\xb6\xf3\x1a\xa9\x43\x3f\x06\x46\x7b\xf8\xf3\x35\x45\x84\x11\x56\x91\x53\x43\xd7\xe1\x6d\x80\x64\x14\x45\x35\x4e\x93\x7d\x5e\x48\xec\xe0\x79\x7b\x44\x8e\xab\x0f\xc4\x5f\xc6\xa1\x71\xee\x37\xb1\x55\x51\x98\x44\x57\xe3\xc3\x56\x3a\x50\x27\xaf\xa5\x1d\x1a\x0a\x90\x19\x0d\x14\xed\x3d\x93\x40\x62\x76\xa3\xaa\x00\x23\x86\xca\x98\xb2\x6e\x02\x43\xa7\xbc\xb1\xb2\xf1" + ) + , + ( 128 + , 75152325976543603337003024341071663845101857195436434620947904288957274825323005869230041326941600298094896018190395352332646796347130114769768242670539699217743549573961461985255265474392937773768121046339453584830072421569334022498680626938734088755136253492360177084153487115846920446085149631919580041 + , "\x00\x1b\x65\xb1\x73\x74\xed\xd2\xcb\xb8\xf3\x6b\x3f\xc2\x05\xaa\x91\xab\x48\x5b\x03\x30\xae\x24\xa3\xec\x7a\x6a\xf0\x34\x73\x18\x04\xea\xe4\xd6\x19\x97\xc4\xc1\x13\x7d\x12\x0d\xd5\xcb\xbd\x18\x05\xc2\xce\x87\x66\x84\x12\xe8\x24\xa3\x31\x69\xfa\xf4\x2c\x21\x53\xa6\x04\x74\x78\xc4\x93\x0d\x38\x7f\x28\xfe\x80\x8e\xd2\x7b\x20\xc8\xf5\x1f\x0f\x73\x68\xb2\xe5\x08\xf1\x94\xa1\xe6\xcf\x3a\x2c\x12\x63\xda\x08\x3a\x78\x12\xb8\x11\x23\x3c\x38\x38\x10\x94\x2b\xac\x64\x5d\x67\x0c\xb6\x0d\xc3\x9a\x45\x39\x50\x8a\x63\x89" + ) + , + ( 128 + , 132094272981815297755209818914225029878347650582749561568514551350741192910991391836297682842650690115955454061006435646226436379226218676796260483719213285072886626400953065229934239690821114513313427305727000011361769875430428291375851099221794646192854831002408178061474948738788927399080262963320752452 + , "\x00\x30\x27\xe0\xbf\x46\xec\x77\x2d\xc6\x06\x77\xbc\x68\x87\x3c\x1b\x2e\xc7\xb7\x6c\x88\x25\xec\x8c\x95\xbf\x74\xe5\x37\x01\x25\x96\xe1\x70\x33\x5c\x7d\xab\x1f\xc2\x9c\xad\xf7\xca\x26\x85\x2d\xfc\x8f\xc7\xab\x49\x28\xa4\x47\xe6\xd5\x6e\xfa\x0a\xbb\x57\xe4\xa2\x51\xc7\xc6\x12\x0f\xa9\x98\x69\xb8\x05\x84\xc5\xe3\x28\x86\x0f\x54\x1d\xf9\x92\x42\x9f\xb1\x77\x2b\x58\x89\xe2\xfc\x22\xb0\x1e\x71\x78\xea\x39\xc1\x87\x4f\xd4\x83\x2c\x96\x1d\xea\xd5\xf9\xf9\xb9\x7b\x86\xfa\xf6\xad\x5b\xb1\x3c\xe7\x11\xd7\x96\x89\x44" + ) + , + ( 128 + , 577245873336454863811643140721674509319073059708446946821011267146688442860798353087462545395033001525475835015592425207995480357299993009193426638306801669333644226765032464458284920004140299209138389393494751627076239104390434285377314678827349631962212281858308570255468721491493027423799738158196939966 + , "\x00\xd2\x70\x41\xdb\x3d\xb5\xfe\x8c\xef\x79\xcf\x5b\x7b\x37\xb0\x05\xb8\x5a\x9b\x7d\x01\x28\xc7\xf5\x5a\x02\xba\xce\xbc\xf5\x8e\x91\x59\xd0\x42\x6f\x04\x82\x4b\x78\xb0\xdd\x91\x2e\x15\x9d\xea\x4f\x0c\x21\xc0\x67\x54\xa2\x39\xa8\xe1\x13\x8f\xa9\xff\x46\x2d\x11\x56\x04\xa0\xde\x64\xc8\x0f\xf4\x2c\xd2\x31\xdf\x2a\xfd\xac\xc7\x25\x58\xc8\xea\xfd\x47\x6e\xdd\x2a\x53\x02\x77\x49\xa7\x0d\x18\xfb\x05\x18\x4b\x28\xd3\xa2\x39\x8c\x83\x80\x90\xd1\xa8\x81\x56\x6f\xd1\x94\x9d\x65\x34\x95\x79\xc1\x27\xbc\x76\xc3\x5c\xbe" + ) + ] + +-- | Numbers built to fool a primality test: the Carmichael numbers, which +-- pass a Fermat test to every base coprime with them, and the strong +-- pseudoprimes to base 2. +carmichaels :: [Integer] +carmichaels = + [ 561 + , 1105 + , 1729 + , 2465 + , 2821 + , 6601 + , 8911 + , 10585 + , 15841 + , 29341 + , 41041 + , 46657 + , 52633 + , 62745 + , 63973 + , 75361 + , 101101 + , 115921 + , 126217 + , 162401 + ] + +strongPseudoprimesBase2 :: [Integer] +strongPseudoprimesBase2 = + [ 2047 + , 3277 + , 4033 + , 4681 + , 8321 + , 15841 + , 29341 + , 42799 + , 49141 + , 52633 + , 65281 + , 74665 + , 80581 + , 85489 + , 88357 + , 90751 + ] + +-- a 512-bit prime, and a 512-bit composite that is the product of two primes +bigPrime, bigComposite :: Integer +bigPrime = 2 ^ (512 :: Int) - 569 +bigComposite = (2 ^ (256 :: Int) - 189) * (2 ^ (256 :: Int) - 357) + +-- the index is threaded through so that repeated calls cannot be shared +askAgain :: Int -> Integer -> Bool +askAgain i n = i `seq` primalityTestMillerRabin 1 n +{-# NOINLINE askAgain #-} + +-- | Miller-Rabin draws witnesses from a generator this library builds itself, +-- so check the answers it reaches: against trial division over a range, over +-- the numbers built to fool such a test, and at a size a key would use. The +-- last test is about the generator rather than the arithmetic: a pure function +-- has to give one answer, so the same number asked many times has to reach the +-- same verdict. +primalityTests :: Spec +primalityTests = describe "primality" $ do + it "agrees with trial division on the odd numbers from 5 to 5001" $ + [n | n <- [5, 7 .. 5001], primalityTestMillerRabin 30 n /= primalityTestNaive n] + `shouldBe` [] + it "calls the Carmichael numbers composite" $ + filter (primalityTestMillerRabin 30) carmichaels `shouldBe` [] + it "calls the strong pseudoprimes to base 2 composite" $ + filter (primalityTestMillerRabin 30) strongPseudoprimesBase2 `shouldBe` [] + it "sees through them from isProbablyPrime too" $ + filter isProbablyPrime (carmichaels ++ strongPseudoprimesBase2) `shouldBe` [] + it "calls a 512-bit prime prime and a 512-bit composite composite" $ do + primalityTestMillerRabin 30 bigPrime `shouldBe` True + primalityTestMillerRabin 30 bigComposite `shouldBe` False + it "answers for the small numbers and the edges of the shortcut" $ do + -- below two, nothing is prime, and the list of small primes answers up + -- to its own end at 2903; past that the Miller-Rabin path takes over, + -- and it has no answer for anything below four + filter isProbablyPrime [-3, -1, 0, 1, 4, 6, 8, 9, 2911] `shouldBe` [] + filter (not . isProbablyPrime) [2, 3, 5, 7, 2897, 2903, 2909, 2917] + `shouldBe` [] + it "reaches the same verdict every time it is asked" $ + map (`askAgain` 2465) [1 .. 20] `shouldBe` replicate 20 (askAgain 0 2465) + +-- | The two exponentiations have to agree on every shape of argument: the +-- safe one is only meant to differ in how it spends its time. The pairs are +-- (base, exponent, modulus), and cover a zero exponent, a zero base, a base +-- above the modulus, a negative base, a negative exponent (which GMP reads as +-- a request for the inverse), a modulus of one, and an even modulus, which +-- sends expSafe down the fast path. +exponentiationCorners :: [(Integer, Integer, Integer)] +exponentiationCorners = + [ (2, 3, 1) + , (0, 0, 7) + , (0, 5, 7) + , (1, 0, 7) + , (9, 3, 7) + , (-2, 3, 7) + , (3, -1, 7) + , (2, 3, 8) + , (2, 0, 9) + , (5, 1, 3) + , (2, 256, 255) + , (bigPrime, bigPrime - 2, bigComposite + 1) + , (bigComposite, 65537, bigPrime) + , (bigPrime + 1, 2 ^ (600 :: Int), bigPrime) + , (3, 2 * bigPrime * bigComposite, 2 * bigPrime * bigComposite + 1) + ] + +exponentiationTests :: Spec +exponentiationTests = describe "exponentiation" $ do + it "agrees with the fast one on the corners" $ + map safely exponentiationCorners `shouldBe` map fastly exponentiationCorners + it "agrees with repetitive squaring on the corners" $ + [ (b, e, m) + | (b, e, m) <- exponentiationCorners + , e >= 0 + , m > 1 + , safely (b, e, m) /= naivePow (b `mod` m) e m + ] + `shouldBe` [] + prop "agrees with the fast one" $ \(QAInteger b) (QAInteger e) (QAInteger m') -> + let m = abs m' + 1 + in expSafe b (abs e) m === expFast b (abs e) m + prop "agrees with the fast one on a modulus a key would have" $ + \(QAInteger b) (QAInteger e) -> + let m = 2 * bigPrime * bigComposite + 1 -- odd, and 1025 bits + in expSafe b (abs e) m === expFast b (abs e) m + where + safely (b, e, m) = expSafe b e m + fastly (b, e, m) = expFast b e m + -- an answer owing nothing to the library, for the corners to be held to + naivePow b e m = foldl (\acc isSet -> acc * acc * (if isSet then b else 1) `mod` m) 1 bits + where + bits = [testBit e i | i <- [numBits e - 1, numBits e - 2 .. 0]] + +-- | The safe inverse has to answer exactly what the ordinary one answers, on +-- a prime modulus, where it works out the inverse by Fermat, and on every +-- other one, where that answer is not an inverse and it has to notice and ask +-- the ordinary one instead. +inverseTests :: Spec +inverseTests = describe "inverse" $ do + it "agrees with the plain inverse on the corners" $ + [ (g, m) + | (g, m) <- + [ (0, 1) + , (0, 7) + , (1, 7) + , (3, 7) + , (7, 7) + , (8, 7) + , (3, 9) + , (6, 9) + , (2, 8) + , (4, 8) + , (5, 8) + , (-3, 7) + , (bigPrime, bigComposite) + , (bigComposite, bigPrime) + ] + , inverseSafe g m /= inverse g m + ] + `shouldBe` [] + prop "agrees with the plain inverse" $ \(QAInteger g) (QAInteger m') -> + let m = abs m' + 1 + in inverseSafe g m === inverse g m + prop "agrees with the plain inverse on a prime modulus" $ \(QAInteger g) -> + inverseSafe g bigPrime === inverse g bigPrime + prop "inverts" $ \(QAInteger g) -> + let g' = g `mod` bigPrime + in g' + /= 0 ==> fmap (\i -> g' * i `mod` bigPrime) (inverseSafe g' bigPrime) === Just 1 + +spec :: Spec +spec = do + primalityTests + exponentiationTests + inverseTests + prop "num-bits" $ \(Int1_2901 i) -> + and + [ (numBits (2 ^ i - 1) == i) + , (numBits (2 ^ i) == i + 1) + , (numBits (2 ^ i + (2 ^ i - 1)) == i + 1) + ] + prop "num-bits2" $ \(Positive i) -> + not (i `testBit` numBits i) && (i `testBit` (numBits i - 1)) + -- how many bytes it takes to write the number, which is what every + -- serialization here asks before it allocates. Held to counting the + -- divisions, so that it is not the same expression on both sides. + prop "num-bytes" $ \(Positive i) -> + numBytes i == byteCount i + prop "num-bytes-small" $ \() -> + map numBytes [0, 1, 255, 256, 257, 65535, 65536] == [0, 1, 1, 2, 2, 2, 3] + prop "generate-param" $ \testDRG (Int1_2901 bits) -> + let r = withTestDRG testDRG $ generateParams bits (Just SetHighest) False + in r >= 0 && numBits r == bits && testBit r (bits - 1) + prop "generate-param2" $ \testDRG (Int1_2901 m1bits) -> + let bits = m1bits + 1 -- make sure minimum is 2 + r = withTestDRG testDRG $ generateParams bits (Just SetTwoHighest) False + in r >= 0 && numBits r == bits && testBit r (bits - 1) && testBit r (bits - 2) + prop "generate-param-odd" $ \testDRG (Int1_2901 bits) -> + let r = withTestDRG testDRG $ generateParams bits Nothing True + in r >= 0 && odd r + prop "generate-range" $ \testDRG (Positive range) -> + let r = withTestDRG testDRG $ generateMax range + in 0 <= r && r < range + prop "generate-prime" $ \testDRG (Int0_2901 baseBits') -> + let baseBits = baseBits' `mod` 800 + bits = 5 + baseBits -- generating lower than 5 bits causes an error .. + prime = withTestDRG testDRG $ generatePrime bits + in bits == numBits prime + -- what generatePrime settles on has to answer to the test anyone else + -- would put it to, whatever it did to convince itself + prop "generate-prime-is-prime" $ \testDRG (Int0_2901 baseBits') -> + let baseBits = baseBits' `mod` 800 + bits = 5 + baseBits + prime = withTestDRG testDRG $ generatePrime bits + in isProbablyPrime prime + prop "generate-safe-prime-is-prime" $ \testDRG (Int0_2901 baseBits') -> + let baseBits = baseBits' `mod` 200 + bits = 6 + baseBits + prime = withTestDRG testDRG $ generateSafePrime bits + in isProbablyPrime prime && isProbablyPrime ((prime - 1) `div` 2) + prop "generate-safe-prime" $ \testDRG (Int0_2901 baseBits') -> + let baseBits = baseBits' `mod` 200 + bits = 6 + baseBits + prime = withTestDRG testDRG $ generateSafePrime bits + in bits == numBits prime + prop "as-power-of-2-and-odd" $ \n -> + let (e, a1) = asPowerOf2AndOdd n + in n == (2 ^ e) * a1 + prop "squareRoot" $ \testDRG (Int0_2901 baseBits') -> do + let baseBits = baseBits' `mod` 500 + bits = 5 + baseBits -- generating lower than 5 bits causes an error .. + p = withTestDRG testDRG $ generatePrime bits + g <- choose (1, p - 1) + let square x = (x * x) `mod` p + r = square <$> squareRoot p g + case jacobi g p of + Just 1 -> return $ Just g `assertEq` r + Just (-1) -> return $ Nothing `assertEq` r + _ -> error "invalid jacobi result" + prop "marshalling-be" $ \qaInt -> + getQAInteger qaInt == BE.os2ip (BE.i2osp (getQAInteger qaInt) :: Bytes) + prop "marshalling-le" $ \qaInt -> + getQAInteger qaInt == LE.os2ip (LE.i2osp (getQAInteger qaInt) :: Bytes) + prop "be-rev-le" $ \qaInt -> + getQAInteger qaInt + == LE.os2ip (B.reverse (BE.i2osp (getQAInteger qaInt) :: Bytes)) + prop "be-rev-le-40" $ \qaInt -> + getQAInteger qaInt + == LE.os2ip (B.reverse (BE.i2ospOf_ 40 (getQAInteger qaInt) :: Bytes)) + prop "le-rev-be" $ \qaInt -> + getQAInteger qaInt + == BE.os2ip (B.reverse (LE.i2osp (getQAInteger qaInt) :: Bytes)) + prop "le-rev-be-40" $ \qaInt -> + getQAInteger qaInt + == BE.os2ip (B.reverse (LE.i2ospOf_ 40 (getQAInteger qaInt) :: Bytes)) + describe "marshalling-kat-to-bytearray" $ + sequence_ $ + zipWith toSerializationKat [katZero ..] serializationVectors + describe "marshalling-kat-to-integer" $ + sequence_ $ + zipWith toSerializationKatInteger [katZero ..] serializationVectors + where + toSerializationKat i (sz, n, ba) = it (show i) (BE.i2ospOf_ sz n `shouldBe` ba) + toSerializationKatInteger i (_, n, ba) = it (show i) (BE.os2ip ba `shouldBe` n) + +-- | How many bytes the number takes, by taking them off one at a time. +byteCount :: Integer -> Int +byteCount = length . takeWhile (> 0) . iterate (`div` 256) diff --git a/tests/OTPSpec.hs b/tests/OTPSpec.hs new file mode 100644 --- /dev/null +++ b/tests/OTPSpec.hs @@ -0,0 +1,231 @@ +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE OverloadedStrings #-} + +module OTPSpec ( + spec, +) +where + +import Control.Exception (evaluate) +import Crypto.Hash.Algorithms ( + Blake2b (..), + MD5 (..), + SHA1 (..), + SHA256 (..), + SHA512 (..), + ) +import Crypto.OTP +import qualified Crypto.OTP as TOTP +import Data.Either (isLeft) +import Imports + +-- | Test values from Appendix D of http://tools.ietf.org/html/rfc4226 +hotpExpected :: [(Word64, Word32)] +hotpExpected = + [ (0, 755224) + , (1, 287082) + , (3, 969429) + , (4, 338314) + , (5, 254676) + , (6, 287922) + , (7, 162583) + , (8, 399871) + , (9, 520489) + ] + +-- | Test data from Appendix B of http://tools.ietf.org/html/rfc6238 +-- Note that the shared keys for the non SHA-1 values are actually +-- different (see the errata, or the Java example code). +totpSHA1Expected :: [(Word64, Word32)] +totpSHA1Expected = + [ (59, 94287082) + , (1111111109, 07081804) + , (1111111111, 14050471) + , (1234567890, 89005924) + , (2000000000, 69279037) + , (20000000000, 65353130) + ] + +totpSHA256Expected :: [(Word64, Word32)] +totpSHA256Expected = + [ (59, 46119246) + , (1111111109, 68084774) + , (1111111111, 67062674) + , (1234567890, 91819424) + , (2000000000, 90698825) + , (20000000000, 77737706) + ] + +totpSHA512Expected :: [(Word64, Word32)] +totpSHA512Expected = + [ (59, 90693936) + , (1111111109, 25091201) + , (1111111111, 99943326) + , (1234567890, 93441116) + , (2000000000, 38618901) + , (20000000000, 47863826) + ] + +otpKey :: ByteString +otpKey = "12345678901234567890" + +totpSHA256Key :: ByteString +totpSHA256Key = "12345678901234567890123456789012" + +totpSHA512Key :: ByteString +totpSHA512Key = + "1234567890123456789012345678901234567890123456789012345678901234" + +makeKATs :: (Eq a, Show a) => (t -> a) -> [(t, a)] -> [Spec] +makeKATs otp expected = concatMap (makeTest otp) (zip3 is counts otps) + where + is :: [Int] + is = [1 ..] + + counts = map fst expected + otps = map snd expected + +makeTest :: (Eq a1, Show a2, Show a1) => (t -> a1) -> (a2, t, a1) -> [Spec] +makeTest otp (i, count, password) = + [ it (show i) (assertEqual "" password (otp count)) + ] + +totpSHA1Params :: TOTPParams SHA1 +totpSHA1Params = case mkTOTPParams SHA1 0 30 OTP8 TwoSteps of + Right x -> x + _ -> error "totpSHA1Params" + +totpSHA256Params :: TOTPParams SHA256 +totpSHA256Params = case mkTOTPParams SHA256 0 30 OTP8 TwoSteps of + Right x -> x + _ -> error "totpSHA256Params" + +totpSHA512Params :: TOTPParams SHA512 +totpSHA512Params = case mkTOTPParams SHA512 0 30 OTP8 TwoSteps of + Right x -> x + _ -> error "totpSHA512Params" + +-- resynching with the expected value should just return the current counter + 1 +prop_resyncExpected :: Word64 -> Word16 -> Bool +prop_resyncExpected ctr window = resynchronize SHA1 OTP6 window key ctr (otp, []) == Just (ctr + 1) + where + key = "1234" :: ByteString + otp = hotp SHA1 OTP6 key ctr + +-- | RFC 4226 dynamic truncation reads the offset from the low four bits of +-- the MAC's last byte, so the offset can be any of 0..15, and then reads four +-- bytes starting there -- reaching byte 18. A digest shorter than that leaves +-- 'hotp' indexing past the end of the MAC, and 'Data.ByteArray.index' does not +-- bounds check, so the OTP is built from whatever happens to follow the MAC in +-- memory. Such a digest must be refused instead. +digestSizeTests :: [Spec] +digestSizeTests = + [ it "SHA-1 (20 bytes) is accepted" $ + hotp SHA1 OTP6 otpKey 1 `shouldBe` 287082 + , rejects "MD5 (16 bytes)" (hotp MD5 OTP6 otpKey 1) + , rejects "Blake2b-64 (8 bytes)" (hotp (Blake2b :: Blake2b 64) OTP6 otpKey 1) + , it "resynchronize with a short digest is rejected" $ + evaluate (resynchronize MD5 OTP6 10 otpKey 0 (0, [])) + `shouldThrow` anyErrorCall + , it "mkTOTPParams rejects a short digest" $ + mkTOTPParams MD5 0 30 OTP6 TwoSteps `shouldSatisfy` isLeft + ] + where + rejects name otp = + it (name ++ " is rejected") $ evaluate otp `shouldThrow` anyErrorCall + +-- | resynchronize hunts for the client's counter in a window of values +-- derived from the shared secret, and reports how far it got only through the +-- counter it returns. Pin down which submissions it accepts, which it +-- refuses, and the counter each accepted one leaves behind, before that search +-- is rewritten. +resyncTests :: [Spec] +resyncTests = + [ it "the value for the current counter moves the server on by one" $ + resync 20 (at 0, []) `shouldBe` serverAfter 1 + , it "a value from inside the window is found" $ + resync 20 (at 7, []) `shouldBe` serverAfter 8 + , it "the last value in the window is found" $ + resync 20 (at 20, []) `shouldBe` serverAfter 21 + , it "the value just past the window is not" $ + resync 20 (at 21, []) `shouldBe` Nothing + , it "a value no counter produces is refused" $ + resync 20 (at 0 + 1, []) `shouldBe` Nothing + , it "a window of zero looks at the current counter only" $ do + resync 0 (at 0, []) `shouldBe` serverAfter 1 + resync 0 (at 1, []) `shouldBe` Nothing + , it "the extra values carry the counter past all of them" $ + resync 20 (at 7, [at 8, at 9]) `shouldBe` serverAfter 10 + , it "an extra value that is wrong refuses the whole submission" $ + sequence_ + [ resync 20 (at 7, wrongAt i [at 8, at 9, at 10]) `shouldBe` Nothing + | i <- [0 .. 2] + ] + , it "extra values that are right do not rescue a wrong first value" $ + resync 20 (at 0 + 1, [at 1, at 2]) `shouldBe` Nothing + , it "extra values from the wrong counters are refused" $ + resync 20 (at 7, [at 9, at 10]) `shouldBe` Nothing + ] + where + ctr = 1000 + resync w submitted = resynchronize SHA1 OTP6 w otpKey ctr submitted + -- the value the client would show at the counter n ahead of the server's + at n = hotp SHA1 OTP6 otpKey (ctr + n) + -- the server counter n ahead of where it started + serverAfter n = Just (ctr + n) + wrongAt i vs = [if j == i then v + 1 else v | (j, v) <- zip [0 :: Int ..] vs] + +-- | totpVerify accepts a value from any step within the skew window and +-- nothing else. It compares a submitted value against secret-derived ones, so +-- pin the accepted and rejected cases down before that comparison is rewritten. +verifyTests :: [Spec] +verifyTests = + [ it "the value for the current step is accepted" $ + assertBool "expected acceptance" (verifyAt 0) + , it "every step within the window is accepted" $ + assertBool "expected acceptance" (all verifyAt [-2 .. 2]) + , it "the step just outside the window is refused" $ + assertBool "expected refusal" (not (any verifyAt [-3, 3])) + , it "a value no step produces is refused" $ + assertBool "expected refusal" $ + not (totpVerify params otpKey now (totp params otpKey now + 1)) + , it "a window of no skew accepts only the current step" $ + assertBool "expected only the current step" $ + let noSkew = TOTP.mkTOTPParams SHA1 0 30 OTP6 NoSkew + in case noSkew of + Left e -> error e + Right ps -> + totpVerify ps otpKey now (totp ps otpKey now) + && not (totpVerify ps otpKey now (totp ps otpKey (now + 30))) + ] + where + params = defaultTOTPParams + now = 1111111109 + + -- one step is 30 seconds under defaultTOTPParams. The offset is taken as + -- an Integer so a step before the current one is an actual subtraction + -- rather than a wrap around OTPTime, which is a Word64. + verifyAt :: Integer -> Bool + verifyAt steps = + totpVerify params otpKey now (totp params otpKey (at steps)) + at steps = fromInteger (toInteger now + 30 * steps) + +spec :: Spec +spec = do + describe "HOTP" $ do + describe "KATs" $ sequence_ (makeKATs (hotp SHA1 OTP6 otpKey) hotpExpected) + describe "digest size" $ sequence_ digestSizeTests + describe "resynchronize" $ sequence_ resyncTests + describe "properties" $ do + prop "resync-expected" prop_resyncExpected + describe "TOTP" $ do + describe "KATs" $ do + describe "SHA1" $ + sequence_ (makeKATs (totp totpSHA1Params otpKey) totpSHA1Expected) + describe "SHA256" $ + sequence_ $ + (makeKATs (totp totpSHA256Params totpSHA256Key) totpSHA256Expected) + describe "SHA512" $ + sequence_ $ + (makeKATs (totp totpSHA512Params totpSHA512Key) totpSHA512Expected) + describe "verify" $ sequence_ verifyTests diff --git a/tests/Padding.hs b/tests/Padding.hs deleted file mode 100644 --- a/tests/Padding.hs +++ /dev/null @@ -1,46 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module Padding (tests) where - -import qualified Data.ByteString as B -import Imports - -import Crypto.Data.Padding - -cases = - [ ("abcdef", 8, "abcdef\x02\x02") - , ("abcd", 4, "abcd\x04\x04\x04\x04") - , ("xyze", 5, "xyze\x01") - ] - -zeroCases = - [ ("", 4, "\NUL\NUL\NUL\NUL", Nothing) - , ("abcdef", 8, "abcdef\NUL\NUL", Nothing) - , ("0123456789abcdef", 16, "0123456789abcdef", Just "0123456789abcdef") - ] - --- instance Arbitrary where - -testPad :: Int -> (B.ByteString, Int, B.ByteString) -> TestTree -testPad n (inp, sz, padded) = - testCase (show n) $ - propertyHoldCase - [ eqTest "padded" padded (pad (PKCS7 sz) inp) - , eqTest "unpadded" (Just inp) (unpad (PKCS7 sz) padded) - ] - -testZeroPad - :: Int -> (B.ByteString, Int, B.ByteString, Maybe B.ByteString) -> TestTree -testZeroPad n (inp, sz, padded, unpadded) = - testCase (show n) $ - propertyHoldCase - [ eqTest "padded" padded (pad (ZERO sz) inp) - , eqTest "unpadded" unpadded (unpad (ZERO sz) padded) - ] - -tests = - testGroup - "Padding" - [ testGroup "Cases" $ zipWith testPad [1 ..] cases - , testGroup "ZeroCases" $ zipWith testZeroPad [1 ..] zeroCases - ] diff --git a/tests/PaddingSpec.hs b/tests/PaddingSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PaddingSpec.hs @@ -0,0 +1,125 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PaddingSpec (spec) where + +import Control.Exception (ErrorCall (..), evaluate) +import qualified Data.ByteString as B +import Data.List (isInfixOf) +import Imports + +import Crypto.Data.Padding + +cases = + [ ("abcdef", 8, "abcdef\x02\x02") + , ("abcd", 4, "abcd\x04\x04\x04\x04") + , ("xyze", 5, "xyze\x01") + ] + +zeroCases = + [ ("", 4, "\NUL\NUL\NUL\NUL", Nothing) + , ("abcdef", 8, "abcdef\NUL\NUL", Nothing) + , ("0123456789abcdef", 16, "0123456789abcdef", Just "0123456789abcdef") + ] + +-- instance Arbitrary where + +testPad :: Int -> (B.ByteString, Int, B.ByteString) -> Spec +testPad n (inp, sz, padded) = + it (show n) $ + propertyHoldCase + [ eqTest "padded" padded (pad (PKCS7 sz) inp) + , eqTest "unpadded" (Just inp) (unpad (PKCS7 sz) padded) + ] + +testZeroPad + :: Int -> (B.ByteString, Int, B.ByteString, Maybe B.ByteString) -> Spec +testZeroPad n (inp, sz, padded, unpadded) = + it (show n) $ + propertyHoldCase + [ eqTest "padded" padded (pad (ZERO sz) inp) + , eqTest "unpadded" unpadded (unpad (ZERO sz) padded) + ] + +-- | The padding octet of a PKCS7 block carries the number of octets added, so +-- it cannot describe a block longer than 255, and a block of zero has nothing +-- to describe. Outside that range the octet is computed as an Int and then +-- narrowed to a Word8, which wraps: pad and unpad agree on the wrapped value +-- and hand back something that is not what was padded. +blockSizeTests :: Spec +blockSizeTests = describe "PKCS7 block size" $ do + it "round trips at the smallest size" $ + unpad (PKCS7 1) (pad (PKCS7 1) msg) `shouldBe` Just msg + it "round trips at the largest size" $ + unpad (PKCS7 255) (pad (PKCS7 255) msg) `shouldBe` Just msg + it "refuses to pad with a block size above 255" $ + evaluate (B.length (pad (PKCS7 256) msg)) `shouldThrow` rangeError + it "refuses to pad with a block size far above 255" $ + evaluate (B.length (pad (PKCS7 300) msg)) `shouldThrow` rangeError + it "refuses to pad with a block size of zero" $ + evaluate (B.length (pad (PKCS7 0) msg)) `shouldThrow` rangeError + it "refuses to pad with a negative block size" $ + evaluate (B.length (pad (PKCS7 (-1)) msg)) `shouldThrow` rangeError + it "refuses to unpad with a block size outside the range" $ + mapM_ + (\sz -> unpad (PKCS7 sz) oversized `shouldBe` Nothing) + [-1, 0, 256, 300] + where + msg = "a" :: B.ByteString + -- what pad (PKCS7 300) produced while the octet was allowed to wrap + oversized = msg `B.append` B.replicate 299 43 + rangeError (ErrorCall m) = "between 1 and 255" `isInfixOf` m + +-- | PKCS#7 padding runs from one octet to a whole block and no further: the +-- padded length is a multiple of the block size, and the padding is whatever +-- was added to reach it, so it can never exceed one block. unpad weighed the +-- octet against the length of the whole input instead, which only rules out +-- padding longer than the message. A block of sixteen therefore accepted a +-- claim of twenty and handed back twenty octets fewer than it was given. +paddingLengthTests :: Spec +paddingLengthTests = describe "PKCS7 padding length" $ do + it "accepts padding of exactly one block" $ + unpad (PKCS7 16) (pad (PKCS7 16) block) `shouldBe` Just block + it "accepts padding of a single octet" $ + unpad (PKCS7 16) (pad (PKCS7 16) (B.take 15 block)) + `shouldBe` Just (B.take 15 block) + it "rejects padding longer than the block" $ + unpad (PKCS7 16) (claiming 32 20) `shouldBe` Nothing + it "rejects padding longer than the block by one" $ + unpad (PKCS7 16) (claiming 32 17) `shouldBe` Nothing + it "rejects the largest octet a block of sixteen cannot mean" $ + unpad (PKCS7 16) (claiming 256 255) `shouldBe` Nothing + it "still rejects padding longer than the input" $ + unpad (PKCS7 16) (claiming 16 200) `shouldBe` Nothing + where + block = B.replicate 16 0x41 + -- len octets whose last n say that n octets of padding were added + claiming len n = + B.replicate (len - n) 0x41 `B.append` B.replicate n (fromIntegral n) + :: B.ByteString + +-- | ZERO took the remainder of the length by the block size without looking +-- at the size first, so a block size of zero divided by it. PKCS7 has been +-- checking its size since it gained a range; ZERO has a smaller range -- any +-- size from one up works, since the octets say nothing -- but zero and below +-- are still not sizes. +zeroBlockSizeTests :: Spec +zeroBlockSizeTests = describe "ZERO block size" $ do + it "refuses to pad with a block size of zero" $ + evaluate (B.length (pad (ZERO 0) msg)) `shouldThrow` zeroError + it "refuses to pad with a negative block size" $ + evaluate (B.length (pad (ZERO (-1)) msg)) `shouldThrow` zeroError + it "refuses to unpad with a block size of zero" $ + unpad (ZERO 0) msg `shouldBe` Nothing + it "refuses to unpad with a negative block size" $ + unpad (ZERO (-1)) msg `shouldBe` Nothing + where + msg = "ab" :: B.ByteString + zeroError (ErrorCall m) = "at least 1" `isInfixOf` m + +spec :: Spec +spec = do + describe "Cases" $ zipWithM_ testPad [1 ..] cases + describe "ZeroCases" $ zipWithM_ testZeroPad [1 ..] zeroCases + blockSizeTests + paddingLengthTests + zeroBlockSizeTests diff --git a/tests/Poly1305.hs b/tests/Poly1305.hs deleted file mode 100644 --- a/tests/Poly1305.hs +++ /dev/null @@ -1,48 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module Poly1305 (tests) where - -import qualified Data.ByteString as B -import qualified Data.ByteString.Char8 as B () - -import Crypto.Error -import Imports - -import qualified Crypto.MAC.Poly1305 as Poly1305 -import qualified Data.ByteArray as B (convert) - -instance Show Poly1305.Auth where - show _ = "Auth" - -data Chunking = Chunking Int Int - deriving (Show, Eq) - -instance Arbitrary Chunking where - arbitrary = Chunking <$> choose (1, 34) <*> choose (1, 2048) - -tests = - testGroup - "Poly1305" - [ testCase "V0" $ - let key = - "\x85\xd6\xbe\x78\x57\x55\x6d\x33\x7f\x44\x52\xfe\x42\xd5\x06\xa8\x01\x03\x80\x8a\xfb\x0d\xb2\xfd\x4a\xbf\xf6\xaf\x41\x49\xf5\x1b" - :: ByteString - msg = "Cryptographic Forum Research Group" :: ByteString - tag = - "\xa8\x06\x1d\xc1\x30\x51\x36\xc6\xc2\x2b\x8b\xaf\x0c\x01\x27\xa9" :: ByteString - in tag @=? B.convert (Poly1305.auth key msg) - , testProperty "Chunking" $ \(Chunking chunkLen totalLen) -> - let key = B.replicate 32 0 - msg = B.pack $ take totalLen $ concat (replicate 10 [1 .. 255]) - in Poly1305.auth key msg - == Poly1305.finalize - ( foldr - (flip Poly1305.update) - (throwCryptoError $ Poly1305.initialize key) - (chunks chunkLen msg) - ) - ] - where - chunks i bs - | B.length bs < i = [bs] - | otherwise = let (b1, b2) = B.splitAt i bs in b1 : chunks i b2 diff --git a/tests/PubKey/DHSpec.hs b/tests/PubKey/DHSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/DHSpec.hs @@ -0,0 +1,113 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.DHSpec (spec) where + +import Control.Exception (evaluate) +import Crypto.Error +import qualified Crypto.PubKey.DH as DH +import qualified Crypto.PubKey.ECC.DH as ECDH +import Crypto.PubKey.ECC.Types + +import Data.ByteArray (convert) +import qualified Data.ByteString as B + +import Imports + +-- | 'DH.SharedKey' wraps its bytes in a newtype, so evaluating it to weak head +-- normal form proves nothing. Convert it to force the bytes themselves. +force :: DH.SharedKey -> IO Int +force sk = evaluate (B.length (convert sk :: ByteString)) + +-- | getShared raises whatever tryGetShared reports, so any CryptoError means the +-- exchange was refused; the exact one is asserted on tryGetShared below. +anyCryptoError :: Selector CryptoError +anyCryptoError = const True + +rejected :: String -> DH.SharedKey -> Spec +rejected name sk = it name $ force sk `shouldThrow` anyCryptoError + +p256 :: Curve +p256 = getCurveByName SEC_p256r1 + +-- | A peer point is attacker supplied, so it has to be checked to be on the +-- curve before it is multiplied by our private number: the curve equation is +-- what confines the result to the group the private number was chosen for. +-- Multiplying an off-curve point instead lands in whatever group that point +-- generates, and a small one leaks the private number. +ecdhTests :: Spec +ecdhTests = + describe "ECDH" $ do + it "a valid exchange agrees" $ do + let qa = ECDH.calculatePublic p256 da + qb = ECDH.calculatePublic p256 db + ECDH.getShared p256 db qa `shouldBe` ECDH.getShared p256 da qb + rejected "a point not on the curve is refused" $ + ECDH.getShared p256 da (Point 1 1) + rejected "a point with a negative coordinate is refused" $ + ECDH.getShared p256 da (Point (-1) 1) + rejected "the point at infinity is refused" $ + ECDH.getShared p256 da PointO + it "tryGetShared agrees with getShared on a valid exchange" $ do + let qb = ECDH.calculatePublic p256 db + ECDH.tryGetShared p256 da qb `shouldBe` CryptoPassed (ECDH.getShared p256 da qb) + it "tryGetShared reports a point not on the curve" $ + ECDH.tryGetShared p256 da (Point 1 1) + `shouldBe` CryptoFailed CryptoError_PointCoordinatesInvalid + it "tryGetShared reports a negative coordinate" $ + ECDH.tryGetShared p256 da (Point (-1) 1) + `shouldBe` CryptoFailed CryptoError_PointCoordinatesInvalid + it "tryGetShared reports the point at infinity" $ + ECDH.tryGetShared p256 da PointO + `shouldBe` CryptoFailed CryptoError_ScalarMultiplicationInvalid + where + da = 0x2eb7ef8e5dcbd0f0fbf70b5d4d43ea0b5f0dbcb45a3e3d8b3f1eaf7a35b1fb31 + db = 0x6c2f5e5b1e9a8d4c3b2a190807f6e5d4c3b2a1908f7e6d5c4b3a29180706f5e4d + +-- | RFC 7919 section 5.1 requires the peer's public value y to satisfy +-- 1 < y < p-1. The excluded values generate the subgroup {1} or {1, p-1}, so +-- the shared secret they produce is one of a handful of constants and carries +-- none of our private number's secrecy. +-- +-- 'Params' also carries the size of p separately from p itself, and only p and +-- g travel on the wire, so the two can disagree; the shared secret must still +-- be the size p calls for rather than raising from i2ospOf_. +ffdhTests :: Spec +ffdhTests = + describe "finite field" $ do + it "a valid exchange agrees" $ do + let ya = DH.calculatePublic params xa + yb = DH.calculatePublic params xb + DH.getShared params xb ya `shouldBe` DH.getShared params xa yb + rejected "y = 0 is refused" $ DH.getShared params xa 0 + rejected "y = 1 is refused" $ DH.getShared params xa 1 + rejected "y = p-1 is refused" $ + DH.getShared params xa (DH.PublicNumber (p - 1)) + rejected "y = p is refused" $ DH.getShared params xa (DH.PublicNumber p) + rejected "y > p is refused" $ DH.getShared params xa (DH.PublicNumber (p + 1)) + it "tryGetShared agrees with getShared on a valid exchange" $ do + let yb = DH.calculatePublic params xb + DH.tryGetShared params xa yb `shouldBe` CryptoPassed (DH.getShared params xa yb) + it "tryGetShared reports a public number out of range" $ + mapM_ + ( \y -> + DH.tryGetShared params xa (DH.PublicNumber y) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + ) + [0, 1, p - 1, p, p + 1] + it "an understated bit size still yields p-sized output" $ do + let understated = DH.Params p 2 8 + yb = DH.calculatePublic understated xb + force (DH.getShared understated xa yb) `shouldReturn` 128 + where + -- RFC 7919 ffdhe1024 is not defined, so use the 1024-bit MODP group of + -- RFC 2409 section 6.2, whose generator is 2 + p = + 0xFFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E088A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE649286651ECE65381FFFFFFFFFFFFFFFF + params = DH.Params p 2 1024 + xa = DH.PrivateNumber 0x1f3b5d79a2c4e60813579bdf2468ace0 + xb = DH.PrivateNumber 0x2c4e60813579bdf2468ace01f3b5d79a + +spec :: Spec +spec = do + ecdhTests + ffdhTests diff --git a/tests/PubKey/DSASpec.hs b/tests/PubKey/DSASpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/DSASpec.hs @@ -0,0 +1,457 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.DSASpec (spec) where + +import Crypto.Hash +import qualified Crypto.PubKey.DSA as DSA +import Data.Maybe (isJust) + +import Imports + +data VectorDSA = VectorDSA + { pgq :: DSA.Params + , msg :: ByteString + , x :: Integer + , y :: Integer + , k :: Integer + , r :: Integer + , s :: Integer + } + +vectorsSHA1 = + [ VectorDSA + { msg = + "\x3b\x46\x73\x6d\x55\x9b\xd4\xe0\xc2\xc1\xb2\x55\x3a\x33\xad\x3c\x6c\xf2\x3c\xac\x99\x8d\x3d\x0c\x0e\x8f\xa4\xb1\x9b\xca\x06\xf2\xf3\x86\xdb\x2d\xcf\xf9\xdc\xa4\xf4\x0a\xd8\xf5\x61\xff\xc3\x08\xb4\x6c\x5f\x31\xa7\x73\x5b\x5f\xa7\xe0\xf9\xe6\xcb\x51\x2e\x63\xd7\xee\xa0\x55\x38\xd6\x6a\x75\xcd\x0d\x42\x34\xb5\xcc\xf6\xc1\x71\x5c\xca\xaf\x9c\xdc\x0a\x22\x28\x13\x5f\x71\x6e\xe9\xbd\xee\x7f\xc1\x3e\xc2\x7a\x03\xa6\xd1\x1c\x5c\x5b\x36\x85\xf5\x19\x00\xb1\x33\x71\x53\xbc\x6c\x4e\x8f\x52\x92\x0c\x33\xfa\x37\xf4\xe7" + , x = 0xc53eae6d45323164c7d07af5715703744a63fc3a + , y = + 0x313fd9ebca91574e1c2eebe1517c57e0c21b0209872140c5328761bbb2450b33f1b18b409ce9ab7c4cd8fda3391e8e34868357c199e16a6b2eba06d6749def791d79e95d3a4d09b24c392ad89dbf100995ae19c01062056bb14bce005e8731efde175f95b975089bdcdaea562b32786d96f5a31aedf75364008ad4fffebb970b + , k = 0x98cbcc4969d845e2461b5f66383dd503712bbcfa + , r = 0x50ed0e810e3f1c7cb6ac62332058448bd8b284c0 + , s = 0xc6aded17216b46b7e4b6f2a97c1ad7cc3da83fde + , pgq = dsaParams + } + , VectorDSA + { msg = + "\xd2\xbc\xb5\x3b\x04\x4b\x3e\x2e\x4b\x61\xba\x2f\x91\xc0\x99\x5f\xb8\x3a\x6a\x97\x52\x5e\x66\x44\x1a\x3b\x48\x9d\x95\x94\x23\x8b\xc7\x40\xbd\xee\xa0\xf7\x18\xa7\x69\xc9\x77\xe2\xde\x00\x38\x77\xb5\xd7\xdc\x25\xb1\x82\xae\x53\x3d\xb3\x3e\x78\xf2\xc3\xff\x06\x45\xf2\x13\x7a\xbc\x13\x7d\x4e\x7d\x93\xcc\xf2\x4f\x60\xb1\x8a\x82\x0b\xc0\x7c\x7b\x4b\x5f\xe0\x8b\x4f\x9e\x7d\x21\xb2\x56\xc1\x8f\x3b\x9d\x49\xac\xc4\xf9\x3e\x2c\xe6\xf3\x75\x4c\x78\x07\x75\x7d\x2e\x11\x76\x04\x26\x12\xcb\x32\xfc\x3f\x4f\x70\x70\x0e\x25" + , x = 0xe65131d73470f6ad2e5878bdc9bef536faf78831 + , y = + 0x29bdd759aaa62d4bf16b4861c81cf42eac2e1637b9ecba512bdbc13ac12a80ae8de2526b899ae5e4a231aef884197c944c732693a634d7659abc6975a773f8d3cd5a361fe2492386a3c09aaef12e4a7e73ad7dfc3637f7b093f2c40d6223a195c136adf2ea3fbf8704a675aa7817aa7ec7f9adfb2854d4e05c3ce7f76560313b + , k = 0x87256a64e98cf5be1034ecfa766f9d25d1ac7ceb + , r = 0xa26c00b5750a2d27fe7435b93476b35438b4d8ab + , s = 0x61c9bfcb2938755afa7dad1d1e07c6288617bf70 + , pgq = dsaParams + } + , VectorDSA + { msg = + "\xd5\x43\x1e\x6b\x16\xfd\xae\x31\x48\x17\x42\xbd\x39\x47\x58\xbe\xb8\xe2\x4f\x31\x94\x7e\x19\xb7\xea\x7b\x45\x85\x21\x88\x22\x70\xc1\xf4\x31\x92\xaa\x05\x0f\x44\x85\x14\x5a\xf8\xf3\xf9\xc5\x14\x2d\x68\xb8\x50\x18\xd2\xec\x9c\xb7\xa3\x7b\xa1\x2e\xd2\x3e\x73\xb9\x5f\xd6\x80\xfb\xa3\xc6\x12\x65\xe9\xf5\xa0\xa0\x27\xd7\x0f\xad\x0c\x8a\xa0\x8a\x3c\xbf\xbe\x99\x01\x8d\x00\x45\x38\x61\x73\xe5\xfa\xe2\x25\xfa\xeb\xe0\xce\xf5\xdd\x45\x91\x0f\x40\x0a\x86\xc2\xbe\x4e\x15\x25\x2a\x16\xde\x41\x20\xa2\x67\xbe\x2b\x59\x4d" + , x = 0x20bcabc6d9347a6e79b8e498c60c44a19c73258c + , y = + 0x23b4f404aa3c575e550bb320fdb1a085cd396a10e5ebc6771da62f037cab19eacd67d8222b6344038c4f7af45f5e62b55480cbe2111154ca9697ca76d87b56944138084e74c6f90a05cf43660dff8b8b3fabfcab3f0e4416775fdf40055864be102b4587392e77752ed2aeb182ee4f70be4a291dbe77b84a44ee34007957b1e0 + , k = 0x7d9bcfc9225432de9860f605a38d389e291ca750 + , r = 0x3f0a4ad32f0816821b8affb518e9b599f35d57c2 + , s = 0xea06638f2b2fc9d1dfe99c2a492806b497e2b0ea + , pgq = dsaParams + } + , VectorDSA + { msg = + "\x85\x66\x2b\x69\x75\x50\xe4\x91\x5c\x29\xe3\x38\xb6\x24\xb9\x12\x84\x5d\x6d\x1a\x92\x0d\x9e\x4c\x16\x04\xdd\x47\xd6\x92\xbc\x7c\x0f\xfb\x95\xae\x61\x4e\x85\x2b\xeb\xaf\x15\x73\x75\x8a\xd0\x1c\x71\x3c\xac\x0b\x47\x6e\x2f\x12\x17\x45\xa3\xcf\xee\xff\xb2\x44\x1f\xf6\xab\xfb\x9b\xbe\xb9\x8a\xa6\x34\xca\x6f\xf5\x41\x94\x7d\xcc\x99\x27\x65\x9d\x44\xf9\x5c\x5f\xf9\x17\x0f\xdc\x3c\x86\x47\x3c\xb6\x01\xba\x31\xb4\x87\xfe\x59\x36\xba\xc5\xd9\xc6\x32\xcb\xcc\x3d\xb0\x62\x46\xba\x01\xc5\x5a\x03\x8d\x79\x7f\xe3\xf6\xc3" + , x = 0x52d1fbe687aa0702a51a5bf9566bd51bd569424c + , y = + 0x6bc36cb3fa61cecc157be08639a7ca9e3de073b8a0ff23574ce5ab0a867dfd60669a56e60d1c989b3af8c8a43f5695d503e3098963990e12b63566784171058eace85c728cd4c08224c7a6efea75dca20df461013c75f40acbc23799ebee7f3361336dadc4a56f305708667bfe602b8ea75a491a5cf0c06ebd6fdc7161e10497 + , k = 0x960c211891c090d05454646ebac1bfe1f381e82b + , r = 0x3bc29dee96957050ba438d1b3e17b02c1725d229 + , s = 0x0af879cf846c434e08fb6c63782f4d03e0d88865 + , pgq = dsaParams + } + , VectorDSA + { msg = + "\x87\xb6\xe7\x5b\x9f\x8e\x99\xc4\xdd\x62\xad\xb6\x93\xdd\x58\x90\xed\xff\x1b\xd0\x02\x8f\x4e\xf8\x49\xdf\x0f\x1d\x2c\xe6\xb1\x81\xfc\x3a\x55\xae\xa6\xd0\xa1\xf0\xae\xca\xb8\xed\x9e\x24\x8a\x00\xe9\x6b\xe7\x94\xa7\xcf\xba\x12\x46\xef\xb7\x10\xef\x4b\x37\x47\x1c\xef\x0a\x1b\xcf\x55\xce\xbc\x8d\x5a\xd0\x71\x61\x2b\xd2\x37\xef\xed\xd5\x10\x23\x62\xdb\x07\xa1\xe2\xc7\xa6\xf1\x5e\x09\xfe\x64\xba\x42\xb6\x0a\x26\x28\xd8\x69\xae\x05\xef\x61\x1f\xe3\x8d\x9c\xe1\x5e\xee\xc9\xbb\x3d\xec\xc8\xdc\x17\x80\x9f\x3b\x6e\x95" + , x = 0xc86a54ec5c4ec63d7332cf43ddb082a34ed6d5f5 + , y = + 0x014ac746d3605efcb8a2c7dae1f54682a262e27662b252c09478ce87d0aaa522d7c200043406016c0c42896d21750b15dbd57f9707ec37dcea5651781b67ad8d01f5099fe7584b353b641bb159cc717d8ceb18b66705e656f336f1214b34f0357e577ab83641969e311bf40bdcb3ffd5e0bb59419f229508d2f432cc2859ff75 + , k = 0x6c445cee68042553fbe63be61be4ddb99d8134af + , r = 0x637e07a5770f3dc65e4506c68c770e5ef6b8ced3 + , s = 0x7dfc6f83e24f09745e01d3f7ae0ed1474e811d47 + , pgq = dsaParams + } + , VectorDSA + { msg = + "\x22\x59\xee\xad\x2d\x6b\xbc\x76\xd4\x92\x13\xea\x0d\xc8\xb7\x35\x0a\x97\x69\x9f\x22\x34\x10\x44\xc3\x94\x07\x82\x36\x4a\xc9\xea\x68\x31\x79\xa4\x38\xa5\xea\x45\x99\x8d\xf9\x7c\x29\x72\xda\xe0\x38\x51\xf5\xbe\x23\xfa\x9f\x04\x18\x2e\x79\xdd\xb2\xb5\x6d\xc8\x65\x23\x93\xec\xb2\x7f\x3f\x3b\x7c\x8a\x8d\x76\x1a\x86\xb3\xb8\xf4\xd4\x1a\x07\xb4\xbe\x7d\x02\xfd\xde\xfc\x42\xb9\x28\x12\x4a\x5a\x45\xb9\xf4\x60\x90\x42\x20\x9b\x3a\x7f\x58\x5b\xd5\x14\xcc\x39\xc0\x0e\xff\xcc\x42\xc7\xfe\x70\xfa\x83\xed\xf8\xa3\x2b\xf4" + , x = 0xaee6f213b9903c8069387e64729a08999e5baf65 + , y = + 0x0fe74045d7b0d472411202831d4932396f242a9765e92be387fd81bbe38d845054528b348c03984179b8e505674cb79d88cc0d8d3e8d7392f9aa773b29c29e54a9e326406075d755c291fcedbcc577934c824af988250f64ed5685fce726cff65e92d708ae11cbfaa958ab8d8b15340a29a137b5b4357f7ed1c7a5190cbf98a4 + , k = 0xe1704bae025942e2e63c6d76bab88da79640073a + , r = 0x83366ba3fed93dfb38d541203ecbf81c363998e2 + , s = 0x1fe299c36a1332f23bf2e10a6c6a4e0d3cdd2bf4 + , pgq = dsaParams + } + , VectorDSA + { msg = + "\x21\x9e\x8d\xf5\xbf\x88\x15\x90\x43\x0e\xce\x60\x82\x50\xf7\x67\x0d\xc5\x65\x37\x24\x93\x02\x42\x9e\x28\xec\xfe\xb9\xce\xaa\xa5\x49\x10\xa6\x94\x90\xf7\x65\xf3\xdf\x82\xe8\xb0\x1c\xd7\xd7\x6e\x56\x1d\x0f\x6c\xe2\x26\xef\x3c\xf7\x52\xca\xda\x6f\xeb\xdc\x5b\xf0\x0d\x67\x94\x7f\x92\xd4\x20\x51\x6b\x9e\x37\xc9\x6c\x8f\x1f\x2d\xa0\xb0\x75\x09\x7c\x3b\xda\x75\x8a\x8d\x91\xbd\x2e\xbe\x9c\x75\xcf\x14\x7f\x25\x4c\x25\x69\x63\xb3\x3b\x67\xd0\x2b\x6a\xa0\x9e\x7d\x74\x65\xd0\x38\xe5\x01\x95\xec\xe4\x18\x9b\x41\xe7\x68" + , x = 0x699f1c07aa458c6786e770b40197235fe49cf21a + , y = + 0x3a41b0678ff3c4dde20fa39772bac31a2f18bae4bedec9e12ee8e02e30e556b1a136013bef96b0d30b568233dcecc71e485ed75c922afb4d0654e709bee84993792130220e3005fdb06ebdfc0e2df163b5ec424e836465acd6d92e243c86f2b94b26b8d73bd9cf722c757e0b80b0af16f185de70e8ca850b1402d126ea60f309 + , k = 0x5bbb795bfa5fa72191fed3434a08741410367491 + , r = 0x579761039ae0ddb81106bf4968e320083bbcb947 + , s = 0x503ea15dbac9dedeba917fa8e9f386b93aa30353 + , pgq = dsaParams + } + , VectorDSA + { msg = + "\x2d\xa7\x9d\x06\x78\x85\xeb\x3c\xcf\x5e\x29\x3a\xe3\xb1\xd8\x22\x53\x22\x20\x3a\xbb\x5a\xdf\xde\x3b\x0f\x53\xbb\xe2\x4c\x4f\xe0\x01\x54\x1e\x11\x83\xd8\x70\xa9\x97\xf1\xf9\x46\x01\x00\xb5\xd7\x11\x92\x31\x80\x15\x43\x45\x28\x7a\x02\x14\xcf\x1c\xac\x37\xb7\xa4\x7d\xfb\xb2\xa0\xe8\xce\x49\x16\xf9\x4e\xbd\x6f\xa5\x4e\x31\x5b\x7a\x8e\xb5\xb6\x3c\xd9\x54\xc5\xba\x05\xc1\xbf\x7e\x33\xa4\xe8\xa1\x51\xf3\x2d\x28\x77\xb0\x17\x29\xc1\xad\x0e\x7c\x01\xbb\x8a\xe7\x23\xc9\x95\x18\x38\x03\xe4\x56\x36\x52\x0e\xa3\x8c\xa1" + , x = 0xd6e08c20c82949ddba93ea81eb2fea8c595894dc + , y = + 0x56f7272210f316c51af8bfc45a421fd4e9b1043853271b7e79f40936f0adcf262a86097aa86e19e6cb5307685d863dba761342db6c973b3849b1e060aca926f41fe07323601062515ae85f3172b8f34899c621d59fa21f73d5ae97a3deb5e840b25a18fd580862fd7b1cf416c7ae9fc5842a0197fdb0c5173ff4a4f102a8cf89 + , k = 0x6d72c30d4430959800740f2770651095d0c181c2 + , r = 0x5dd90d69add67a5fae138eec1aaff0229aa4afc4 + , s = 0x47f39c4db2387f10762f45b80dfd027906d7ef04 + , pgq = dsaParams + } + , VectorDSA + { msg = + "\xba\x30\xd8\x5b\xe3\x57\xe7\xfb\x29\xf8\xa0\x7e\x1f\x12\x7b\xaa\xa2\x4b\x2e\xe0\x27\xf6\x4c\xb5\xef\xee\xc6\xaa\xea\xbc\xc7\x34\x5c\x5d\x55\x6e\xbf\x4b\xdc\x7a\x61\xc7\x7c\x7b\x7e\xa4\x3c\x73\xba\xbc\x18\xf7\xb4\x80\x77\x22\xda\x23\x9e\x45\xdd\xf2\x49\x84\x9c\xbb\xfe\x35\x07\x11\x2e\xbf\x87\xd7\xef\x56\x0c\x2e\x7d\x39\x1e\xd8\x42\x4f\x87\x10\xce\xa4\x16\x85\x14\x3e\x30\x06\xf8\x1b\x68\xfb\xb4\xd5\xf9\x64\x4c\x7c\xd1\x0f\x70\x92\xef\x24\x39\xb8\xd1\x8c\x0d\xf6\x55\xe0\x02\x89\x37\x2a\x41\x66\x38\x5d\x64\x0c" + , x = 0x50018482864c1864e9db1f04bde8dbfd3875c76d + , y = + 0x0942a5b7a72ab116ead29308cf658dfe3d55d5d61afed9e3836e64237f9d6884fdd827d2d5890c9a41ae88e7a69fc9f345ade9c480c6f08cff067c183214c227236cedb6dd1283ca2a602574e8327510221d4c27b162143b7002d8c726916826265937b87be9d5ec6d7bd28fb015f84e0ab730da7a4eaf4ef3174bf0a22a6392 + , k = 0xdf3a9348f37b5d2d4c9176db266ae388f1fa7e0f + , r = 0x448434b214eee38bde080f8ec433e8d19b3ddf0d + , s = 0x0c02e881b777923fe0ea674f2621298e00199d5f + , pgq = dsaParams + } + , VectorDSA + { msg = + "\x83\x49\x9e\xfb\x06\xbb\x7f\xf0\x2f\xfb\x46\xc2\x78\xa5\xe9\x26\x30\xac\x5b\xc3\xf9\xe5\x3d\xd2\xe7\x8f\xf1\x5e\x36\x8c\x7e\x31\xaa\xd7\x7c\xf7\x71\xf3\x5f\xa0\x2d\x0b\x5f\x13\x52\x08\xa4\xaf\xdd\x86\x7b\xb2\xec\x26\xea\x2e\x7d\xd6\x4c\xde\xf2\x37\x50\x8a\x38\xb2\x7f\x39\xd8\xb2\x2d\x45\xca\xc5\xa6\x8a\x90\xb6\xea\x76\x05\x86\x45\xf6\x35\x6a\x93\x44\xd3\x6f\x00\xec\x66\x52\xea\xa4\xe9\xba\xe7\xb6\x94\xf9\xf1\xfc\x8c\x6c\x5e\x86\xfa\xdc\x7b\x27\xa2\x19\xb5\xc1\xb2\xae\x80\xa7\x25\xe5\xf6\x11\x65\xfe\x2e\xdc" + , x = 0xae56f66b0a9405b9cca54c60ec4a3bb5f8be7c3f + , y = + 0xa01542c3da410dd57930ca724f0f507c4df43d553c7f69459939685941ceb95c7dcc3f175a403b359621c0d4328e98f15f330a63865baf3e7eb1604a0715e16eed64fd14b35d3a534259a6a7ddf888c4dbb5f51bbc6ed339e5bb2a239d5cfe2100ac8e2f9c16e536f25119ab435843af27dc33414a9e4602f96d7c94d6021cec + , k = 0x8857ff301ad0169d164fa269977a116e070bac17 + , r = 0x8c2fab489c34672140415d41a65cef1e70192e23 + , s = 0x3df86a9e2efe944a1c7ea9c30cac331d00599a0e + , pgq = dsaParams + } + , VectorDSA -- 1024-bit example from RFC 6979 with SHA-1 + { msg = "sample" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x7BDB6B0FF756E1BB5D53583EF979082F9AD5BD5B + , r = 0x2E1A0C2562B2912CAAF89186FB0F42001585DA55 + , s = 0x29EFB6B0AFF2D7A68EB70CA313022253B9A88DF5 + , pgq = rfc6979Params1024 + } + , VectorDSA -- 1024-bit example from RFC 6979 with SHA-1 + { msg = "test" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x5C842DF4F9E344EE09F056838B42C7A17F4A6433 + , r = 0x42AB2052FD43E123F0607F115052A67DCD9C5C77 + , s = 0x183916B0230D45B9931491D4C6B0BD2FB4AAF088 + , pgq = rfc6979Params1024 + } + , VectorDSA -- 2048-bit example from RFC 6979 with SHA-1 + { msg = "sample" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x888FA6F7738A41BDC9846466ABDB8174C0338250AE50CE955CA16230F9CBD53E + , r = 0x3A1B2DBD7489D6ED7E608FD036C83AF396E290DBD602408E8677DAABD6E7445A + , s = 0xD26FCBA19FA3E3058FFC02CA1596CDBB6E0D20CB37B06054F7E36DED0CDBBCCF + , pgq = rfc6979Params2048 + } + , VectorDSA -- 2048-bit example from RFC 6979 with SHA-1 + { msg = "test" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x6EEA486F9D41A037B2C640BC5645694FF8FF4B98D066A25F76BE641CCB24BA4F + , r = 0xC18270A93CFC6063F57A4DFA86024F700D980E4CF4E2CB65A504397273D98EA0 + , s = 0x414F22E5F31A8B6D33295C7539C1C1BA3A6160D7D68D50AC0D3A5BEAC2884FAA + , pgq = rfc6979Params2048 + } + ] + where + -- (p,g,q) + dsaParams = + DSA.Params + { DSA.params_p = + 0xa8f9cd201e5e35d892f85f80e4db2599a5676a3b1d4f190330ed3256b26d0e80a0e49a8fffaaad2a24f472d2573241d4d6d6c7480c80b4c67bb4479c15ada7ea8424d2502fa01472e760241713dab025ae1b02e1703a1435f62ddf4ee4c1b664066eb22f2e3bf28bb70a2a76e4fd5ebe2d1229681b5b06439ac9c7e9d8bde283 + , DSA.params_g = + 0x2b3152ff6c62f14622b8f48e59f8af46883b38e79b8c74deeae9df131f8b856e3ad6c8455dab87cc0da8ac973417ce4f7878557d6cdf40b35b4a0ca3eb310c6a95d68ce284ad4e25ea28591611ee08b8444bd64b25f3f7c572410ddfb39cc728b9c936f85f419129869929cdb909a6a3a99bbe089216368171bd0ba81de4fe33 + , DSA.params_q = 0xf85f0f83ac4df7ea0cdf8f469bfeeaea14156495 + } + +vectorsSHA224 = + [ VectorDSA + { msg = "sample" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x562097C06782D60C3037BA7BE104774344687649 + , r = 0x4BC3B686AEA70145856814A6F1BB53346F02101E + , s = 0x410697B92295D994D21EDD2F4ADA85566F6F94C1 + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "test" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x4598B8EFC1A53BC8AECD58D1ABBB0C0C71E67297 + , r = 0x6868E9964E36C1689F6037F91F28D5F2C30610F2 + , s = 0x49CEC3ACDC83018C5BD2674ECAAD35B8CD22940F + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "sample" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0xBC372967702082E1AA4FCE892209F71AE4AD25A6DFD869334E6F153BD0C4D806 + , r = 0xDC9F4DEADA8D8FF588E98FED0AB690FFCE858DC8C79376450EB6B76C24537E2C + , s = 0xA65A9C3BC7BABE286B195D5DA68616DA8D47FA0097F36DD19F517327DC848CEC + , pgq = rfc6979Params2048 + } + , VectorDSA + { msg = "test" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x06BD4C05ED74719106223BE33F2D95DA6B3B541DAD7BFBD7AC508213B6DA6670 + , r = 0x272ABA31572F6CC55E30BF616B7A265312018DD325BE031BE0CC82AA17870EA3 + , s = 0xE9CC286A52CCE201586722D36D1E917EB96A4EBDB47932F9576AC645B3A60806 + , pgq = rfc6979Params2048 + } + ] + +vectorsSHA256 = + [ VectorDSA + { msg = "sample" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x519BA0546D0C39202A7D34D7DFA5E760B318BCFB + , r = 0x81F2F5850BE5BC123C43F71A3033E9384611C545 + , s = 0x4CDD914B65EB6C66A8AAAD27299BEE6B035F5E89 + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "test" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x5A67592E8128E03A417B0484410FB72C0B630E1A + , r = 0x22518C127299B0F6FDC9872B282B9E70D0790812 + , s = 0x6837EC18F150D55DE95B5E29BE7AF5D01E4FE160 + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "sample" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x8926A27C40484216F052F4427CFD5647338B7B3939BC6573AF4333569D597C52 + , r = 0xEACE8BDBBE353C432A795D9EC556C6D021F7A03F42C36E9BC87E4AC7932CC809 + , s = 0x7081E175455F9247B812B74583E9E94F9EA79BD640DC962533B0680793A38D53 + , pgq = rfc6979Params2048 + } + , VectorDSA + { msg = "test" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x1D6CE6DDA1C5D37307839CD03AB0A5CBB18E60D800937D67DFB4479AAC8DEAD7 + , r = 0x8190012A1969F9957D56FCCAAD223186F423398D58EF5B3CEFD5A4146A4476F0 + , s = 0x7452A53F7075D417B4B013B278D1BB8BBD21863F5E7B1CEE679CF2188E1AB19E + , pgq = rfc6979Params2048 + } + ] + +vectorsSHA384 = + [ VectorDSA + { msg = "sample" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x95897CD7BBB944AA932DBC579C1C09EB6FCFC595 + , r = 0x07F2108557EE0E3921BC1774F1CA9B410B4CE65A + , s = 0x54DF70456C86FAC10FAB47C1949AB83F2C6F7595 + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "test" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x220156B761F6CA5E6C9F1B9CF9C24BE25F98CD89 + , r = 0x854CF929B58D73C3CBFDC421E8D5430CD6DB5E66 + , s = 0x91D0E0F53E22F898D158380676A871A157CDA622 + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "sample" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0xC345D5AB3DA0A5BCB7EC8F8FB7A7E96069E03B206371EF7D83E39068EC564920 + , r = 0xB2DA945E91858834FD9BF616EBAC151EDBC4B45D27D0DD4A7F6A22739F45C00B + , s = 0x19048B63D9FD6BCA1D9BAE3664E1BCB97F7276C306130969F63F38FA8319021B + , pgq = rfc6979Params2048 + } + , VectorDSA + { msg = "test" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x206E61F73DBE1B2DC8BE736B22B079E9DACD974DB00EEBBC5B64CAD39CF9F91C + , r = 0x239E66DDBE8F8C230A3D071D601B6FFBDFB5901F94D444C6AF56F732BEB954BE + , s = 0x6BD737513D5E72FE85D1C750E0F73921FE299B945AAD1C802F15C26A43D34961 + , pgq = rfc6979Params2048 + } + ] + +vectorsSHA512 = + [ VectorDSA + { msg = "sample" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x09ECE7CA27D0F5A4DD4E556C9DF1D21D28104F8B + , r = 0x16C3491F9B8C3FBBDD5E7A7B667057F0D8EE8E1B + , s = 0x02C36A127A7B89EDBB72E4FFBC71DABC7D4FC69C + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "test" + , x = 0x411602CB19A6CCC34494D79D98EF1E7ED5AF25F7 + , y = + 0x5DF5E01DED31D0297E274E1691C192FE5868FEF9E19A84776454B100CF16F65392195A38B90523E2542EE61871C0440CB87C322FC4B4D2EC5E1E7EC766E1BE8D4CE935437DC11C3C8FD426338933EBFE739CB3465F4D3668C5E473508253B1E682F65CBDC4FAE93C2EA212390E54905A86E2223170B44EAA7DA5DD9FFCFB7F3B + , k = 0x65D2C2EEB175E370F28C75BFCDC028D22C7DBE9C + , r = 0x8EA47E475BA8AC6F2D821DA3BD212D11A3DEB9A0 + , s = 0x7C670C7AD72B6C050C109E1790008097125433E8 + , pgq = rfc6979Params1024 + } + , VectorDSA + { msg = "sample" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0x5A12994431785485B3F5F067221517791B85A597B7A9436995C89ED0374668FC + , r = 0x2016ED092DC5FB669B8EFB3D1F31A91EECB199879BE0CF78F02BA062CB4C942E + , s = 0xD0C76F84B5F091E141572A639A4FB8C230807EEA7D55C8A154A224400AFF2351 + , pgq = rfc6979Params2048 + } + , VectorDSA + { msg = "test" + , x = 0x69C7548C21D0DFEA6B9A51C9EAD4E27C33D3B3F180316E5BCAB92C933F0E4DBC + , y = + 0x667098C654426C78D7F8201EAC6C203EF030D43605032C2F1FA937E5237DBD949F34A0A2564FE126DC8B715C5141802CE0979C8246463C40E6B6BDAA2513FA611728716C2E4FD53BC95B89E69949D96512E873B9C8F8DFD499CC312882561ADECB31F658E934C0C197F2C4D96B05CBAD67381E7B768891E4DA3843D24D94CDFB5126E9B8BF21E8358EE0E0A30EF13FD6A664C0DCE3731F7FB49A4845A4FD8254687972A2D382599C9BAC4E0ED7998193078913032558134976410B89D2C171D123AC35FD977219597AA7D15C1A9A428E59194F75C721EBCBCFAE44696A499AFA74E04299F132026601638CB87AB79190D4A0986315DA8EEC6561C938996BEADF + , k = 0xAFF1651E4CD6036D57AA8B2A05CCF1A9D5A40166340ECBBDC55BE10B568AA0AA + , r = 0x89EC4BB1400ECCFF8E7D9AA515CD1DE7803F2DAFF09693EE7FD1353E90A68307 + , s = 0xC9F0BDABCC0D880BB137A994CC7F3980CE91CC10FAF529FC46565B15CEA854E1 + , pgq = rfc6979Params2048 + } + ] + +rfc6979Params1024 = + DSA.Params + { DSA.params_p = + 0x86F5CA03DCFEB225063FF830A0C769B9DD9D6153AD91D7CE27F787C43278B447E6533B86B18BED6E8A48B784A14C252C5BE0DBF60B86D6385BD2F12FB763ED8873ABFD3F5BA2E0A8C0A59082EAC056935E529DAF7C610467899C77ADEDFC846C881870B7B19B2B58F9BE0521A17002E3BDD6B86685EE90B3D9A1B02B782B1779 + , DSA.params_g = + 0x07B0F92546150B62514BB771E2A0C0CE387F03BDA6C56B505209FF25FD3C133D89BBCD97E904E09114D9A7DEFDEADFC9078EA544D2E401AEECC40BB9FBBF78FD87995A10A1C27CB7789B594BA7EFB5C4326A9FE59A070E136DB77175464ADCA417BE5DCE2F40D10A46A3A3943F26AB7FD9C0398FF8C76EE0A56826A8A88F1DBD + , DSA.params_q = 0x996F967F6C8E388D9E28D01E205FBA957A5698B1 + } + +rfc6979Params2048 = + DSA.Params + { DSA.params_p = + 0x9DB6FB5951B66BB6FE1E140F1D2CE5502374161FD6538DF1648218642F0B5C48C8F7A41AADFA187324B87674FA1822B00F1ECF8136943D7C55757264E5A1A44FFE012E9936E00C1D3E9310B01C7D179805D3058B2A9F4BB6F9716BFE6117C6B5B3CC4D9BE341104AD4A80AD6C94E005F4B993E14F091EB51743BF33050C38DE235567E1B34C3D6A5C0CEAA1A0F368213C3D19843D0B4B09DCB9FC72D39C8DE41F1BF14D4BB4563CA28371621CAD3324B6A2D392145BEBFAC748805236F5CA2FE92B871CD8F9C36D3292B5509CA8CAA77A2ADFC7BFD77DDA6F71125A7456FEA153E433256A2261C6A06ED3693797E7995FAD5AABBCFBE3EDA2741E375404AE25B + , DSA.params_g = + 0x5C7FF6B06F8F143FE8288433493E4769C4D988ACE5BE25A0E24809670716C613D7B0CEE6932F8FAA7C44D2CB24523DA53FBE4F6EC3595892D1AA58C4328A06C46A15662E7EAA703A1DECF8BBB2D05DBE2EB956C142A338661D10461C0D135472085057F3494309FFA73C611F78B32ADBB5740C361C9F35BE90997DB2014E2EF5AA61782F52ABEB8BD6432C4DD097BC5423B285DAFB60DC364E8161F4A2A35ACA3A10B1C4D203CC76A470A33AFDCBDD92959859ABD8B56E1725252D78EAC66E71BA9AE3F1DD2487199874393CD4D832186800654760E1E34C09E4D155179F9EC0DC4473F996BDCE6EED1CABED8B6F116F7AD9CF505DF0F998E34AB27514B0FFE7 + , DSA.params_q = + 0xF2C3119374CE76C9356990B465374A17F23F9ED35089BD969F61C6DDE9998C1F + } + +vectorToPrivate :: VectorDSA -> DSA.PrivateKey +vectorToPrivate vector = + DSA.PrivateKey + { DSA.private_x = x vector + , DSA.private_params = pgq vector + } + +vectorToPublic :: VectorDSA -> DSA.PublicKey +vectorToPublic vector = + DSA.PublicKey + { DSA.public_y = y vector + , DSA.public_params = pgq vector + } + +doSignatureTest hashAlg i vector = it (show i) (actual `shouldBe` expected) + where + expected = Just $ DSA.Signature (r vector) (s vector) + actual = DSA.signWith (k vector) (vectorToPrivate vector) hashAlg (msg vector) + +doVerifyTest hashAlg i vector = it (show i) (actual `shouldBe` True) + where + actual = + DSA.verify + hashAlg + (vectorToPublic vector) + (DSA.Signature (r vector) (s vector)) + (msg vector) + +-- | Both sign and verify invert a value modulo q with 'fromJust'. The +-- inverse does not exist when the value shares a factor with q, and neither +-- path rules that out: signWith takes k from the caller, and verify takes both +-- the signature and the parameters from whoever supplied the public key, so a +-- composite q admits an s that is not invertible. Each has a way to say no -- +-- signWith returns Maybe, verify returns Bool -- so neither should raise. +nonInvertibleTests :: Spec +nonInvertibleTests = + describe "non-invertible values" $ do + it "signWith with k = 0 returns Nothing" $ + DSA.signWith 0 priv SHA1 message `shouldBe` Nothing + it "signWith with k = q returns Nothing" $ + DSA.signWith q priv SHA1 message `shouldBe` Nothing + it "signWith with k sharing a factor with q returns Nothing" $ + DSA.signWith 3 compositePriv SHA1 message `shouldBe` Nothing + it "signWith with a usable k still signs" $ + DSA.signWith 4 priv SHA1 message `shouldSatisfy` isJust + it "verify with a non-invertible s returns False" $ + DSA.verify SHA1 compositePub (DSA.Signature 1 3) message `shouldBe` False + where + message = "message" :: ByteString + q = 11 + params = DSA.Params{DSA.params_p = 23, DSA.params_g = 4, DSA.params_q = q} + priv = DSA.PrivateKey params 3 + -- q = 9 is composite, so 3 has no inverse modulo q + compositeParams = DSA.Params{DSA.params_p = 23, DSA.params_g = 4, DSA.params_q = 9} + compositePriv = DSA.PrivateKey compositeParams 3 + compositePub = DSA.PublicKey compositeParams 4 + +spec :: Spec +spec = do + describe "SHA1" $ do + describe "signature" $ zipWithM_ (doSignatureTest SHA1) [katZero ..] vectorsSHA1 + describe "verify" $ zipWithM_ (doVerifyTest SHA1) [katZero ..] vectorsSHA1 + describe "SHA224" $ do + describe "signature" $ + sequence_ $ + zipWith (doSignatureTest SHA224) [katZero ..] vectorsSHA224 + describe "verify" $ zipWithM_ (doVerifyTest SHA224) [katZero ..] vectorsSHA224 + describe "SHA256" $ do + describe "signature" $ + sequence_ $ + zipWith (doSignatureTest SHA256) [katZero ..] vectorsSHA256 + describe "verify" $ zipWithM_ (doVerifyTest SHA256) [katZero ..] vectorsSHA256 + describe "SHA384" $ do + describe "signature" $ + sequence_ $ + zipWith (doSignatureTest SHA384) [katZero ..] vectorsSHA384 + describe "verify" $ zipWithM_ (doVerifyTest SHA384) [katZero ..] vectorsSHA384 + describe "SHA512" $ do + describe "signature" $ + sequence_ $ + zipWith (doSignatureTest SHA512) [katZero ..] vectorsSHA512 + describe "verify" $ zipWithM_ (doVerifyTest SHA512) [katZero ..] vectorsSHA512 + nonInvertibleTests diff --git a/tests/PubKey/ECCSpec.hs b/tests/PubKey/ECCSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/ECCSpec.hs @@ -0,0 +1,466 @@ +-- The binary curves are deprecated and still supported, so the tests +-- that hold them to their behaviour name them on purpose. +{-# OPTIONS_GHC -Wno-deprecations #-} +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.ECCSpec (spec) where + +import Crypto.Error (CryptoError (..), CryptoFailable (..)) +import Crypto.Number.Basic (numBits) +import Crypto.Number.F2m (squareF2m) +import qualified Crypto.PubKey.ECC.DH as ECDH +import qualified Crypto.PubKey.ECC.Prim as ECC +import qualified Crypto.PubKey.ECC.Types as ECC +import Data.Bits (testBit) + +import Imports + +instance Arbitrary ECC.Curve where + arbitrary = + ECC.getCurveByName + <$> elements + [ ECC.SEC_p112r1 + , ECC.SEC_p112r2 + , ECC.SEC_p128r1 + , ECC.SEC_p128r2 + , ECC.SEC_p160k1 + , ECC.SEC_p160r1 + , ECC.SEC_p160r2 + , ECC.SEC_p192k1 + , ECC.SEC_p192r1 + , ECC.SEC_p224k1 + , ECC.SEC_p224r1 + , ECC.SEC_p256k1 + , ECC.SEC_p256r1 + , ECC.SEC_p384r1 + , ECC.SEC_p521r1 + , ECC.SEC_t113r1 + , ECC.SEC_t113r2 + , ECC.SEC_t131r1 + , ECC.SEC_t131r2 + , ECC.SEC_t163k1 + , ECC.SEC_t163r1 + , ECC.SEC_t163r2 + , ECC.SEC_t193r1 + , ECC.SEC_t193r2 + , ECC.SEC_t233k1 + , ECC.SEC_t233r1 + , ECC.SEC_t239k1 + , ECC.SEC_t283k1 + , ECC.SEC_t283r1 + , ECC.SEC_t409k1 + , ECC.SEC_t409r1 + , ECC.SEC_t571k1 + , ECC.SEC_t571r1 + ] + +data VectorPoint = VectorPoint + { curve :: ECC.Curve + , x :: Integer + , y :: Integer + , valid :: Bool + } + +vectorsPoint :: [VectorPoint] +vectorsPoint = + [ VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x491c0c4761b0a4a147b5e4ce03a531546644f5d1e3d05e57 + , y = 0x6fa5addd47c5d6be3933fbff88f57a6c8ca0232c471965de + , valid = False -- point not on curve + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x646c22e8aa5f7833390e0399155ac198ae42470bba4fc834 + , y = 0x8d4afcfffd80e69a4d180178b37c44572495b7b267ee32a9 + , valid = True + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x4c6b9ea0dec92ecfff7799470be6a2277b9169daf45d54bb + , y = 0xf0eab42826704f51b26ae98036e83230becb639dd1964627 + , valid = False -- point not on curve + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x0673c8bb717b055c3d6f55c06acfcfb7260361ed3ec0f414 + , y = 0xba8b172826eb0b854026968d2338a180450a27906f6eddea + , valid = True + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x82c949295156192df0b52480e38c810751ac570daec460a3 + , y = 0x200057ada615c80b8ff256ce8d47f2562b74a438f1921ac3 + , valid = False -- point not on curve + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x284fbaa76ce0faae2ca4867d01092fa1ace5724cd12c8dd0 + , y = 0xe42af3dbf3206be3fcbcc3a7ccaf60c73dc29e7bb9b44fca + , valid = True + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x1b574acd4fb0f60dde3e3b5f3f0e94211f95112e43cba6fd2 + , y = 0xbcc1b8a770f01a22e84d7f14e44932ffe094d8e3b1e6ac26 + , valid = False -- x or y out of range + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x16ba109f1f1bb44e0d05b80181c03412ea764a59601d17e9f + , y = 0x0569a843dbb4e287db420d6b9fe30cd7b5d578b052315f56 + , valid = False -- x or y out of range + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x1333308a7c833ede5189d25ea3525919c9bd16370d904938d + , y = 0xb10fd01d67df75ff9b726c700c1b50596c9f0766ea56f80e + , valid = False -- x or y out of range + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x9671ec444cff24c8a5be80b018fa505ed6109a731e88c91a + , y = 0xfe79dae23008e46bf4230c895aab261a95845a77f06d0655 + , valid = True + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0x158e8b6f0b14216bc52fe8897b4305d870ede70436a96741d + , y = 0xfb3f970b19a313571a1a23be310923f85acc1cab0a157cbd + , valid = False -- x or y out of range + } + , VectorPoint + { curve = ECC.getCurveByName ECC.SEC_p192r1 + , x = 0xace95b650c08f73dbb4fa7b4bbdebd6b809a25b28ed135ef + , y = 0xe9b8679404166d1329dd539ad52aad9a1b6681f5f26bb9aa + , valid = False -- point not on curve + } + ] + +doPointValidTest :: Show a => a -> VectorPoint -> Spec +doPointValidTest i vector = + it + (show i) + ( ECC.isPointValid (curve vector) (ECC.Point (x vector) (y vector)) + `shouldBe` valid vector + ) + +arbitraryPoint :: ECC.Curve -> Gen ECC.Point +arbitraryPoint aCurve = + frequency [(5, return ECC.PointO), (95, pointGen)] + where + n = ECC.ecc_n (ECC.common_curve aCurve) + pointGen = ECC.pointBaseMul aCurve <$> choose (1, n - 1) + +-- | P-256 is the one curve here with a C implementation, and multiplication +-- on it is about to be routed to that. The properties below cover scalars +-- QuickCheck draws; these are the values at the edges of what a +-- multiplication has to answer for, and the shapes that signature +-- verification uses. +p256Tests :: Spec +p256Tests = + describe "P-256" $ do + it "the whole order takes a point to infinity" $ + ECC.pointMul p256curve order g `shouldBe` ECC.PointO + it "one past the order is one" $ + ECC.pointMul p256curve (order + 1) g `shouldBe` g + it "a negative scalar is the negation of the positive one" $ do + ECC.pointMul p256curve (-1) g `shouldBe` ECC.pointNegate p256curve g + ECC.pointMul p256curve (-7) g + `shouldBe` ECC.pointNegate p256curve (ECC.pointMul p256curve 7 g) + it "a scalar past the order wraps" $ + ECC.pointMul p256curve (3 * order + 11) g `shouldBe` ECC.pointMul p256curve 11 g + it "a scalar wraps on either side of what 256 bits hold" $ do + -- the order is under 2^256 and twice it is over, so these are the + -- values around the boundary of a fixed-width reduction + ECC.pointMul p256curve (order - 1) g + `shouldBe` ECC.pointNegate p256curve g + ECC.pointMul p256curve (2 ^ (256 :: Int) - 1) g + `shouldBe` ECC.pointMul p256curve ((2 ^ (256 :: Int) - 1) `mod` order) g + ECC.pointMul p256curve (2 ^ (256 :: Int)) g + `shouldBe` ECC.pointMul p256curve (2 ^ (256 :: Int) `mod` order) g + ECC.pointMul p256curve (2 * order) g `shouldBe` ECC.PointO + ECC.pointMul p256curve (2 * order + 3) g `shouldBe` ECC.pointMul p256curve 3 g + it "zero and the point at infinity give infinity" $ do + ECC.pointMul p256curve 0 g `shouldBe` ECC.PointO + ECC.pointMul p256curve 5 ECC.PointO `shouldBe` ECC.PointO + it "multiplying a point that is not on the p256curve is unchanged" $ + -- the C implementation has no answer for these, so they stay with + -- the generic code; this pins what that answers + ECC.pointMul p256curve 5 offCurve + `shouldBe` ECC.pointMul p256curve 5 offCurve + it "the arithmetic modulo the order is the plain one" $ do + -- the C implementation takes 256 bits and the order is under + -- that, so these cross both the reduction and the fallback + let pairs = + [ (0, 0) + , (0, 7) + , (1, order - 1) + , (order - 1, order - 1) + , (order, order) + , (order + 1, 2) + , (2 ^ (256 :: Int) - 1, 2 ^ (256 :: Int) - 1) + , (2 ^ (256 :: Int), 3) + , (2 ^ (300 :: Int) + 5, 2 ^ (256 :: Int) + 9) + , (-3, 5) + , (3, -5) + ] + [ (a, b) + | (a, b) <- pairs + , ECC.scalarAdd p256curve a b /= (a + b) `mod` order + || ECC.scalarMul p256curve a b /= (a * b) `mod` order + ] + `shouldBe` [] + it "two muls is the sum of the muls, base point either side" $ do + ECC.pointAddTwoMuls p256curve 3 g 5 q + `shouldBe` ECC.pointAdd + p256curve + (ECC.pointMul p256curve 3 g) + (ECC.pointMul p256curve 5 q) + ECC.pointAddTwoMuls p256curve 5 q 3 g + `shouldBe` ECC.pointAdd + p256curve + (ECC.pointMul p256curve 5 q) + (ECC.pointMul p256curve 3 g) + ECC.pointAddTwoMuls p256curve order g 5 q `shouldBe` ECC.pointMul p256curve 5 q + where + p256curve = ECC.getCurveByName ECC.SEC_p256r1 + order = ECC.ecc_n (ECC.common_curve p256curve) + g = ECC.ecc_g (ECC.common_curve p256curve) + q = ECC.pointMul p256curve 0x2a3f1c9e g + offCurve = ECC.Point 1 1 + +-- | Multiplication the long way, out of the affine addition and doubling, +-- for the fast one to be held to. +doubleAndAdd :: ECC.Curve -> Integer -> ECC.Point -> ECC.Point +doubleAndAdd c k q = go (numBits k - 1) ECC.PointO + where + go i acc + | i < 0 = acc + | testBit k i = go (i - 1) (ECC.pointAdd c (ECC.pointDouble c acc) q) + | otherwise = go (i - 1) (ECC.pointDouble c acc) + +-- | A scalar multiplication over a prime field walks the bits of the scalar, +-- and what it does at a bit that is set differs from what it does at one that +-- is not. These are the scalars where that difference is starkest -- one bit +-- set, every bit set, alternating bits -- and what they pin is that all of +-- them still come out right. +weightTests :: Spec +weightTests = describe "scalars of every weight" $ do + check "P-384" ECC.SEC_p384r1 + check "P-521" ECC.SEC_p521r1 + where + check name curveName = describe name $ do + it "answers the same for a point that is not the base one" $ + -- the base point has a table of its own, and everything else + -- goes the long way round; both have to come out the same + [ k + | k <- [1, 2, 3, 15, 16, 17, n - 1, n, n + 1] + , ECC.pointMul c k other /= doubleAndAdd c k other + ] + `shouldBe` [] + it "adding two scalars is adding their multiples" $ + [ (a, b) + | (a, b) <- pairs + , ECC.pointMul c (a + b) g + /= ECC.pointAdd c (ECC.pointMul c a g) (ECC.pointMul c b g) + ] + `shouldBe` [] + where + c = ECC.getCurveByName curveName + n = ECC.ecc_n (ECC.common_curve c) + g = ECC.ecc_g (ECC.common_curve c) + other = ECC.pointMul c 5 g + bits = numBits n + ones k = 2 ^ k - 1 + alternating k = sum [2 ^ i | i <- [0, 2 .. k]] + pairs = + [ (1, 1) + , (2 ^ (bits - 2), 1) + , (ones (bits - 2), 1) + , (alternating (bits - 2), 3) + , (ones (bits - 2), alternating (bits - 2)) + , (n - 1, n - 1) + , -- twice the width of the order and more, which is what + -- recovering a public key hands to a multiplication + (n * n, 3) + , (n * n * n, alternating (bits - 2)) + ] + +-- | The curves over a binary field, whose multiplication is its own. The +-- point with no x is on every one of them -- y^2 = b has a root, since +-- squaring is a bijection there -- and it is its own negation, so doubling it +-- reaches infinity, which is the shape a multiplication is most likely to get +-- wrong. +binaryTests :: Spec +binaryTests = describe "binary curves" $ mapM_ check names + where + names = [ECC.SEC_t113r1, ECC.SEC_t163k1, ECC.SEC_t233r1, ECC.SEC_t283k1] + check name = describe (show name) $ do + it "agrees with a double-and-add at the edges" $ + [k | k <- scalars, ECC.pointMul c k g /= doubleAndAdd c k g] + `shouldBe` [] + it "answers for the point of order two" $ do + ECC.isPointValid c two `shouldBe` True + ECC.pointMul c 1 two `shouldBe` two + ECC.pointMul c 2 two `shouldBe` ECC.PointO + ECC.pointMul c 3 two `shouldBe` two + ECC.pointMul c (2 * order) two `shouldBe` ECC.PointO + it "agrees with a double-and-add from the point of order two" $ + [k | k <- take 6 scalars, ECC.pointMul c k two /= doubleAndAdd c k two] + `shouldBe` [] + where + c = ECC.getCurveByName name + cc = ECC.common_curve c + order = ECC.ecc_n cc + g = ECC.ecc_g cc + fx = ECC.ecc_fx (case c of ECC.CurveF2m b -> b; _ -> error "not binary") + -- the square root of b, which squaring being a bijection provides + two = ECC.Point 0 (iterate (squareF2m fx) (ECC.ecc_b cc) !! (numBits fx - 2)) + scalars = + [ 1 + , 2 + , 3 + , 15 + , 16 + , 17 + , order - 1 + , order + , order + 1 + , 2 * order + 3 + , order * order + ] + +-- | Points that satisfy the curve equation but lie outside the subgroup the +-- base point generates. One exists on every curve whose cofactor is not one, +-- and multiplying such a point by our private number gives a result that +-- depends on that number only through its residue modulo a small order, so +-- the other party learns those bits by offering the point and watching what +-- comes back. An exchange has to refuse them. +-- +-- On a binary curve the point with no x serves: y^2 = b has a root, since +-- squaring is a bijection there, and the point is its own negation, so its +-- order is two. The two prime curves that have a cofactor are given by their +-- coordinates, found by walking x upwards until the curve equation has a root +-- and the point it names is outside the subgroup. +outOfSubgroup :: [(ECC.CurveName, ECC.Point)] +outOfSubgroup = + [(name, orderTwo name) | name <- binaryNames] + ++ [ (ECC.SEC_p112r2, ECC.Point 0x2 0xbe6aa4938ef7cfe6fe29595b6b00) + , (ECC.SEC_p128r2, ECC.Point 0x1 0xcc7215732e64bd2ed528938cd8ef7b63) + ] + where + binaryNames = + [ ECC.SEC_t113r1 + , ECC.SEC_t113r2 + , ECC.SEC_t131r1 + , ECC.SEC_t131r2 + , ECC.SEC_t163k1 + , ECC.SEC_t163r1 + , ECC.SEC_t163r2 + , ECC.SEC_t193r1 + , ECC.SEC_t193r2 + , ECC.SEC_t233k1 + , ECC.SEC_t233r1 + , ECC.SEC_t239k1 + , ECC.SEC_t283k1 + , ECC.SEC_t283r1 + , ECC.SEC_t409k1 + , ECC.SEC_t409r1 + , ECC.SEC_t571k1 + , ECC.SEC_t571r1 + ] + orderTwo name = + let c = ECC.getCurveByName name + cc = ECC.common_curve c + fx = case c of + ECC.CurveF2m bc -> ECC.ecc_fx bc + _ -> error "orderTwo: not a binary curve" + in ECC.Point 0 (iterate (squareF2m fx) (ECC.ecc_b cc) !! (numBits fx - 2)) + +subgroupTests :: Spec +subgroupTests = + describe "public points outside the prime-order subgroup" $ + mapM_ check outOfSubgroup + where + -- either side of even, and either side of a number that needs more than + -- one limb, since what leaks is the residue and nothing else + privateNumbers = [2, 3, 100, 101, 3141592653589793238, 3141592653589793239] + check (name, q) = describe (show name) $ do + it "the point is on the curve" $ + ECC.isPointValid c q `shouldBe` True + it "the base point does not generate it" $ + ECC.pointMul c (ECC.ecc_n (ECC.common_curve c)) q + `shouldNotBe` ECC.PointO + it "and an exchange refuses it, whatever the private number" $ + [ d + | d <- privateNumbers + , ECDH.tryGetShared c d q + /= CryptoFailed CryptoError_PointSubgroupInvalid + ] + `shouldBe` [] + it "while a point the base point does generate is still accepted" $ + [d | d <- privateNumbers, not (passed (ECDH.tryGetShared c d peer))] + `shouldBe` [] + where + c = ECC.getCurveByName name + cc = ECC.common_curve c + -- what the other party would actually send: a multiple of the base + -- point, and so inside the subgroup by construction + peer = ECC.pointMul c 7 (ECC.ecc_g cc) + passed (CryptoPassed _) = True + passed (CryptoFailed _) = False + +spec :: Spec +spec = do + describe "valid-point" $ zipWithM_ doPointValidTest [katZero ..] vectorsPoint + p256Tests + weightTests + binaryTests + subgroupTests + modifyMaxSuccess (const 20) $ + describe "property" $ do + prop "point-add" $ \aCurve (QAInteger r1) (QAInteger r2) -> + let curveN = ECC.ecc_n . ECC.common_curve $ aCurve + curveGen = ECC.ecc_g . ECC.common_curve $ aCurve + p1 = ECC.pointMul aCurve r1 curveGen + p2 = ECC.pointMul aCurve r2 curveGen + pR = ECC.pointMul aCurve ((r1 + r2) `mod` curveN) curveGen + in pR `propertyEq` ECC.pointAdd aCurve p1 p2 + prop "point-negate-add" $ \aCurve -> do + p <- arbitraryPoint aCurve + let o = ECC.pointAdd aCurve p (ECC.pointNegate aCurve p) + return $ ECC.PointO `propertyEq` o + prop "point-negate-negate" $ \aCurve -> do + p <- arbitraryPoint aCurve + return $ p `propertyEq` ECC.pointNegate aCurve (ECC.pointNegate aCurve p) + prop "point-mul-mul" $ \aCurve (QAInteger n1) (QAInteger n2) -> do + p <- arbitraryPoint aCurve + let pRes = ECC.pointMul aCurve (n1 * n2) p + let pDef = ECC.pointMul aCurve n1 (ECC.pointMul aCurve n2 p) + return $ pRes `propertyEq` pDef + prop "point-mul-matches-double-and-add" $ \aCurve (QAInteger k) -> + let n = ECC.ecc_n (ECC.common_curve aCurve) + g = ECC.ecc_g (ECC.common_curve aCurve) + k' = 1 + k `mod` (n - 1) + in ECC.pointMul aCurve k' g == doubleAndAdd aCurve k' g + prop "scalar-arithmetic" $ \aCurve (QAInteger n1) (QAInteger n2) -> + let n = ECC.ecc_n (ECC.common_curve aCurve) + in ECC.scalarAdd aCurve n1 n2 + == (n1 + n2) `mod` n + && ECC.scalarMul aCurve n1 n2 + == (n1 * n2) `mod` n + -- against the long way round, not against pointMul: what + -- pointAddTwoMuls does with the two multiplications is the thing + -- under test, so holding it to the same pointMul it calls would + -- pin nothing. + prop "double-scalar-mult" $ \aCurve (QAInteger n1) (QAInteger n2) -> do + p1 <- arbitraryPoint aCurve + p2 <- arbitraryPoint aCurve + let pRes = ECC.pointAddTwoMuls aCurve n1 p1 n2 p2 + let pDef = + ECC.pointAdd + aCurve + (doubleAndAdd aCurve n1 p1) + (doubleAndAdd aCurve n2 p2) + return $ pRes `propertyEq` pDef diff --git a/tests/PubKey/ECDSASpec.hs b/tests/PubKey/ECDSASpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/ECDSASpec.hs @@ -0,0 +1,1671 @@ +-- The binary curves are deprecated and still supported, so the tests +-- that hold them to their behaviour name them on purpose. +{-# OPTIONS_GHC -Wno-deprecations #-} +{-# LANGUAGE ExistentialQuantification #-} +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.ECDSASpec (spec) where + +import Crypto.Hash +import Crypto.Number.Serialize +import Crypto.PubKey.ECC.ECDSA ( + PrivateKey (..), + PublicKey (..), + Signature (..), + deterministicNonce, + signWith, + verify, + ) +import Crypto.PubKey.ECC.Generate +import Crypto.PubKey.ECC.Types +import Data.ByteString (ByteString) +import qualified Data.ByteString as B +import Data.Maybe (isJust) +import Test.Hspec +import Text.Printf +import Utils (assertBool, assertFailure) + +-- existential type allows storing different hash algorithms in the same value +data HashAlg = forall hash. (Show hash, HashAlgorithm hash) => HashAlg hash +instance Show HashAlg where show (HashAlg alg) = show alg + +data Entry = Entry + { curveName :: CurveName + , privateNumber :: PrivateNumber + , publicPoint :: PublicPoint + , hashAlgorithm :: HashAlg + , message :: ByteString + , nonce :: Integer + , signature :: Signature + } +instance Show Entry where + show entry = + printf + "%s.%s.%s" + (show $ curveName entry) + (show $ B.take 8 $ message entry) + (show $ hashAlgorithm entry) + +normalize :: Entry -> Entry +normalize entry + | s <= n `div` 2 = entry + | otherwise = entry{signature = Signature r (n - s)} + where + Signature r s = signature entry + n = ecc_n $ common_curve $ getCurveByName $ curveName entry + +-- taken from GEC 2: Test Vectors for SEC 1 +gec2Entries :: [Entry] +gec2Entries = + [ Entry + { curveName = SEC_p160r1 + , privateNumber = 971761939728640320549601132085879836204587084162 + , publicPoint = + Point + 466448783855397898016055842232266600516272889280 + 1110706324081757720403272427311003102474457754220 + , hashAlgorithm = HashAlg SHA1 + , message = "abc" + , nonce = 702232148019446860144825009548118511996283736794 + , signature = + Signature + { sign_r = 1176954224688105769566774212902092897866168635793 + , sign_s = 299742580584132926933316745664091704165278518100 + } + } + , Entry + { curveName = SEC_t163k1 + , privateNumber = 0x00000011f2626d90d26cb4c0379043b26e64107fc + , publicPoint = + Point + 0x0389fa5ad7f8304325a8c060ef7dcb83042c045bc + 0x0eefa094a5054da196943cc80509dcb9f59e5bc2e + , hashAlgorithm = HashAlg SHA1 + , message = + i2osp + 0xa2c1a03fdd00521bb08fc88d20344321977aaf637ef9d5470dd7d2c8628fc8d0d1f1d3587c6b3fd02386f8c13db341b14748a9475cc63baf065df64054b27d5c2cdf0f98e3bbb81d0b5dc94f8cdb87acf75720f6163de394c8c6af360bc1acb85b923a493b7b27cc111a257e36337bd94eb0fab9d5e633befb1ae7f1b244bfaa + , nonce = 0x0000000c3a4ff97286126dab1e5089395fcc47ebb + , signature = + Signature + { sign_r = 0x0dbe6c3a1dc851e7f2338b5c26c62b4b37bf8035c + , sign_s = 0x1c76458135b1ff9fbd23009b8414a47996126b56a + } + } + , Entry + { curveName = SEC_t163k1 + , privateNumber = 0x00000006a3803301daee9af09bb5b6c991a4f49a4 + , publicPoint = + Point + 0x4b500f555e857da8c299780130c5c3f48f02ee322 + 0x5c1c0ae25b47f06cc46fb86b12d2d8c0ba6a4bf07 + , hashAlgorithm = HashAlg SHA1 + , message = + i2osp + 0x67048080daaeb77d3ac31babdf8be23dbe75ceb4dfb94aa8113db5c5dcb6fe14b70f717b7b0ed0881835a66a86e6d840ffcb7d976c75ef2d1d4322fbbc86357384e24707aef88cea2c41a01a9a3d1b9e72ce650c7fdecc4f9448d3a77df6cdf13647ab295bb3132de0b1b2c402d8d2de7d452f1e003e0695de1470d1064eee16 + , nonce = 0x0000002f39fbf77f3e0dc046116de692b6cf91b16 + , signature = + Signature + { sign_r = 0x3d3eeda42f65d727f4a564f1415654356c6c57a6c + , sign_s = 0x35e4d43c5f08baddf138449db1ad0b7872552b7cd + } + } + , Entry + { curveName = SEC_t163k1 + , privateNumber = 0x0000002e28676514bd93fea11b62db0f6e324b18d + , publicPoint = + Point + 0x3f9c90b71f6a1de20a2716f38ef1b5f98c757bd42 + 0x2ff0a5d266d447ef62d43fbca6c34c08c1ce35a40 + , hashAlgorithm = HashAlg SHA1 + , message = + i2osp + 0x77e007dc2acd7248256165a4b30e98986f51a81efd926b85f74c81bc2a6d2bcd030060a844091e22fbb0ff3db5a20caaefb5d58ccdcbc27f0ff8a4d940e78f303079ec1ca5b0ca3d4ecc7580f8b34a9f0496c9e719d2ec3e1614b7644bc11179e895d2c0b58a1da204fbf0f6e509f97f983eacb6487092caf6e8e4e6b3c458b2 + , nonce = 0x00000001233ae699883e74e7f4dfb5279ff22280a + , signature = + Signature + { sign_r = 0x39de3cd2cf04145e522b8fba3f23e9218226e0860 + , sign_s = 0x2af62bfb3cfa202e2342606ee5bb0934c3b0375b6 + } + } + , Entry + { curveName = SEC_t163k1 + , privateNumber = 0x000000361dd088e3a6d3c910686c8dce57e5d4d8e + , publicPoint = + Point + 0x064f905c1da9d7e9c32d81890ae6f30dcc7839d32 + 0x06f1faedb6d9032016d3b681e7cf69c29d29eb27b + , hashAlgorithm = HashAlg SHA1 + , message = + i2osp + 0xfbacfcce4688748406ddf5c3495021eef8fb399865b649eb2395a04a1ab28335da2c236d306fcc59f7b65ea931cf0139571e1538ede5688958c3ac69f47a285362f5ad201f89cc735b7b465408c2c41b310fc8908d0be45054df2a7351fae36b390e842f3b5cdd9ad832940df5b2d25c2ed43ce86eaf2508bcf401ae58bb1d47 + , nonce = 0x00000022f723e9f5da56d3d0837d5dca2f937395f + , signature = + Signature + { sign_r = 0x374cdc8571083fecfbd4e25e1cd69ecc66b715f2d + , sign_s = 0x313b10949222929b2f20b15d446c27d6dcae3f086 + } + } + ] + +data EntryCurve = EntryCurve + { ecName :: CurveName + , ecPrivate :: PrivateNumber + , ecPublic :: PublicPoint + , ecMessages :: [EntryMessage] + } +data EntryMessage = EntryMessage + { emMessage :: ByteString + , emHashes :: [EntryHash] + } +data EntryHash = EntryHash + { ehAlgorithm :: HashAlg + , ehK :: Integer + , ehR :: Integer + , ehS :: Integer + } + +flatten :: [EntryCurve] -> [Entry] +flatten hierarchy = do + entryCurve <- hierarchy + entryMessage <- ecMessages entryCurve + entryHash <- emHashes entryMessage + pure $ + Entry + { curveName = ecName entryCurve + , privateNumber = ecPrivate entryCurve + , publicPoint = ecPublic entryCurve + , hashAlgorithm = ehAlgorithm entryHash + , message = emMessage entryMessage + , nonce = ehK entryHash + , signature = Signature (ehR entryHash) (ehS entryHash) + } + +-- taken from RFC 6979 +rfc6979Entries :: [EntryCurve] +rfc6979Entries = + [ EntryCurve + { ecName = SEC_p192r1 + , ecPrivate = 0x6FAB034934E4C0FC9AE67F5B5659A9D7D1FEFD187EE09FD4 + , ecPublic = + Point + 0xAC2C77F529F91689FEA0EA5EFEC7F210D8EEA0B9E047ED56 + 0x3BC723E57670BD4887EBC732C523063D0A7C957BC97C1C43 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x37D7CA00D2C7B0E5E412AC03BD44BA837FDD5B28CD3B0021 + , ehR = 0x98C6BD12B23EAF5E2A2045132086BE3EB8EBD62ABF6698FF + , ehS = 0x57A22B07DEA9530F8DE9471B1DC6624472E8E2844BC25B64 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x4381526B3FC1E7128F202E194505592F01D5FF4C5AF015D8 + , ehR = 0xA1F00DAD97AEEC91C95585F36200C65F3C01812AA60378F5 + , ehS = 0xE07EC1304C7C6C9DEBBE980B9692668F81D4DE7922A0F97A + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x32B1B6D7D42A05CB449065727A84804FB1A3E34D8F261496 + , ehR = 0x4B0B8CE98A92866A2820E20AA6B75B56382E0F9BFD5ECB55 + , ehS = 0xCCDB006926EA9565CBADC840829D8C384E06DE1F1E381B85 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x4730005C4FCB01834C063A7B6760096DBE284B8252EF4311 + , ehR = 0xDA63BF0B9ABCF948FBB1E9167F136145F7A20426DCC287D5 + , ehS = 0xC3AA2C960972BD7A2003A57E1C4C77F0578F8AE95E31EC5E + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0xA2AC7AB055E4F20692D49209544C203A7D1F2C0BFBC75DB1 + , ehR = 0x4D60C5AB1996BD848343B31C00850205E2EA6922DAC2E4B8 + , ehS = 0x3F6E837448F027A1BF4B34E796E32A811CBB4050908D8F67 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0xD9CF9C3D3297D3260773A1DA7418DB5537AB8DD93DE7FA25 + , ehR = 0x0F2141A0EBBC44D2E1AF90A50EBCFCE5E197B3B7D4DE036D + , ehS = 0xEB18BC9E1F3D7387500CB99CF5F7C157070A8961E38700B7 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0xF5DC805F76EF851800700CCE82E7B98D8911B7D510059FBE + , ehR = 0x6945A1C1D1B2206B8145548F633BB61CEF04891BAF26ED34 + , ehS = 0xB7FB7FDFC339C0B9BD61A9F5A8EAF9BE58FC5CBA2CB15293 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x5C4CE89CF56D9E7C77C8585339B006B97B5F0680B4306C6C + , ehR = 0x3A718BD8B4926C3B52EE6BBE67EF79B18CB6EB62B1AD97AE + , ehS = 0x5662E6848A4A19B1F1AE2F72ACD4B8BBE50F1EAC65D9124F + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x5AFEFB5D3393261B828DB6C91FBC68C230727B030C975693 + , ehR = 0xB234B60B4DB75A733E19280A7A6034BD6B1EE88AF5332367 + , ehS = 0x7994090B2D59BB782BE57E74A44C9A1C700413F8ABEFE77A + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x0758753A5254759C7CFBAD2E2D9B0792EEE44136C9480527 + , ehR = 0xFE4F4AE86A58B6507946715934FE2D8FF9D95B6B098FE739 + , ehS = 0x74CF5605C98FBA0E1EF34D4B5A1577A7DCF59457CAE52290 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_p224r1 + , ecPrivate = 0xF220266E1105BFE3083E03EC7A3A654651F45E37167E88600BF257C1 + , ecPublic = + Point + 0x00CF08DA5AD719E42707FA431292DEA11244D64FC51610D94B130D6C + 0xEEAB6F3DEBE455E3DBF85416F7030CBD94F34F2D6F232C69F3C1385A + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x7EEFADD91110D8DE6C2C470831387C50D3357F7F4D477054B8B426BC + , ehR = 0x22226F9D40A96E19C4A301CE5B74B115303C0F3A4FD30FC257FB57AC + , ehS = 0x66D1CDD83E3AF75605DD6E2FEFF196D30AA7ED7A2EDF7AF475403D69 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0xC1D1F2F10881088301880506805FEB4825FE09ACB6816C36991AA06D + , ehR = 0x1CDFE6662DDE1E4A1EC4CDEDF6A1F5A2FB7FBD9145C12113E6ABFD3E + , ehS = 0xA6694FD7718A21053F225D3F46197CA699D45006C06F871808F43EBC + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0xAD3029E0278F80643DE33917CE6908C70A8FF50A411F06E41DEDFCDC + , ehR = 0x61AA3DA010E8E8406C656BC477A7A7189895E7E840CDFE8FF42307BA + , ehS = 0xBC814050DAB5D23770879494F9E0A680DC1AF7161991BDE692B10101 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x52B40F5A9D3D13040F494E83D3906C6079F29981035C7BD51E5CAC40 + , ehR = 0x0B115E5E36F0F9EC81F1325A5952878D745E19D7BB3EABFABA77E953 + , ehS = 0x830F34CCDFE826CCFDC81EB4129772E20E122348A2BBD889A1B1AF1D + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x9DB103FFEDEDF9CFDBA05184F925400C1653B8501BAB89CEA0FBEC14 + , ehR = 0x074BD1D979D5F32BF958DDC61E4FB4872ADCAFEB2256497CDAC30397 + , ehS = 0xA4CECA196C3D5A1FF31027B33185DC8EE43F288B21AB342E5D8EB084 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x2519178F82C3F0E4F87ED5883A4E114E5B7A6E374043D8EFD329C253 + , ehR = 0xDEAA646EC2AF2EA8AD53ED66B2E2DDAA49A12EFD8356561451F3E21C + , ehS = 0x95987796F6CF2062AB8135271DE56AE55366C045F6D9593F53787BD2 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0xDF8B38D40DCA3E077D0AC520BF56B6D565134D9B5F2EAE0D34900524 + , ehR = 0xC441CE8E261DED634E4CF84910E4C5D1D22C5CF3B732BB204DBEF019 + , ehS = 0x902F42847A63BDC5F6046ADA114953120F99442D76510150F372A3F4 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0xFF86F57924DA248D6E44E8154EB69F0AE2AEBAEE9931D0B5A969F904 + , ehR = 0xAD04DDE87B84747A243A631EA47A1BA6D1FAA059149AD2440DE6FBA6 + , ehS = 0x178D49B1AE90E3D8B629BE3DB5683915F4E8C99FDF6E666CF37ADCFD + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x7046742B839478C1B5BD31DB2E862AD868E1A45C863585B5F22BDC2D + , ehR = 0x389B92682E399B26518A95506B52C03BC9379A9DADF3391A21FB0EA4 + , ehS = 0x414A718ED3249FF6DBC5B50C27F71F01F070944DA22AB1F78F559AAB + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0xE39C2AA4EA6BE2306C72126D40ED77BF9739BB4D6EF2BBB1DCB6169D + , ehR = 0x049F050477C5ADD858CAC56208394B5A55BAEBBE887FDF765047C17C + , ehS = 0x077EB13E7005929CEFA3CD0403C7CDCC077ADF4E44F3C41B2F60ECFF + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_p256r1 + , ecPrivate = 0xC9AFA9D845BA75166B5C215767B1D6934E50C3DB36E89B127B8A622B120F6721 + , ecPublic = + Point + 0x60FED4BA255A9D31C961EB74C6356D68C049B8923B61FA6CE669622E60F29FB6 + 0x7903FE1008B8BC99A41AE9E95628BC64F2F1B20C2D7E9F5177A3C294D4462299 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x882905F1227FD620FBF2ABF21244F0BA83D0DC3A9103DBBEE43A1FB858109DB4 + , ehR = 0x61340C88C3AAEBEB4F6D667F672CA9759A6CCAA9FA8811313039EE4A35471D32 + , ehS = 0x6D7F147DAC089441BB2E2FE8F7A3FA264B9C475098FDCF6E00D7C996E1B8B7EB + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x103F90EE9DC52E5E7FB5132B7033C63066D194321491862059967C715985D473 + , ehR = 0x53B2FFF5D1752B2C689DF257C04C40A587FABABB3F6FC2702F1343AF7CA9AA3F + , ehS = 0xB9AFB64FDC03DC1A131C7D2386D11E349F070AA432A4ACC918BEA988BF75C74C + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0xA6E3C57DD01ABE90086538398355DD4C3B17AA873382B0F24D6129493D8AAD60 + , ehR = 0xEFD48B2AACB6A8FD1140DD9CD45E81D69D2C877B56AAF991C34D0EA84EAF3716 + , ehS = 0xF7CB1C942D657C41D436C7A1B6E29F65F3E900DBB9AFF4064DC4AB2F843ACDA8 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x09F634B188CEFD98E7EC88B1AA9852D734D0BC272F7D2A47DECC6EBEB375AAD4 + , ehR = 0x0EAFEA039B20E9B42309FB1D89E213057CBF973DC0CFC8F129EDDDC800EF7719 + , ehS = 0x4861F0491E6998B9455193E34E7B0D284DDD7149A74B95B9261F13ABDE940954 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x5FA81C63109BADB88C1F367B47DA606DA28CAD69AA22C4FE6AD7DF73A7173AA5 + , ehR = 0x8496A60B5E9B47C825488827E0495B0E3FA109EC4568FD3F8D1097678EB97F00 + , ehS = 0x2362AB1ADBE2B8ADF9CB9EDAB740EA6049C028114F2460F96554F61FAE3302FE + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x8C9520267C55D6B980DF741E56B4ADEE114D84FBFA2E62137954164028632A2E + , ehR = 0x0CBCC86FD6ABD1D99E703E1EC50069EE5C0B4BA4B9AC60E409E8EC5910D81A89 + , ehS = 0x01B9D7B73DFAA60D5651EC4591A0136F87653E0FD780C3B1BC872FFDEAE479B1 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x669F4426F2688B8BE0DB3A6BD1989BDAEFFF84B649EEB84F3DD26080F667FAA7 + , ehR = 0xC37EDB6F0AE79D47C3C27E962FA269BB4F441770357E114EE511F662EC34A692 + , ehS = 0xC820053A05791E521FCAAD6042D40AEA1D6B1A540138558F47D0719800E18F2D + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0xD16B6AE827F17175E040871A1C7EC3500192C4C92677336EC2537ACAEE0008E0 + , ehR = 0xF1ABB023518351CD71D881567B1EA663ED3EFCF6C5132B354F28D3B0B7D38367 + , ehS = 0x019F4113742A2B14BD25926B49C649155F267E60D3814B4C0CC84250E46F0083 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x16AEFFA357260B04B1DD199693960740066C1A8F3E8EDD79070AA914D361B3B8 + , ehR = 0x83910E8B48BB0C74244EBDF7F07A1C5413D61472BD941EF3920E623FBCCEBEB6 + , ehS = 0x8DDBEC54CF8CD5874883841D712142A56A8D0F218F5003CB0296B6B509619F2C + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x6915D11632ACA3C40D5D51C08DAF9C555933819548784480E93499000D9F0B7F + , ehR = 0x461D93F31B6540894788FD206C07CFA0CC35F46FA3C91816FFF1040AD1581A04 + , ehS = 0x39AF9F15DE0DB8D97E72719C74820D304CE5226E32DEDAE67519E840D1194E55 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_p384r1 + , ecPrivate = + 0x6B9D3DAD2E1B8C1C05B19875B6659F4DE23C3B667BF297BA9AA47740787137D896D5724E4C70A825F872C9EA60D2EDF5 + , ecPublic = + Point + 0xEC3A4E415B4E19A4568618029F427FA5DA9A8BC4AE92E02E06AAE5286B300C64DEF8F0EA9055866064A254515480BC13 + 0x8015D9B72D7D57244EA8EF9AC0C621896708A59367F9DFB9F54CA84B3F1C9DB1288B231C3AE0D4FE7344FD2533264720 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x4471EF7518BB2C7C20F62EAE1C387AD0C5E8E470995DB4ACF694466E6AB096630F29E5938D25106C3C340045A2DB01A7 + , ehR = + 0xEC748D839243D6FBEF4FC5C4859A7DFFD7F3ABDDF72014540C16D73309834FA37B9BA002899F6FDA3A4A9386790D4EB2 + , ehS = + 0xA3BCFA947BEEF4732BF247AC17F71676CB31A847B9FF0CBC9C9ED4C1A5B3FACF26F49CA031D4857570CCB5CA4424A443 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0xA4E4D2F0E729EB786B31FC20AD5D849E304450E0AE8E3E341134A5C1AFA03CAB8083EE4E3C45B06A5899EA56C51B5879 + , ehR = + 0x42356E76B55A6D9B4631C865445DBE54E056D3B3431766D0509244793C3F9366450F76EE3DE43F5A125333A6BE060122 + , ehS = + 0x9DA0C81787064021E78DF658F2FBB0B042BF304665DB721F077A4298B095E4834C082C03D83028EFBF93A3C23940CA8D + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x180AE9F9AEC5438A44BC159A1FCB277C7BE54FA20E7CF404B490650A8ACC414E375572342863C899F9F2EDF9747A9B60 + , ehR = + 0x21B13D1E013C7FA1392D03C5F99AF8B30C570C6F98D4EA8E354B63A21D3DAA33BDE1E888E63355D92FA2B3C36D8FB2CD + , ehS = + 0xF3AA443FB107745BF4BD77CB3891674632068A10CA67E3D45DB2266FA7D1FEEBEFDC63ECCD1AC42EC0CB8668A4FA0AB0 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x94ED910D1A099DAD3254E9242AE85ABDE4BA15168EAF0CA87A555FD56D10FBCA2907E3E83BA95368623B8C4686915CF9 + , ehR = + 0x94EDBB92A5ECB8AAD4736E56C691916B3F88140666CE9FA73D64C4EA95AD133C81A648152E44ACF96E36DD1E80FABE46 + , ehS = + 0x99EF4AEB15F178CEA1FE40DB2603138F130E740A19624526203B6351D0A3A94FA329C145786E679E7B82C71A38628AC8 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x92FC3C7183A883E24216D1141F1A8976C5B0DD797DFA597E3D7B32198BD35331A4E966532593A52980D0E3AAA5E10EC3 + , ehR = + 0xED0959D5880AB2D869AE7F6C2915C6D60F96507F9CB3E047C0046861DA4A799CFE30F35CC900056D7C99CD7882433709 + , ehS = + 0x512C8CCEEE3890A84058CE1E22DBC2198F42323CE8ACA9135329F03C068E5112DC7CC3EF3446DEFCEB01A45C2667FDD5 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x66CC2C8F4D303FC962E5FF6A27BD79F84EC812DDAE58CF5243B64A4AD8094D47EC3727F3A3C186C15054492E30698497 + , ehR = + 0x4BC35D3A50EF4E30576F58CD96CE6BF638025EE624004A1F7789A8B8E43D0678ACD9D29876DAF46638645F7F404B11C7 + , ehS = + 0xD5A6326C494ED3FF614703878961C0FDE7B2C278F9A65FD8C4B7186201A2991695BA1C84541327E966FA7B50F7382282 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x18FA39DB95AA5F561F30FA3591DC59C0FA3653A80DAFFA0B48D1A4C6DFCBFF6E3D33BE4DC5EB8886A8ECD093F2935726 + , ehR = + 0xE8C9D0B6EA72A0E7837FEA1D14A1A9557F29FAA45D3E7EE888FC5BF954B5E62464A9A817C47FF78B8C11066B24080E72 + , ehS = + 0x07041D4A7A0379AC7232FF72E6F77B6DDB8F09B16CCE0EC3286B2BD43FA8C6141C53EA5ABEF0D8231077A04540A96B66 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x0CFAC37587532347DC3389FDC98286BBA8C73807285B184C83E62E26C401C0FAA48DD070BA79921A3457ABFF2D630AD7 + , ehR = + 0x6D6DEFAC9AB64DABAFE36C6BF510352A4CC27001263638E5B16D9BB51D451559F918EEDAF2293BE5B475CC8F0188636B + , ehS = + 0x2D46F3BECBCC523D5F1A1256BF0C9B024D879BA9E838144C8BA6BAEB4B53B47D51AB373F9845C0514EEFB14024787265 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x015EE46A5BF88773ED9123A5AB0807962D193719503C527B031B4C2D225092ADA71F4A459BC0DA98ADB95837DB8312EA + , ehR = + 0x8203B63D3C853E8D77227FB377BCF7B7B772E97892A80F36AB775D509D7A5FEB0542A7F0812998DA8F1DD3CA3CF023DB + , ehS = + 0xDDD0760448D42D8A43AF45AF836FCE4DE8BE06B485E9B61B827C2F13173923E06A739F040649A667BF3B828246BAA5A5 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x3780C4F67CB15518B6ACAE34C9F83568D2E12E47DEAB6C50A4E4EE5319D1E8CE0E2CC8A136036DC4B9C00E6888F66B6C + , ehR = + 0xA0D5D090C9980FAF3C2CE57B7AE951D31977DD11C775D314AF55F76C676447D06FB6495CD21B4B6E340FC236584FB277 + , ehS = + 0x976984E59B4C77B0E8E4460DCA3D9F20E07B9BB1F63BEEFAF576F6B2E8B224634A2092CD3792E0159AD9CEE37659C736 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_p521r1 + , ecPrivate = + 0x0FAD06DAA62BA3B25D2FB40133DA757205DE67F5BB0018FEE8C86E1B68C7E75CAA896EB32F1F47C70855836A6D16FCC1466F6D8FBEC67DB89EC0C08B0E996B83538 + , ecPublic = + Point + 0x1894550D0785932E00EAA23B694F213F8C3121F86DC97A04E5A7167DB4E5BCD371123D46E45DB6B5D5370A7F20FB633155D38FFA16D2BD761DCAC474B9A2F5023A4 + 0x0493101C962CD4D2FDDF782285E64584139C2F91B47F87FF82354D6630F746A28A0DB25741B5B34A828008B22ACC23F924FAAFBD4D33F81EA66956DFEAA2BFDFCF5 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x089C071B419E1C2820962321787258469511958E80582E95D8378E0C2CCDB3CB42BEDE42F50E3FA3C71F5A76724281D31D9C89F0F91FC1BE4918DB1C03A5838D0F9 + , ehR = + 0x0343B6EC45728975EA5CBA6659BBB6062A5FF89EEA58BE3C80B619F322C87910FE092F7D45BB0F8EEE01ED3F20BABEC079D202AE677B243AB40B5431D497C55D75D + , ehS = + 0x0E7B0E675A9B24413D448B8CC119D2BF7B2D2DF032741C096634D6D65D0DBE3D5694625FB9E8104D3B842C1B0E2D0B98BEA19341E8676AEF66AE4EBA3D5475D5D16 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x121415EC2CD7726330A61F7F3FA5DE14BE9436019C4DB8CB4041F3B54CF31BE0493EE3F427FB906393D895A19C9523F3A1D54BB8702BD4AA9C99DAB2597B92113F3 + , ehR = + 0x1776331CFCDF927D666E032E00CF776187BC9FDD8E69D0DABB4109FFE1B5E2A30715F4CC923A4A5E94D2503E9ACFED92857B7F31D7152E0F8C00C15FF3D87E2ED2E + , ehS = + 0x050CB5265417FE2320BBB5A122B8E1A32BD699089851128E360E620A30C7E17BA41A666AF126CE100E5799B153B60528D5300D08489CA9178FB610A2006C254B41F + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x0EDF38AFCAAECAB4383358B34D67C9F2216C8382AAEA44A3DAD5FDC9C32575761793FEF24EB0FC276DFC4F6E3EC476752F043CF01415387470BCBD8678ED2C7E1A0 + , ehR = + 0x1511BB4D675114FE266FC4372B87682BAECC01D3CC62CF2303C92B3526012659D16876E25C7C1E57648F23B73564D67F61C6F14D527D54972810421E7D87589E1A7 + , ehS = + 0x04A171143A83163D6DF460AAF61522695F207A58B95C0644D87E52AA1A347916E4F7A72930B1BC06DBE22CE3F58264AFD23704CBB63B29B931F7DE6C9D949A7ECFC + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x1546A108BC23A15D6F21872F7DED661FA8431DDBD922D0DCDB77CC878C8553FFAD064C95A920A750AC9137E527390D2D92F153E66196966EA554D9ADFCB109C4211 + , ehR = + 0x1EA842A0E17D2DE4F92C15315C63DDF72685C18195C2BB95E572B9C5136CA4B4B576AD712A52BE9730627D16054BA40CC0B8D3FF035B12AE75168397F5D50C67451 + , ehS = + 0x1F21A3CEE066E1961025FB048BD5FE2B7924D0CD797BABE0A83B66F1E35EEAF5FDE143FA85DC394A7DEE766523393784484BDF3E00114A1C857CDE1AA203DB65D61 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x1DAE2EA071F8110DC26882D4D5EAE0621A3256FC8847FB9022E2B7D28E6F10198B1574FDD03A9053C08A1854A168AA5A57470EC97DD5CE090124EF52A2F7ECBFFD3 + , ehR = + 0x0C328FAFCBD79DD77850370C46325D987CB525569FB63C5D3BC53950E6D4C5F174E25A1EE9017B5D450606ADD152B534931D7D4E8455CC91F9B15BF05EC36E377FA + , ehS = + 0x0617CCE7CF5064806C467F678D3B4080D6F1CC50AF26CA209417308281B68AF282623EAA63E5B5C0723D8B8C37FF0777B1A20F8CCB1DCCC43997F1EE0E44DA4A67A + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x0BB9F2BF4FE1038CCF4DABD7139A56F6FD8BB1386561BD3C6A4FC818B20DF5DDBA80795A947107A1AB9D12DAA615B1ADE4F7A9DC05E8E6311150F47F5C57CE8B222 + , ehR = + 0x13BAD9F29ABE20DE37EBEB823C252CA0F63361284015A3BF430A46AAA80B87B0693F0694BD88AFE4E661FC33B094CD3B7963BED5A727ED8BD6A3A202ABE009D0367 + , ehS = + 0x1E9BB81FF7944CA409AD138DBBEE228E1AFCC0C890FC78EC8604639CB0DBDC90F717A99EAD9D272855D00162EE9527567DD6A92CBD629805C0445282BBC916797FF + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x040D09FCF3C8A5F62CF4FB223CBBB2B9937F6B0577C27020A99602C25A01136987E452988781484EDBBCF1C47E554E7FC901BC3085E5206D9F619CFF07E73D6F706 + , ehR = + 0x1C7ED902E123E6815546065A2C4AF977B22AA8EADDB68B2C1110E7EA44D42086BFE4A34B67DDC0E17E96536E358219B23A706C6A6E16BA77B65E1C595D43CAE17FB + , ehS = + 0x177336676304FCB343CE028B38E7B4FBA76C1C1B277DA18CAD2A8478B2A9A9F5BEC0F3BA04F35DB3E4263569EC6AADE8C92746E4C82F8299AE1B8F1739F8FD519A4 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x01DE74955EFAABC4C4F17F8E84D881D1310B5392D7700275F82F145C61E843841AF09035BF7A6210F5A431A6A9E81C9323354A9E69135D44EBD2FCAA7731B909258 + , ehR = + 0x00E871C4A14F993C6C7369501900C4BC1E9C7B0B4BA44E04868B30B41D8071042EB28C4C250411D0CE08CD197E4188EA4876F279F90B3D8D74A3C76E6F1E4656AA8 + , ehS = + 0x0CD52DBAA33B063C3A6CD8058A1FB0A46A4754B034FCC644766CA14DA8CA5CA9FDE00E88C1AD60CCBA759025299079D7A427EC3CC5B619BFBC828E7769BCD694E86 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x1F1FC4A349A7DA9A9E116BFDD055DC08E78252FF8E23AC276AC88B1770AE0B5DCEB1ED14A4916B769A523CE1E90BA22846AF11DF8B300C38818F713DADD85DE0C88 + , ehR = + 0x14BEE21A18B6D8B3C93FAB08D43E739707953244FDBE924FA926D76669E7AC8C89DF62ED8975C2D8397A65A49DCC09F6B0AC62272741924D479354D74FF6075578C + , ehS = + 0x133330865C067A0EAF72362A65E2D7BC4E461E8C8995C3B6226A21BD1AA78F0ED94FE536A0DCA35534F0CD1510C41525D163FE9D74D134881E35141ED5E8E95B979 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x16200813020EC986863BEDFC1B121F605C1215645018AEA1A7B215A564DE9EB1B38A67AA1128B80CE391C4FB71187654AAA3431027BFC7F395766CA988C964DC56D + , ehR = + 0x13E99020ABF5CEE7525D16B69B229652AB6BDF2AFFCAEF38773B4B7D08725F10CDB93482FDCC54EDCEE91ECA4166B2A7C6265EF0CE2BD7051B7CEF945BABD47EE6D + , ehS = + 0x1FBD0013C674AA79CB39849527916CE301C66EA7CE8B80682786AD60F98F7E78A19CA69EFF5C57400E3B3A0AD66CE0978214D13BAF4E9AC60752F7B155E2DE4DCE3 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t163k1 + , ecPrivate = 0x09A4D6792295A7F730FC3F2B49CBC0F62E862272F + , ecPublic = + Point + 0x79AEE090DB05EC252D5CB4452F356BE198A4FF96F + 0x782E29634DDC9A31EF40386E896BAA18B53AFA5A3 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x09744429FA741D12DE2BE8316E35E84DB9E5DF1CD + , ehR = 0x30C45B80BA0E1406C4EFBBB7000D6DE4FA465D505 + , ehS = 0x38D87DF89493522FC4CD7DE1553BD9DBBA2123011 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x323E7B28BFD64E6082F5B12110AA87BC0D6A6E159 + , ehR = 0x38A2749F7EA13BD5DA0C76C842F512D5A65FFAF32 + , ehS = 0x064F841F70112B793FD773F5606BFA5AC2A04C1E8 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x23AF4074C90A02B3FE61D286D5C87F425E6BDD81B + , ehR = 0x113A63990598A3828C407C0F4D2438D990DF99A7F + , ehS = 0x1313A2E03F5412DDB296A22E2C455335545672D9F + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x2132ABE0ED518487D3E4FA7FD24F8BED1F29CCFCE + , ehR = 0x34D4DE955871BB84FEA4E7D068BA5E9A11BD8B6C4 + , ehS = 0x2BAAF4D4FD57F175C405A2F39F9755D9045C820BD + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x00BBCC2F39939388FDFE841892537EC7B1FF33AA3 + , ehR = 0x38E487F218D696A7323B891F0CCF055D895B77ADC + , ehS = 0x0972D7721093F9B3835A5EB7F0442FA8DCAA873C4 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x14CAB9192F39C8A0EA8E81B4B87574228C99CD681 + , ehR = 0x1375BEF93F21582F601497036A7DC8014A99C2B79 + , ehS = 0x254B7F1472FFFEE9002D081BB8CE819CCE6E687F9 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x091DD986F38EB936BE053DD6ACE3419D2642ADE8D + , ehR = 0x110F17EF209957214E35E8C2E83CBE73B3BFDEE2C + , ehS = 0x057D5022392D359851B95DEC2444012502A5349CB + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x193649CE51F0CFF0784CFC47628F4FA854A93F7A2 + , ehR = 0x0354D5CD24F9C41F85D02E856FA2B0001C83AF53E + , ehS = 0x020B200677731CD4FE48612A92F72A19853A82B65 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x37C73C6F8B404EC83DA17A6EBCA724B3FF1F7EEBA + , ehR = 0x11B6A84206515495AD8DBB2E5785D6D018D75817E + , ehS = 0x1A7D4C1E17D4030A5D748ADEA785C77A54581F6D0 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x331AD98D3186F73967B1E0B120C80B1E22EFC2988 + , ehR = 0x148934745B351F6367FF5BB56B1848A2F508902A9 + , ehS = 0x36214B19444FAB504DBA61D4D6FF2D2F9640F4837 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t233k1 + , ecPrivate = 0x103B2142BDC2A3C3B55080D09DF1808F79336DA2399F5CA7171D1BE9B0 + , ecPublic = + Point + 0x0682886F36C68473C1A221720C2B12B9BE13458BA907E1C4736595779F2 + 0x1B20639B41BE0927090999B7817A3B3928D20503A39546044EC13A10309 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x273179E3E12C69591AD3DD9C7CCE3985820E3913AB6696EB14486DDBCF + , ehR = 0x5474541C988A9A1F73899F55EF28963DFFBBF0C2B1A1EE787C6A76C6A4 + , ehS = 0x46301F9EC6624257BFC70D72186F17898EDBD0A3522560A88DD1B7D45A + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x71626A309D9CD80AD0B975D757FE6BF4B84E49F8F34C780070D7746F19 + , ehR = 0x667F2FCE3E1C497EBD8E4B7C6372A8234003FE4ED6D4515814E7E11430 + , ehS = 0x6A1C41340DAA730320DB9475F10E29A127D7AE3432F155E1F7954E1B57 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x73552F9CAC5774F74F485FA253871F2109A0C86040552EAA67DBA92DC9 + , ehR = 0x38AD9C1D2CB29906E7D63C24601AC55736B438FB14F4093D6C32F63A10 + , ehS = 0x647AAD2599C21B6EE89BE7FF957D98F684B7921DE1FD3CC82C079624F4 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x17D726A67539C609BD99E29AA3737EF247724B71455C3B6310034038C8 + , ehR = 0x0C6510F57559C36FBCFF8C7BA4B81853DC618AD0BAAB03CFFDF3FD09FD + , ehS = 0x0AD331EE1C9B91A88BA77997235769C60AD07EE69E11F7137E17C5CF67 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x0E535C328774CDE546BE3AF5D7FCD263872F107E807435105BA2FDC166 + , ehR = 0x47C4AC1B344028CC740BA7BB9F8AA59D6390E3158153D4F2ADE4B74950 + , ehS = 0x26CE0CDE18A1B884B3EE1A879C13B42F11BB7C85F7A3745C8BECEC8E6E + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x1D8BBF5CB6EFFA270A1CDC22C81E269F0CC16E27151E0A460BA9B51AFF + , ehR = 0x4780B2DE4BAA5613872179AD90664249842E8B96FCD5653B55DD63EED4 + , ehS = 0x6AF46BA322E21D4A88DAEC1650EF38774231276266D6A45ED6A64ECB44 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x67634D0ABA2C9BF7AE54846F26DCD166E7100654BCE6FDC96667631AA2 + , ehR = 0x61D9CC8C842DF19B3D9F4BDA0D0E14A957357ADABC239444610FB39AEA + , ehS = 0x66432278891CB594BA8D08A0C556053D15917E53449E03C2EF88474CF6 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x2CE5AEDC155ACC0DDC5E679EBACFD21308362E5EFC05C5E99B2557A8D7 + , ehR = 0x05E4E6B4DB0E13034E7F1F2E5DBAB766D37C15AE4056C7EE607C8AC7F4 + , ehS = 0x5FC46AA489BF828B34FBAD25EC432190F161BEA8F60D3FCADB0EE3B725 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x1B4BD3903E74FD0B31E23F956C70062014DFEFEE21832032EA5352A055 + , ehR = 0x50F1EFEDFFEC1088024620280EE0D7641542E4D4B5D61DB32358FC571B + , ehS = 0x4614EAE449927A9EB2FCC42EA3E955B43D194087719511A007EC9217A5 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x1775ED919CA491B5B014C5D5E86AF53578B5A7976378F192AF665CB705 + , ehR = 0x6FE6D0D3A953BB66BB01BC6B9EDFAD9F35E88277E5768D1B214395320F + , ehS = 0x7C01A236E4BFF0A771050AD01EC1D24025D3130BBD9E4E81978EB3EC09 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t283k1 + , ecPrivate = + 0x06A0777356E87B89BA1ED3A3D845357BE332173C8F7A65BDC7DB4FAB3C4CC79ACC8194E + , ecPublic = + Point + 0x25330D0A651D5A20DC6389BC02345117725640AEC3C126612CE444EDD19649BDECC03D6 + 0x505BD60A4B67182474EC4D1C668A73140F70504A68F39EFCD972487E9530E0508A76193 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x0A96F788DECAF6C9DBE24DC75ABA6EAAE85E7AB003C8D4F83CB1540625B2993BF445692 + , ehR = 0x1B66D1E33FBDB6E107A69B610995C93C744CEBAEAF623CB42737C27D60188BD1D045A68 + , ehS = 0x02E45B62C9C258643532FD536594B46C63B063946494F95DAFF8759FD552502324295C5 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x1B4C4E3B2F6B08B5991BD2BDDE277A7016DA527AD0AAE5BC61B64C5A0EE63E8B502EF61 + , ehR = 0x018CF2F371BE86BB62E02B27CDE56DDAC83CCFBB3141FC59AEE022B66AC1A60DBBD8B76 + , ehS = 0x1854E02A381295EA7F184CEE71AB7222D6974522D3B99B309B1A8025EB84118A28BF20E + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x1CEB9E8E0DFF53CE687DEB81339ACA3C98E7A657D5A9499EF779F887A934408ECBE5A38 + , ehR = 0x19E90AA3DE5FB20AED22879F92C6FED278D9C9B9293CC5E94922CD952C9DBF20DF1753A + , ehS = 0x135AA7443B6A25D11BB64AC482E04D47902D017752882BD72527114F46CF8BB56C5A8C3 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x1460A5C41745A5763A9D548AE62F2C3630BBED71B6AA549D7F829C22442A728C5D965DA + , ehR = 0x0F8C1CA9C221AD9907A136F787D33BA56B0495A40E86E671C940FD767EDD75EB6001A49 + , ehS = 0x1071A56915DEE89E22E511975AA09D00CDC4AA7F5054CBE83F5977EE6F8E1CC31EC43FD + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x00F3B59FCB5C1A01A1A2A0019E98C244DFF61502D6E6B9C4E957EDDCEB258EF4DBEF04A + , ehR = 0x1D0008CF4BA4A701BEF70771934C2A4A87386155A2354140E2ED52E18553C35B47D9E50 + , ehS = 0x0D15F4FA1B7A4D41D9843578E22EF98773179103DC4FF0DD1F74A6B5642841B91056F78 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x168B5F8C0881D4026C08AC5894A2239D219FA9F4DA0600ADAA56D5A1781AF81F08A726E + , ehR = 0x140932FA7307666A8CCB1E1A09656CC40F5932965841ABD5E8E43559D93CF2311B02767 + , ehS = 0x16A2FD46DA497E5E739DED67F426308C45C2E16528BF2A17EB5D65964FD88B770FBB9C6 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x045E13EA645CE01D9B25EA38C8A8A170E04C83BB7F231EE3152209FE10EC8B2E565536C + , ehR = 0x0E72AF7E39CD72EF21E61964D87C838F977485FA6A7E999000AFA97A381B2445FCEE541 + , ehS = 0x1644FF7D848DA1A040F77515082C27C763B1B4BF332BCF5D08251C6B57D806319778208 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x0B585A7A68F51089691D6EDE2B43FC4451F66C10E65F134B963D4CBD4EB844B0E1469A6 + , ehR = 0x158FAEB2470B306C57764AFC8528174589008449E11DB8B36994B607A65956A59715531 + , ehS = 0x0521BC667CA1CA42B5649E78A3D76823C678B7BB3CD58D2E93CD791D53043A6F83F1FD1 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x1E88738E14482A09EE16A73D490A7FE8739DF500039538D5C4B6C8D6D7F208D6CA56760 + , ehR = 0x1CC4DC5479E0F34C4339631A45AA690580060BF0EB518184C983E0E618C3B93AAB14BBE + , ehS = 0x0284D72FF8AFA83DE364502CBA0494BB06D40AE08F9D9746E747EA87240E589BA0683B7 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x00E5F24A223BD459653F682763C3BB322D4EE75DD89C63D4DC61518D543E76585076BBA + , ehR = 0x1E7912517C6899732E09756B1660F6B96635D638283DF9A8A11D30E008895D7F5C9C7F3 + , ehS = 0x0887E75CBD0B7DD9DE30ED79BDB3D78E4F1121C5EAFF5946918F594F88D363644789DA7 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t409k1 + , ecPrivate = + 0x29C16768F01D1B8A89FDA85E2EFD73A09558B92A178A2931F359E4D70AD853E569CDAF16DAA569758FB4E73089E4525D8BBFCF + , ecPublic = + Point + 0x0CF923F523FE34A6E863D8BA45FB1FE6D784C8F219C414EEF4DB8362DBBD3CA71AEB28F568668D5D7A0093E2B84F6FAD759DB42 + 0x13B1C374D5132978A1B1123EBBE9A5C54D1A9D56B09AFDB4ADE93CCD7C4D332E2916F7D4B9D18578EE3C2E2DE4D2ECE0DE63549 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x7866E5247F9A3556F983C86E81EDA696AC8489DB40A2862F278603982D304F08B2B6E1E7848534BEAF1330D37A1CF84C7994C1 + , ehR = + 0x7192EE99EC7AFE23E02CB1F9850D1ECE620475EDA6B65D04984029408EC1E5A6476BC940D81F218FC31D979814CAC6E78340FA + , ehS = + 0x1DE75DE97CBE740FC79A6B5B22BC2B7832C687E6960F0B8173D5D8BE2A75AC6CA43438BAF69C669CE6D64E0FB93BC5854E0F81 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x512340DB682C7B8EBE407BF1AA54194DFE85D49025FE0F632C9B8A06A996F2FCD0D73C752FB09D23DB8FBE50605DC25DF0745C + , ehR = + 0x41C8EDF39D5E4E76A04D24E6BFD4B2EC35F99CD2483478FD8B0A03E99379576EDACC4167590B7D9C387857A5130B1220CB771F + , ehS = + 0x659652EEAC9747BCAD58034B25362B6AA61836E1BA50E2F37630813050D43457E62EAB0F13AE197E6CFE0244F983107555E269 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x782385F18BAF5A36A588637A76DFAB05739A14163BF723A4417B74BD1469D37AC9E8CCE6AEC8FF63F37B815AAF14A876EED962 + , ehR = + 0x49EC220D6D24980693E6D33B191532EAB4C5D924E97E305E2C1CCFE6F1EAEF96C17F6EC27D1E06191023615368628A7E0BD6A9 + , ehS = + 0x1A4AB1DD9BAAA21F77C503E1B39E770FFD44718349D54BA4CF08F688CE89D7D7C5F7213F225944BE5F7C9BA42B8BEE382F8AF9 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x4DA637CB2E5C90E486744E45A73935DD698D4597E736DA332A06EDA8B26D5ABC6153EC2ECE14981CF3E5E023F36FFA55EEA6D7 + , ehR = + 0x562BB99EE027644EC04E493C5E81B41F261F6BD18FB2FAE3AFEAD91FAB8DD44AFA910B13B9C79C87555225219E44E72245BB7C + , ehS = + 0x25BA5F28047DDDBDA7ED7E49DA31B62B20FD9C7E5B8988817BBF738B3F4DFDD2DCD06EE6DF2A1B744C850DAF952C12B9A56774 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x57055B293ECFDFE983CEF716166091E573275C53906A39EADC25C89C5EC8D7A7E5629FCFDFAD514E1348161C9A34EA1C42D58C + , ehR = + 0x16C7E7FB33B5577F7CF6F77762F0F2D531C6E7A3528BD2CF582498C1A48F200789E9DF7B754029DA0D7E3CE96A2DC760932606 + , ehS = + 0x2729617EFBF80DA5D2F201AC7910D3404A992C39921C2F65F8CF4601392DFE933E6457EAFDBD13DFE160D243100378B55C290A + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x545453D8DC05D220F9A12EF322D0B855E664C72835FABE8A41211453EB8A7CFF950D80773839D0043A46852DDA5A536E02291F + , ehR = + 0x565648A5BAD24E747A7D7531FA9DBDFCB184ECFEFDB00A319459242B68D0989E52BED4107AED35C27D8ECA10E876ACA48006C9 + , ehS = + 0x7420BA6FF72ECC5C92B7CA0309258B5879F26393DB22753B9EC5DF905500A04228AC08880C485E2AC8834E13E8FA44FA57BF18 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x3C5352929D4EBE3CCE87A2DCE380F0D2B33C901E61ABC530DAF3506544AB0930AB9BFD553E51FCDA44F06CD2F49E17E07DB519 + , ehR = + 0x251DFE54EAEC8A781ADF8A623F7F36B4ABFC7EE0AE78C8406E93B5C3932A8120AB8DFC49D8E243C7C30CB5B1E021BADBDF9CA4 + , ehS = + 0x77854C2E72EAA6924CC0B5F6751379D132569843B1C7885978DBBAA6678967F643A50DBB06E6EA6102FFAB7766A57C3887BD22 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x251E32DEE10ED5EA4AD7370DF3EFF091E467D5531CA59DE3AA791763715E1169AB5E18C2A11CD473B0044FB45308E8542F2EB0 + , ehR = + 0x58075FF7E8D36844EED0FC3F78B7CFFDEEF6ADE5982D5636552A081923E24841C9E37DF2C8C4BF2F2F7A174927F3B7E6A0BEB2 + , ehS = + 0x0A737469D013A31B91E781CE201100FDE1FA488ABF2252C025C678462D715AD3078C9D049E06555CABDF37878CFB909553FF51 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x11C540EA46C5038FE28BB66E2E9E9A04C9FE9567ADF33D56745953D44C1DC8B5B92922F53A174E431C0ED8267D919329F19014 + , ehR = + 0x1C5C88642EA216682244E46E24B7CE9AAEF9B3F97E585577D158C3CBC3C598250A53F6D46DFB1E2DD9DC302E7DA4F0CAAFF291 + , ehS = + 0x1D3FD721C35872C74514359F88AD983E170E5DE5B31AFC0BE12E9F4AB2B2538C7797686BA955C1D042FD1F8CDC482775579F11 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x59527CE953BC09DF5E85155CAE7BB1D7F342265F41635545B06044F844ECB4FA6476E7D47420ADC8041E75460EC0A4EC760E95 + , ehR = + 0x1A32CD7764149DF79349DBF79451F4585BB490BD63A200700D7111B45DDA414000AE1B0A69AEACBA1364DD7719968AAD123F93 + , ehS = + 0x582AB1076CAFAE23A76244B82341AEFC4C6D8D8060A62A352C33187720C8A37F3DAC227E62758B11DF1562FD249941C1679F82 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t571k1 + , ecPrivate = + 0x0C16F58550D824ED7B95569D4445375D3A490BC7E0194C41A39DEB732C29396CDF1D66DE02DD1460A816606F3BEC0F32202C7BD18A32D87506466AA92032F1314ED7B19762B0D22 + , ecPublic = + Point + 0x6CFB0DF7541CDD4C41EF319EA88E849EFC8605D97779148082EC991C463ED32319596F9FDF4779C17CAF20EFD9BEB57E9F4ED55BFC52A2FA15CA23BC62B7BF019DB59793DD77318 + 0x1CFC91102F7759A561BD8D5B51AAAEEC7F40E659D67870361990D6DE29F6B4F7E18AE13BDE5EA5C1F77B23D676F44050C9DBFCCDD7B3756328DDA059779AAE8446FC5158A75C227 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x17F7E360B21BEAE4A757A19ACA77FB404D273F05719A86EAD9D7B3F4D5ED7B4630584BB153CF7DCD5A87CCA101BD7EA9ECA0CE5EE27CA985833560000BB52B6BBE068740A45B267 + , ehR = + 0x0767913F96C82E38B7146A505938B79EC07E9AA3214377651BE968B52C039D3E4837B4A2DE26C481C4E1DE96F4D9DE63845D9B32E26D0D332725678E3CE57F668A5E3108FB6CEA5 + , ehS = + 0x109F89F55FA39FF465E40EBCF869A9B1DB425AEA53AB4ECBCE3C310572F79315F5D4891461372A0C36E63871BEDDBB3BA2042C6410B67311F1A185589FF4C987DBA02F9D992B9DF + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x0B599D068A1A00498EE0B9AD6F388521F594BD3F234E47F7A1DB6490D7B57D60B0101B36F39CC22885F78641C69411279706F0989E6991E5D5B53619E43EFB397E25E0814EF02BC + , ehR = + 0x010774B9F14DE6C9525131AD61531FA30987170D43782E9FB84FF0D70F093946DF75ECB69D400FE39B12D58C67C19DCE96335CEC1D9AADE004FE5B498AB8A940D46C8444348686A + , ehS = + 0x06DFE9AA5FEA6CF2CEDC06EE1F9FD9853D411F0B958F1C9C519C90A85F6D24C1C3435B3CDF4E207B4A67467C87B7543F6C0948DD382D24D1E48B3763EC27D4D32A0151C240CC5E0 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x0F79D53E63D89FB87F4D9E6DC5949F5D9388BCFE9EBCB4C2F7CE497814CF40E845705F8F18DBF0F860DE0B1CC4A433EF74A5741F3202E958C082E0B76E16ECD5866AA0F5F3DF300 + , ehR = + 0x1604BE98D1A27CEC2D3FA4BD07B42799E07743071E4905D7DCE7F6992B21A27F14F55D0FE5A7810DF65CF07F2F2554658817E5A88D952282EA1B8310514C0B40FFF46F159965168 + , ehS = + 0x18249377C654B8588475510F7B797081F68C2F8CCCE49F730353B2DA3364B1CD3E984813E11BB791824038EA367BA74583AB97A69AF2D77FA691AA694E348E15DA76F5A44EC1F40 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x0308253C022D25F8A9EBCD24459DD6596590BDEC7895618EEE8A2623A98D2A2B2E7594EE6B7AD3A39D70D68CB4ED01CB28E2129F8E2CC0CC8DC7780657E28BCD655F0BE9B7D35A2 + , ehR = + 0x1E6D7FB237040EA1904CCBF0984B81B866DE10D8AA93B06364C4A46F6C9573FA288C8BDDCC0C6B984E6AA75B42E7BF82FF34D51DFFBD7C87FDBFAD971656185BD12E4B8372F4BF1 + , ehS = + 0x04F94550072ADA7E8C82B7E83577DD39959577799CDABCEA60E267F36F1BEB981ABF24E722A7F031582D2CC5D80DAA7C0DEEBBE1AC5E729A6DBB34A5D645B698719FCA409FBA370 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x0C5EE7070AF55F84EBC43A0D481458CEDE1DCEBB57720A3C92F59B4941A044FECFF4F703940F3121773595E880333772ACF822F2449E17C64DA286BCD65711DD5DA44D7155BF004 + , ehR = + 0x086C9E048EADD7D3D2908501086F3AF449A01AF6BEB2026DC381B39530BCDDBE8E854251CBD5C31E6976553813C11213E4761CB8CA2E5352240AD9FB9C635D55FAB13AE42E4EE4F + , ehS = + 0x09FEE0A68F322B380217FCF6ABFF15D78C432BD8DD82E18B6BA877C01C860E24410F5150A44F979920147826219766ECB4E2E11A151B6A15BB8E2E825AC95BCCA228D8A1C9D3568 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x1D056563469E933E4BE064585D84602D430983BFBFD6885A94BA484DF9A7AB031AD6AC090A433D8EEDC0A7643EA2A9BC3B6299E8ABA933B4C1F2652BB49DAEE833155C8F1319908 + , ehR = + 0x1D055F499A3F7E3FC73D6E7D517B470879BDCB14ABC938369F23643C7B96D0242C1FF326FDAF1CCC8593612ACE982209658E73C24C9EC493B785608669DA74A5B7C9A1D8EA843BC + , ehS = + 0x1621376C53CFE3390A0520D2C657B1FF0EBB10E4B9C2510EDC39D04FEBAF12B8502B098A8B8F842EA6E8EB9D55CFEF94B7FF6D145AC3FFCE71BD978FEA3EF8194D4AB5293A8F3EA + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x1DA875065B9D94DBE75C61848D69578BCC267935792624F9887B53C9AF9E43CABFC42E4C3F9A456BA89E717D24F1412F33CFD297A7A4D403B18B5438654C74D592D5022125E0C6B + , ehR = + 0x18709BDE4E9B73D046CE0D48842C97063DA54DCCA28DCB087168FA37DA2BF5FDBE4720EE48D49EDE4DD5BD31AC0149DB8297BD410F9BC02A11EB79B60C8EE63AF51B65267D71881 + , ehS = + 0x12D8B9E98FBF1D264D78669E236319D8FFD8426C56AFB10C76471EE88D7F0AB1B158E685B6D93C850D47FB1D02E4B24527473DB60B8D1AEF26CEEBD3467B65A70FFDDC0DBB64D5F + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x04DDD0707E81BB56EA2D1D45D7FAFDBDD56912CAE224086802FEA1018DB306C4FB8D93338DBF6841CE6C6AB1506E9A848D2C0463E0889268843DEE4ACB552CFFCB858784ED116B2 + , ehR = + 0x1F5BF6B044048E0E310309FFDAC825290A69634A0D3592DBEE7BE71F69E45412F766AC92E174CC99AABAA5C9C89FCB187DFDBCC7A26765DB6D9F1EEC8A6127BBDFA5801E44E3BEC + , ehS = + 0x1B44CBFB233BFA2A98D5E8B2F0B2C27F9494BEAA77FEB59CDE3E7AE9CB2E385BE8DA7B80D7944AA71E0654E5067E9A70E88E68833054EED49F28283F02B229123995AF37A6089F0 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x0141B53DC6E569D8C0C0718A58A5714204502FDA146E7E2133E56D19E905B79413457437095DE13CF68B5CF5C54A1F2E198A55D974FC3E507AFC0ACF95ED391C93CC79E3B3FE37C + , ehR = + 0x11F61A6EFAB6D83053D9C52665B3542FF3F63BD5913E527BDBA07FBAF34BC766C2EC83163C5273243AA834C75FDDD1BC8A2BEAD388CD06C4EBA1962D645EEB35E92D44E8F2E081D + , ehS = + 0x16BF6341876F051DF224770CC8BA0E4D48B3332568A2B014BC80827BAA89DE18D1AEBC73E3BE8F85A8008C682AAC7D5F0E9FB5ECBEFBB637E30E4A0F226D2C2AA3E569BB54AB72B + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x14842F97F263587A164B215DD0F912C588A88DC4AB6AF4C530ADC1226F16E086D62C14435E6BFAB56F019886C88922D2321914EE41A8F746AAA2B964822E4AC6F40EE2492B66824 + , ehR = + 0x0F1E50353A39EA64CDF23081D6BB4B2A91DD73E99D3DD5A1AA1C49B4F6E34A665EAD24FD530B9103D522609A395AF3EF174C85206F67EF84835ED1632E0F6BAB718EA90DF9E2DA0 + , ehS = + 0x0B385004D7596625028E3FDE72282DE4EDC5B4CE33C1127F21CC37527C90B7307AE7D09281B840AEBCECAA711B00718103DDB32B3E9F6A9FBC6AF23E224A73B9435F619D9C62527 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t163r2 + , ecPrivate = 0x35318FC447D48D7E6BC93B48617DDDEDF26AA658F + , ecPublic = + Point + 0x126CF562D95A1D77D387BA75A3EA3A1407F23425A + 0x7D7CB5273C94DA8CA93049AFDA18721C24672BD71 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x0707A94C3D352E0A9FE49FB12F264992152A20004 + , ehR = 0x153FEBD179A69B6122DEBF5BC61EB947B24C93526 + , ehS = 0x37AC9C670F8CF18045049BAE7DD35553545C19E49 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x3B24C5E2C2D935314EABF57A6484289B291ADFE3F + , ehR = 0x0A379E69C44F9C16EA3215EA39EB1A9B5D58CC955 + , ehS = 0x04BAFF5308DA2A7FE2C1742769265AD3ED1D24E74 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x3D7086A59E6981064A9CDB684653F3A81B6EC0F0B + , ehR = 0x134E00F78FC1CB9501675D91C401DE20DDF228CDC + , ehS = 0x373273AEC6C36CB7BAFBB1903A5F5EA6A1D50B624 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x3B1E4443443486C7251A68EF184A936F05F8B17C7 + , ehR = 0x29430B935AF8E77519B0CA4F6903B0B82E6A21A66 + , ehS = 0x1EA1415306E9353FA5AA54BC7C2581DFBB888440D + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x2EDF5CFCAC7553C17421FDF54AD1D2EF928A879D2 + , ehR = 0x0B2F177A99F9DF2D51CCAF55F015F326E4B65E7A0 + , ehS = 0x0DF1FB4487E9B120C5E970EFE48F55E406306C3A1 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x10024F5B324CBC8954BA6ADB320CD3AB9296983B4 + , ehR = 0x256D4079C6C7169B8BC92529D701776A269D56308 + , ehS = 0x341D3FFEC9F1EB6A6ACBE88E3C86A1C8FDEB8B8E1 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x34F46DE59606D56C75406BFB459537A7CC280AA62 + , ehR = 0x28ECC6F1272CE80EA59DCF32F7AC2D861BA803393 + , ehS = 0x0AD4AE2C06E60183C1567D2B82F19421FE3053CE2 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x38145E3FFCA94E4DDACC20AD6E0997BD0E3B669D2 + , ehR = 0x227DF377B3FA50F90C1CB3CDCBBDBA552C1D35104 + , ehS = 0x1F7BEAD92583FE920D353F368C1960D0E88B46A56 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x375813210ECE9C4D7AB42DDC3C55F89189CF6DFFD + , ehR = 0x11811DAFEEA441845B6118A0DFEE8A0061231337D + , ehS = 0x36258301865EE48C5C6F91D63F62695002AB55B57 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x25AD8B393BC1E9363600FDA1A2AB6DF40079179A3 + , ehR = 0x3B6BB95CA823BE2ED8E3972FF516EB8972D765571 + , ehS = 0x13DC6F420628969DF900C3FCC48220B38BE24A541 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t233r1 + , ecPrivate = 0x07ADC13DD5BF34D1DDEEB50B2CE23B5F5E6D18067306D60C5F6FF11E5D3 + , ecPublic = + Point + 0x0FB348B3246B473AA7FBB2A01B78D61B62C4221D0F9AB55FC72DB3DF478 + 0x1162FA1F6C6ACF7FD8D19FC7D74BDD9104076E833898BC4C042A6E6BEBF + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x0A4E0B67A3A081C1B35D7BECEB5FE72A918B422B907145DB5416ED751CE + , ehR = 0x015CC6FD78BB06E0878E71465515EA5A21A2C18E6FC77B4B158DBEB3944 + , ehS = 0x0822A4A6C2EB2DF213A5E90BF40377956365EE8C4B4A5A4E2EB9270CB6A + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x0F2B1C1E80BEB58283AAA79857F7B83BDF724120D0913606FD07F7FFB2C + , ehR = 0x05D9920B53471148E10502AB49AB7A3F11084820A074FD89883CF51BC1A + , ehS = 0x04D3938900C0A9AAA7080D1DFEB56CFB0FADABE4214536C7ED5117ED13A + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x034A53897B0BBDB484302E19BF3F9B34A2ABFED639D109A388DC52006B5 + , ehR = 0x0A797F3B8AEFCE7456202DF1E46CCC291EA5A49DA3D4BDDA9A4B62D5E0D + , ehS = 0x01F6F81DA55C22DA4152134C661588F4BD6F82FDBAF0C5877096B070DC2 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x04D4670B28990BC92EEB49840B482A1FA03FE028D09F3D21F89C67ECA85 + , ehR = 0x015E85A8D46225DD7E314A1C4289731FC14DECE949349FE535D11043B85 + , ehS = 0x03F189D37F50493EFD5111A129443A662AB3C6B289129AD8C0CAC85119C + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x0DE108AAADA760A14F42C057EF81C0A31AF6B82E8FBCA8DC86E443AB549 + , ehR = 0x03B62A4BF783919098B1E42F496E65F7621F01D1D466C46940F0F132A95 + , ehS = 0x0F4BE031C6E5239E7DAA014CBBF1ED19425E49DAEB426EC9DF4C28A2E30 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x0250C5C90A4E2A3F8849FEBA87F0D0AE630AB18CBABB84F4FFFB36CEAC0 + , ehR = 0x02F1FEDC57BE203E4C8C6B8C1CEB35E13C1FCD956AB41E3BD4C8A6EFB1F + , ehS = 0x05738EC8A8EDEA8E435EE7266AD3EDE1EEFC2CEBE2BE1D614008D5D2951 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x07BDB6A7FD080D9EC2FC84BFF9E3E15750789DC04290C84FED00E109BBD + , ehR = 0x0CCE175124D3586BA7486F7146894C65C2A4A5A1904658E5C7F9DF5FA5D + , ehS = 0x08804B456D847ACE5CA86D97BF79FD6335E5B17F6C0D964B5D0036C867E + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x00376886E89013F7FF4B5214D56A30D49C99F53F211A3AFE01AA2BDE12D + , ehR = 0x035C3D6DFEEA1CFB29B93BE3FDB91A7B130951770C2690C16833A159677 + , ehS = 0x0600F7301D12AB376B56D4459774159ADB51F97E282FF384406AFD53A02 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x03726870DE75613C5E529E453F4D92631C03D08A7F63813E497D4CB3877 + , ehR = 0x061602FC8068BFD5FB86027B97455D200EC603057446CCE4D76DB8EF42C + , ehS = 0x03396DD0D59C067BB999B422D9883736CF9311DFD6951F91033BD03CA8D + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x09CE5810F1AC68810B0DFFBB6BEEF2E0053BB937969AE7886F9D064A8C4 + , ehR = 0x07E12CB60FDD614958E8E34B3C12DDFF35D85A9C5800E31EA2CC2EF63B1 + , ehS = 0x0E8970FD99D836F3CC1C807A2C58760DE6EDAA23705A82B9CB1CE93FECC + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t283r1 + , ecPrivate = + 0x14510D4BC44F2D26F4553942C98073C1BD35545CEABB5CC138853C5158D2729EA408836 + , ecPublic = + Point + 0x17E3409A13C399F0CA8A192F028D46E3446BCFFCDF51FF8A905ED2DED786E74F9C3E8A9 + 0x47EFCBCC31C01D86D1992F7BFAC0277DBD02A6D289274099A2C0F039C8F59F318371B0E + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x277F389559667E8AE4B65DC056F8CE2872E1917E7CC59D17D485B0B98343206FBCCD441 + , ehR = 0x201E18D48C6DB3D5D097C4DCE1E25587E1501FC3CF47BDB5B4289D79E273D6A9ACB8285 + , ehS = 0x151AE05712B024CE617358260774C8CA8B0E7A7E72EF8229BF2ACE7609560CB30322C4F + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x14CC8FCFEECD6B999B4DC6084EBB06FDED0B44D5C507802CC7A5E9ECF36E69DA6AE23C6 + , ehR = 0x143E878DDFD4DF40D97B8CD638B3C4706501C2201CF7108F2FB91478C11D69473246925 + , ehS = 0x0CBF1B9717FEEA3AABB09D9654110144267098E0E1E8D0289A6211BE0EEDFDD86A3DB79 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x38C9D662188982943E080B794A4CFB0732DBA37C6F40D5B8CFADED6FF31C5452BA3F877 + , ehR = 0x29FD82497FB3E5CEF65579272138DE59E2B666B8689466572B3B69A172CEE83BE145659 + , ehS = 0x05A89D9166B40795AF0FE5958201B9C0523E500013CA12B4840EA2BC53F25F9B3CE87C0 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x21B7265DEBF90E6F988CFFDB62B121A02105226C652807CC324ED6FB119A287A72680AB + , ehR = 0x2F00689C1BFCD2A8C7A41E0DE55AE182E6463A152828EF89FE3525139B6603294E69353 + , ehS = 0x1744514FE0A37447250C8A329EAAADA81572226CABA16F39270EE5DD03F27B1F665EB5D + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x20583259DC179D9DA8E5387E89BFF2A3090788CF1496BCABFE7D45BB120B0C811EB8980 + , ehR = 0x0DA43A9ADFAA6AD767998A054C6A8F1CF77A562924628D73C62761847AD8286E0D91B47 + , ehS = 0x1D118733AE2C88357827CAFC6F68ABC25C80C640532925E95CFE66D40F8792F3AC44C42 + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = 0x0185C57A743D5BA06193CE2AA47B07EF3D6067E5AE1A6469BCD3FC510128BA564409D82 + , ehR = 0x05A408133919F2CDCDBE5E4C14FBC706C1F71BADAFEF41F5DE4EC27272FC1CA9366FBB2 + , ehS = 0x012966272872C097FEA7BCE64FAB1A81982A773E26F6E4EF7C99969846E67CA9CBE1692 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = 0x2E5C1F00677A0E015EC3F799FA9E9A004309DBD784640EAAF5E1CE64D3045B9FE9C1FA1 + , ehR = 0x08F3824E40C16FF1DDA8DC992776D26F4A5981AB5092956C4FDBB4F1AE0A711EEAA10E5 + , ehS = 0x0A64B91EFADB213E11483FB61C73E3EF63D3B44EEFC56EA401B99DCC60CC28E99F0F1FA + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = 0x018A7D44F2B4341FEFE68F6BD8894960F97E08124AAB92C1FFBBE90450FCC9356C9AAA5 + , ehR = 0x3597B406F5329D11A79E887847E5EC60861CCBB19EC61F252DB7BD549C699951C182796 + , ehS = 0x0A6A100B997BC622D91701D9F5C6F6D3815517E577622DA69D3A0E8917C1CBE63ACD345 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = 0x3C75397BA4CF1B931877076AF29F2E2F4231B117AB4B8E039F7F9704DE1BD3522F150B6 + , ehR = 0x1BB490926E5A1FDC7C5AA86D0835F9B994EDA315CA408002AF54A298728D422EBF59E4C + , ehS = 0x36C682CFC9E2C89A782BFD3A191609D1F0C1910D5FD6981442070393159D65FBCC0A8BA + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = 0x14E66B18441FA54C21E3492D0611D2B48E19DE3108D915FD5CA08E786327A2675F11074 + , ehR = 0x19944AA68F9778C2E3D6E240947613E6DA60EFCE9B9B2C063FF5466D72745B5A0B25BA2 + , ehS = 0x03F1567B3C5B02DF15C874F0EE22850824693D5ADC4663BAA19E384E550B1DD41F31EE6 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t409r1 + , ecPrivate = + 0x0494994CC325B08E7B4CE038BD9436F90B5E59A2C13C3140CD3AE07C04A01FC489F572CE0569A6DB7B8060393DE76330C624177 + , ecPublic = + Point + 0x1A7055961CF1DA4B9A015B18B1524EF01FDD9B93FAEFC26FB1F2F828A7227B7031925DA0AC1A8A075C3B33554B222EA859C17E7 + 0x18105C042F290736088F30AEC7AE7732A45DE47BCE0940113AB8132516D1E059B0F581FD581A9A3CB3A0AC42A1962738ADB86E6 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x042D8A2B34402757EB2CCFDDC3E6E96A7ADD3FDA547FC10A0CB77CFC720B4F9E16EEAAA2A8CC4E4A4B5DBF7D8AC4EA491859E60 + , ehR = + 0x0D8783188E1A540E2022D389E1D35B32F56F8C2BB5636B8ABF7718806B27A713EBAE37F63ECD4B61445CEF5801B62594EF3E982 + , ehS = + 0x03A6B4A80E204DB0DE12E7415C13C9EC091C52935658316B4A0C591216A3879154BEB1712560E346E7EF26517707435B55C3141 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x0C933F1DC4C70838C2AD16564715ACAF545BCDD8DC203D25AF3EC63949C65CB2E68AC1F60CA7EACA2A823F4E240927AA82CEEC5 + , ehR = + 0x0EE4F39ACC2E03CE96C3D9FCBAFA5C22C89053662F8D4117752A9B10F09ADFDA59DB061E247FE5321D6B170EE758ACE1BE4D157 + , ehS = + 0x00A2B83265B456A430A8BF27DCC8A9488B3F126C10F0D6D64BF7B8A218FAAF20E51A295A3AE78F205E5A4A6AE224C3639F1BB34 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x08EC42D13A3909A20C41BEBD2DFED8CACCE56C7A7D1251DF43F3E9E289DAE00E239F6960924AC451E125B784CB687C7F23283FD + , ehR = + 0x02D8B1B31E33E74D7EB46C30FDE5AD2CA04EC8FE08FBA0E73BA5E568953AC5EA307C072942238DFC07F4A4D7C7C6A9F86436D17 + , ehS = + 0x079F7D471E6CB73234AF7F7C381D2CE15DE35BAF8BB68393B73235B3A26EC2DF4842CE433FB492D6E074E604D4870024D42189A + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x0DA881BCE3BA851485879EF8AC585A63F1540B9198ECB8A1096D70CB25A104E2F8A96B108AE76CB49CF34491ABC70E9D2AAD450 + , ehR = + 0x07BC638B7E7CE6FEE5E9C64A0F966D722D01BB4BC3F3A35F30D4CDDA92DFC5F7F0B4BBFE8065D9AD452FD77A1914BE3A2440C18 + , ehS = + 0x06D904429850521B28A32CBF55C7C0FDF35DC4E0BDA2552C7BF68A171E970E6788ACC0B9521EACB4796E057C70DD9B95FED5BFB + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x0750926FFAD7FF5DE85DF7960B3A4F9E3D38CF5A049BFC89739C48D42B34FBEE03D2C047025134CC3145B60AFD22A68DF0A7FB2 + , ehR = + 0x05D178DECAFD2D02A3DA0D8BA1C4C1D95EE083C760DF782193A9F7B4A8BE6FC5C21FD60613BCA65C063A61226E050A680B3ABD4 + , ehS = + 0x013B7581E98F6A63FBBCB3E49BCDA60F816DB230B888506D105DC229600497C3B46588C784BE3AA9343BEF82F7C9C80AEB63C3B + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x017E167EAB1850A3B38EE66BFE2270F2F6BFDAC5E2D227D47B20E75F0719161E6C74E9F23088F0C58B1E63BC6F185AD2EF4EAE6 + , ehR = + 0x049F54E7C10D2732B4638473053782C6919218BBEFCEC8B51640FC193E832291F05FA12371E9B448417B3290193F08EE9319195 + , ehS = + 0x0499E267DEC84E02F6F108B10E82172C414F15B1B7364BE8BFD66ADC0C5DE23FEE3DF0D811134C25AFE0E05A6672F98889F28F1 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x01ADEB94C19951B460A146B8275D81638C07735B38A525D76023AAF26AA8A058590E1D5B1E78AB3C91608BDA67CFFBE6FC8A6CC + , ehR = + 0x0B1527FFAA7DD7C7E46B628587A5BEC0539A2D04D3CF27C54841C2544E1BBDB42FDBDAAF8671A4CA86DFD619B1E3732D7BB56F2 + , ehS = + 0x0442C68C044868DF4832C807F1EDDEBF7F5052A64B826FD03451440794063F52B022DF304F47403D4069234CA9EB4C964B37C02 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x06EBA3D58D0E0DFC406D67FC72EF0C943624CF40019D1E48C3B54CCAB0594AFD5DEE30AEBAA22E693DBCFECAD1A85D774313DAD + , ehR = + 0x0BB27755B991D6D31757BCBF68CB01225A38E1CFA20F775E861055DD108ED7EA455E4B96B2F6F7CD6C6EC2B3C70C3EDDEB9743B + , ehS = + 0x0C5BE90980E7F444B5F7A12C9E9AC7A04CA81412822DD5AD1BE7C45D5032555EA070864245CF69266871FEB8CD1B7EDC30EF6D5 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x0A45B787DB44C06DEAB846511EEDBF7BFCFD3BD2C11D965C92FC195F67328F36A2DC83C0352885DAB96B55B02FCF49DCCB0E2DA + , ehR = + 0x04EFEB7098772187907C87B33E0FBBA4584226C50C11E98CA7AAC6986F8D3BE044E5B52D201A410B852536527724CA5F8CE6549 + , ehS = + 0x09574102FEB3EF87E6D66B94119F5A6062950FF4F902EA1E6BD9E2037F33FF991E31F5956C23AFE48FCDC557FD6F088C7C9B2B3 + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x0B90F8A0E757E81D4EA6891766729C96A6D01F9AEDC0D334932D1F81CC4E1973A4F01C33555FF08530A5098CADB6EDAE268ABB5 + , ehR = + 0x07E0249C68536AE2AEC2EC30090340DA49E6DC9E9EEC8F85E5AABFB234B6DA7D2E9524028CF821F21C6019770474CC40B01FAF6 + , ehS = + 0x08125B5A03FB44AE81EA46D446130C2A415ECCA265910CA69D55F2453E16CD7B2DFA4E28C50FA8137F9C0C6CEE4CD37ABCCF6D8 + } + ] + } + ] + } + , EntryCurve + { ecName = SEC_t571r1 + , ecPrivate = + 0x028A04857F24C1C082DF0D909C0E72F453F2E2340CCB071F0E389BCA2575DA19124198C57174929AD26E348CF63F78D28021EF5A9BF2D5CBEAF6B7CCB6C4DA824DD5C82CFB24E11 + , ecPublic = + Point + 0x4B4B3CE9377550140B62C1061763AA524814DDCEF37B00CD5CDE94F7792BB0E96758E55DA2E9FEA8FF2A8B6830AE1D57A9CA7A77FCB0836BF43EA5454CDD9FEAD5CCFE7375C6A83 + 0x4453B18F261E7A0E7570CD72F235EA750438E43946FBEBD2518B696954767AA7849C1719E18E1C51652C28CA853426F15C09AA4B579487338ABC7F33768FADD61B5A3A6443A8189 + , ecMessages = + [ EntryMessage + { emMessage = "sample" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x2669FAFEF848AF67D437D4A151C3C5D3F9AA8BB66EDC35F090C9118F95BA0041B0993BE2EF55DAAF36B5B3A737C40DB1F6E3D93D97B8419AD6E1BB8A5D4A0E9B2E76832D4E7B862 + , ehR = + 0x147D3EB0EDA9F2152DFD014363D6A9CE816D7A1467D326A625FC4AB0C786E1B74DDF7CD4D0E99541391B266C704BB6B6E8DCCD27B460802E0867143727AA415555454321EFE5CB6 + , ehS = + 0x17319571CAF533D90D2E78A64060B9C53169AB7FC908947B3EDADC54C79CCF0A7920B4C64A4EAB6282AFE9A459677CDA37FD6DD50BEF18709590FE18B923BDF74A66B189A850819 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x2EAFAD4AC8644DEB29095BBAA88D19F31316434F1766AD4423E0B54DD2FE0C05E307758581B0DAED2902683BBC7C47B00E63E3E429BA54EA6BA3AEC33A94C9A24A6EF8E27B7677A + , ehR = + 0x10F4B63E79B2E54E4F4F6A2DBC786D8F4A143ECA7B2AD97810F6472AC6AE20853222854553BE1D44A7974599DB7061AE8560DF57F2675BE5F9DD94ABAF3D47F1582B318E459748B + , ehS = + 0x3BBEA07C6B269C2B7FE9AE4DDB118338D0C2F0022920A7F9DCFCB7489594C03B536A9900C4EA6A10410007222D3DAE1A96F291C4C9275D75D98EB290DC0EEF176037B2C7A7A39A3 + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x15C2C6B7D1A070274484774E558B69FDFA193BDB7A23F27C2CD24298CE1B22A6CC9B7FB8CABFD6CF7C6B1CF3251E5A1CDDD16FBFED28DE79935BB2C631B8B8EA9CC4BCC937E669E + , ehR = + 0x213EF9F3B0CFC4BF996B8AF3A7E1F6CACD2B87C8C63820000800AC787F17EC99C04BCEDF29A8413CFF83142BB88A50EF8D9A086AF4EB03E97C567500C21D865714D832E03C6D054 + , ehS = + 0x3D32322559B094E20D8935E250B6EC139AC4AAB77920812C119AF419FB62B332C8D226C6C9362AE3C1E4AABE19359B8428EA74EC8FBE83C8618C2BCCB6B43FBAA0F2CCB7D303945 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x0FEF0B68CB49453A4C6ECBF1708DBEEFC885C57FDAFB88417AAEFA5B1C35017B4B498507937ADCE2F1D9EFFA5FE8F5AEB116B804FD182A6CF1518FDB62D53F60A0FF6EB707D856B + , ehR = + 0x375D8F49C656A0BBD21D3F54CDA287D853C4BB1849983CD891EF6CD6BB56A62B687807C16685C2C9BCA2663C33696ACCE344C45F3910B1DF806204FF731ECB289C100EF4D1805EC + , ehS = + 0x1CDEC6F46DFEEE44BCE71D41C60550DC67CF98D6C91363625AC2553E4368D2DFB734A8E8C72E118A76ACDB0E58697940A0F3DF49E72894BD799450FC9E550CC04B9FF9B0380021C + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x3FF373833A06C791D7AD586AFA3990F6EF76999C35246C4AD0D519BFF180CA1880E11F2FB38B764854A0AE3BECDDB50F05AC4FCEE542F207C0A6229E2E19652F0E647B9C4882193 + , ehR = + 0x1C26F40D940A7EAA0EB1E62991028057D91FEDA0366B606F6C434C361F04E545A6A51A435E26416F6838FFA260C617E798E946B57215284182BE55F29A355E6024FE32A47289CF0 + , ehS = + 0x3691DE4369D921FE94EDDA67CB71FBBEC9A436787478063EB1CC778B3DCDC1C4162662752D28DEEDF6F32A269C82D1DB80C87CE4D3B662E03AC347806E3F19D18D6D4DE7358DF7E + } + ] + } + , EntryMessage + { emMessage = "test" + , emHashes = + [ EntryHash + { ehAlgorithm = HashAlg SHA1 + , ehK = + 0x019B506FD472675A7140E429AA5510DCDDC21004206EEC1B39B28A688A8FD324138F12503A4EFB64F934840DFBA2B4797CFC18B8BD0B31BBFF3CA66A4339E4EF9D771B15279D1DC + , ehR = + 0x133F5414F2A9BC41466D339B79376038A64D045E5B0F792A98E5A7AA87E0AD016419E5F8D176007D5C9C10B5FD9E2E0AB8331B195797C0358BA05ECBF24ACE59C5F368A6C0997CC + , ehS = + 0x3D16743AE9F00F0B1A500F738719C5582550FEB64689DA241665C4CE4F328BA0E34A7EF527ED13BFA5889FD2D1D214C11EB17D6BC338E05A56F41CAFF1AF7B8D574DB62EF0D0F21 + } + , EntryHash + { ehAlgorithm = HashAlg SHA224 + , ehK = + 0x333C711F8C62F205F926593220233B06228285261D34026232F6F729620C6DE12220F282F4206D223226705608688B20B8BA86D8DFE54F07A37EC48F253283AC33C3F5102C8CC3E + , ehR = + 0x3048E76506C5C43D92B2E33F62B33E3111CEEB87F6C7DF7C7C01E3CDA28FA5E8BE04B5B23AA03C0C70FEF8F723CBCEBFF0B7A52A3F5C8B84B741B4F6157E69A5FB0524B48F31828 + , ehS = + 0x2C99078CCFE5C82102B8D006E3703E020C46C87C75163A2CD839C885550BA5CB501AC282D29A1C26D26773B60FBE05AAB62BFA0BA32127563D42F7669C97784C8897C22CFB4B8FA + } + , EntryHash + { ehAlgorithm = HashAlg SHA256 + , ehK = + 0x328E02CF07C7B5B6D3749D8302F1AE5BFAA8F239398459AF4A2C859C7727A8123A7FE9BE8B228413FC8DC0E9DE16AF3F8F43005107F9989A5D97A5C4455DA895E81336710A3FB2C + , ehR = + 0x184BC808506E11A65D628B457FDA60952803C604CC7181B59BD25AEE1411A66D12A777F3A0DC99E1190C58D0037807A95E5080FA1B2E5CCAA37B50D401CFFC3417C005AEE963469 + , ehS = + 0x27280D45F81B19334DBDB07B7E63FE8F39AC7E9AE14DE1D2A6884D2101850289D70EE400F26ACA5E7D73F534A14568478E59D00594981ABE6A1BA18554C13EB5E03921E4DC98333 + } + , EntryHash + { ehAlgorithm = HashAlg SHA384 + , ehK = + 0x2A77E29EAD9E811A9FDA0284C14CDFA1D9F8FA712DA59D530A06CDE54187E250AD1D4FB5788161938B8DE049616399C5A56B0737C9564C9D4D845A4C6A7CDFCBFF0F01A82BE672E + , ehR = + 0x319EE57912E7B0FAA1FBB145B0505849A89C6DB1EC06EA20A6A7EDE072A6268AF6FD9C809C7E422A5F33C6C3326EAD7402467DF3272A1B2726C1C20975950F0F50D8324578F13EC + , ehS = + 0x2CF3EA27EADD0612DD2F96F46E89AB894B01A10DF985C5FC099CFFE0EA083EB44BE682B08BFE405DAD5F37D0A2C59015BA41027E24B99F8F75A70B6B7385BF39BBEA02513EB880C + } + , EntryHash + { ehAlgorithm = HashAlg SHA512 + , ehK = + 0x21CE6EE4A2C72C9F93BDB3B552F4A633B8C20C200F894F008643240184BE57BB282A1645E47FBBE131E899B4C61244EFC2486D88CDBD1DD4A65EBDD837019D02628D0DCD6ED8FB5 + , ehR = + 0x2AA1888EAB05F7B00B6A784C4F7081D2C833D50794D9FEAF6E22B8BE728A2A90BFCABDC803162020AA629718295A1489EE7ED0ECB8AAA197B9BDFC49D18DDD78FC85A48F9715544 + , ehS = + 0x0AA5371FE5CA671D6ED9665849C37F394FED85D51FEF72DA2B5F28EDFB2C6479CA63320C19596F5E1101988E2C619E302DD05112F47E8823040CE540CD3E90DCF41DBC461744EE9 + } + ] + } + ] + } + ] + +testPublic :: PrivateKey -> PublicPoint -> Spec +testPublic (PrivateKey curve key) pub = + it "public" $ + generateQ curve key `shouldBe` pub + +testNonce :: PrivateKey -> HashAlg -> ByteString -> Integer -> Spec +testNonce key (HashAlg alg) msg nonc = + it "nonce" $ + deterministicNonce alg key (hashWith alg msg) Just `shouldBe` nonc + +testSignature + :: PrivateKey -> HashAlg -> ByteString -> Integer -> Signature -> Spec +testSignature key (HashAlg alg) msg nonc sig = it "signature" $ + case signWith nonc key alg msg of + Nothing -> assertFailure "could not sign message" + Just result -> result `shouldBe` sig + +testVerify :: PublicKey -> HashAlg -> ByteString -> Signature -> Spec +testVerify pub (HashAlg alg) msg sig = + it "verify" $ + assertBool "signature verification failed" $ + verify alg pub sig msg + +testEntry :: Entry -> Spec +testEntry entry = describe (show entry) $ sequence_ tests + where + tests = + [ testPublic key $ publicPoint entry + , testSignature + key + (hashAlgorithm entry) + (message entry) + (nonce entry) + (signature entry) + , testVerify pub (hashAlgorithm entry) (message entry) (signature entry) + ] + pub = PublicKey curve $ publicPoint entry + key = PrivateKey curve $ privateNumber entry + curve = getCurveByName $ curveName entry + +testEntryNonce :: Entry -> Spec +testEntryNonce entry = describe (show entry) $ sequence_ tests + where + tests = + [ testPublic key $ publicPoint entry + , testNonce key (hashAlgorithm entry) (message entry) (nonce entry) + , testSignature + key + (hashAlgorithm entry) + (message entry) + (nonce entry) + (signature entry) + , testVerify pub (hashAlgorithm entry) (message entry) (signature entry) + ] + pub = PublicKey curve $ publicPoint entry + key = PrivateKey curve $ privateNumber entry + curve = getCurveByName $ curveName entry + +-- | Signing inverts k modulo the order of the curve. The inverse does not +-- exist for a k that is zero or the order itself, and on a curve whose order +-- is composite it does not exist for a k that shares a factor with it -- +-- signWith takes k and the curve from the caller, so it has to say no rather +-- than raise. The curve below is the textbook y^2 = x^3 + x + 1 over F23, +-- which has 28 points, with (3, 10) generating all of them. +nonInvertibleTests :: Spec +nonInvertibleTests = + describe "non-invertible values" $ do + it "signWith with k = 0 returns Nothing" $ + signWith 0 tinyKey SHA1 msg0 `shouldBe` Nothing + it "signWith with k = the order returns Nothing" $ + signWith 28 tinyKey SHA1 msg0 `shouldBe` Nothing + it "signWith with k sharing a factor with the order returns Nothing" $ + signWith 14 tinyKey SHA1 msg0 `shouldBe` Nothing + it "signWith with a usable k still signs" $ + signWith 5 tinyKey SHA1 msg0 `shouldSatisfy` isJust + where + msg0 = "message" :: ByteString + tinyCurve = + CurveFP $ + CurvePrime 23 $ + CurveCommon + { ecc_a = 1 + , ecc_b = 1 + , ecc_g = Point 3 10 + , ecc_n = 28 + , ecc_h = 1 + } + tinyKey = PrivateKey tinyCurve 5 + +spec :: Spec +spec = do + nonInvertibleTests + describe "GEC 2" $ sequence_ $ testEntry . normalize <$> gec2Entries + describe "RFC 6979" $ + sequence_ $ + testEntryNonce . normalize <$> flatten rfc6979Entries diff --git a/tests/PubKey/ElGamalSpec.hs b/tests/PubKey/ElGamalSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/ElGamalSpec.hs @@ -0,0 +1,130 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.ElGamalSpec (spec) where + +import Crypto.Error +import Crypto.Hash (SHA256 (..)) +import qualified Crypto.PubKey.DH as DH +import qualified Crypto.PubKey.ElGamal as ElGamal +import Crypto.Random (drgNewTest, withDRG) + +import Imports + +-- | The 1024-bit MODP group of RFC 2409 section 6.2, whose generator is 2. +p :: Integer +p = + 0xFFFFFFFFFFFFFFFFC90FDAA22168C234C4C6628B80DC1CD129024E088A67CC74020BBEA63B139B22514A08798E3404DDEF9519B3CD3A431B302B0A6DF25F14374FE1356D6D51C245E485B576625E7EC6F44C42E9A637ED6B0BFF5CB6F406B7EDEE386BFB5A899FA5AE9F24117C4B1FE649286651ECE65381FFFFFFFFFFFFFFFF + +params :: DH.Params +params = DH.Params p 2 1024 + +priv :: DH.PrivateNumber +priv = DH.PrivateNumber 0x1f3b5d79a2c4e60813579bdf2468ace0 + +pub :: DH.PublicNumber +pub = ElGamal.generatePublic params priv + +message :: Integer +message = 0x48656c6c6f2c20456c47616d616c21 + +-- | A usable ephemeral value: within [1, p-2] and not reused elsewhere here. +ephemeral :: ElGamal.EphemeralKey +ephemeral = ElGamal.EphemeralKey 0x2c4e60813579bdf2468ace01f3b5d79a + +encryptionTests :: Spec +encryptionTests = describe "encryption" $ do + it "decrypts what it encrypts" $ + (ElGamal.encryptWith ephemeral params pub message >>= ElGamal.decrypt params priv) + `shouldBe` CryptoPassed message + it "refuses an ephemeral value of zero" $ + -- it would leave c2 equal to the message + ElGamal.encryptWith (ElGamal.EphemeralKey 0) params pub message + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + it "refuses an ephemeral value at or above p-1" $ + ElGamal.encryptWith (ElGamal.EphemeralKey (p - 1)) params pub message + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + it "refuses a peer public number generating a tiny subgroup" $ + mapM_ + ( \h -> + ElGamal.encryptWith ephemeral params (DH.PublicNumber h) message + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + ) + [0, 1, p - 1, p] + it "refuses a message at or above the modulus" $ + -- it would come back reduced + ElGamal.encryptWith ephemeral params pub p + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + it "refuses a negative message" $ + ElGamal.encryptWith ephemeral params pub (-1) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + +decryptionTests :: Spec +decryptionTests = describe "decryption" $ do + it "refuses a first component of zero rather than raising" $ + -- zero has no inverse modulo p + ElGamal.decrypt params priv (0, 1) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + it "refuses a first component at or above the modulus" $ + ElGamal.decrypt params priv (p, 1) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + it "refuses a second component out of range" $ + ElGamal.decrypt params priv (2, p) + `shouldBe` CryptoFailed CryptoError_ParameterInvalid + +signatureTests :: Spec +signatureTests = describe "signature" $ do + it "verifies what it signs" $ + case ElGamal.signWith k params priv SHA256 msg of + Nothing -> expectationFailure "expected a signature" + Just sig -> ElGamal.verify params pub SHA256 msg sig `shouldBe` True + it "refuses a k of zero" $ + ElGamal.signWith 0 params priv SHA256 msg `shouldBe` Nothing + it "refuses a negative k" $ + ElGamal.signWith (-1) params priv SHA256 msg `shouldBe` Nothing + it "refuses a k at or above p-1" $ + mapM_ + (\k' -> ElGamal.signWith k' params priv SHA256 msg `shouldBe` Nothing) + [p - 1, p, p + 1] + it "accepts the largest usable k" $ + -- p-2 and p-1 are consecutive, so they are coprime + case ElGamal.signWith (p - 2) params priv SHA256 msg of + Nothing -> expectationFailure "expected a signature" + Just sig -> ElGamal.verify params pub SHA256 msg sig `shouldBe` True + it "refuses a k sharing a factor with p-1" $ + -- p is an odd prime, so p-1 is even and no even k is coprime with it + mapM_ + (\k' -> ElGamal.signWith k' params priv SHA256 msg `shouldBe` Nothing) + [2, 4, p - 3] + it "rejects a signature over a different message" $ + case ElGamal.signWith k params priv SHA256 msg of + Nothing -> expectationFailure "expected a signature" + Just sig -> + ElGamal.verify params pub SHA256 ("other" :: ByteString) sig + `shouldBe` False + it "rejects a signature with r out of range" $ + ElGamal.verify params pub SHA256 msg (ElGamal.Signature 0 1) `shouldBe` False + -- 'sign' draws a blinder for the inversion of k, so it takes a path + -- 'signWith' does not: the inverse comes back from a different number + -- than the one wanted, times the blinder + it "verifies what it signs when it draws k itself" $ + mapM_ + ( \seed -> + let (sig, _) = + withDRG (drgNewTest seed) (ElGamal.sign params priv SHA256 msg) + in ElGamal.verify params pub SHA256 msg sig `shouldBe` True + ) + [ (1, 2, 3, 4, 5) + , (5, 4, 3, 2, 1) + , (0, 0, 0, 0, 1) + , (9, 8, 7, 6, 5) + , (0x1234, 0x5678, 0x9abc, 0xdef0, 0x2468) + ] + where + msg = "message" :: ByteString + k = 0x5d79a2c4e60813579bdf2468ace01f3b + +spec :: Spec +spec = do + encryptionTests + decryptionTests + signatureTests diff --git a/tests/PubKey/MGF1Spec.hs b/tests/PubKey/MGF1Spec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/MGF1Spec.hs @@ -0,0 +1,36 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.MGF1Spec (spec) where + +import Control.Monad (zipWithM_) +import Test.Hspec + +import Data.ByteString (ByteString) +import qualified Data.ByteString as B +import Data.ByteString.Char8 () + +import Crypto.Hash +import Crypto.PubKey.MaskGenFunction + +import Utils + +data VectorMgf = VectorMgf + { seed :: ByteString + , dbMask :: ByteString + } + +doMGFTest i vmgf = it (show i) (actual `shouldBe` dbMask vmgf) + where + actual = mgf1 SHA1 (seed vmgf) (B.length $ dbMask vmgf) + +vectorsMGF = + [ VectorMgf + { seed = + "\xdf\x1a\x89\x6f\x9d\x8b\xc8\x16\xd9\x7c\xd7\xa2\xc4\x3b\xad\x54\x6f\xbe\x8c\xfe" + , dbMask = + "\x66\xe4\x67\x2e\x83\x6a\xd1\x21\xba\x24\x4b\xed\x65\x76\xb8\x67\xd9\xa4\x47\xc2\x8a\x6e\x66\xa5\xb8\x7d\xee\x7f\xbc\x7e\x65\xaf\x50\x57\xf8\x6f\xae\x89\x84\xd9\xba\x7f\x96\x9a\xd6\xfe\x02\xa4\xd7\x5f\x74\x45\xfe\xfd\xd8\x5b\x6d\x3a\x47\x7c\x28\xd2\x4b\xa1\xe3\x75\x6f\x79\x2d\xd1\xdc\xe8\xca\x94\x44\x0e\xcb\x52\x79\xec\xd3\x18\x3a\x31\x1f\xc8\x97\x39\xa9\x66\x43\x13\x6e\x8b\x0f\x46\x5e\x87\xa4\x53\x5c\xd4\xc5\x9b\x10\x02\x8d" + } + ] + +spec :: Spec +spec = zipWithM_ doMGFTest [katZero ..] vectorsMGF diff --git a/tests/PubKey/OAEPSpec.hs b/tests/PubKey/OAEPSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/OAEPSpec.hs @@ -0,0 +1,208 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.OAEPSpec (spec) where + +import Crypto.Hash +import Crypto.Number.Serialize (i2ospOf_, os2ip) +import Crypto.PubKey.RSA +import qualified Crypto.PubKey.RSA.OAEP as OAEP +import Crypto.PubKey.RSA.Prim (dp, ep) + +import Data.Bits (xor) +import qualified Data.ByteString as B + +import Imports + +rsaKeyInt = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0xbbf82f090682ce9c2338ac2b9da871f7368d07eed41043a440d6b6f07454f51fb8dfbaaf035c02ab61ea48ceeb6fcd4876ed520d60e1ec4619719d8a5b8b807fafb8e0a3dfc737723ee6b4b7d93a2584ee6a649d060953748834b2454598394ee0aab12d7b61a51f527a9a41f6c1687fe2537298ca2a8f5946f8e5fd091dbdcb + , public_e = 0x11 + , public_size = 128 + } + , private_d = + 0xa5dafc5341faf289c4b988db30c1cdf83f31251e0668b42784813801579641b29410b3c7998d6bc465745e5c392669d6870da2c082a939e37fdcb82ec93edac97ff3ad5950accfbc111c76f1a9529444e56aaf68c56c092cd38dc3bef5d20a939926ed4f74a13eddfbe1a1cecc4894af9428c2b7b8883fe4463a4bc85b1cb3c1 + , private_p = + 0xeecfae81b1b9b3c908810b10a1b5600199eb9f44aef4fda493b81a9e3d84f632124ef0236e5d1e3b7e28fae7aa040a2d5b252176459d1f397541ba2a58fb6599 + , private_q = + 0xc97fb1f027f453f6341233eaaad1d9353f6c42d08866b1d05a0f2035028b9d869840b41666b42e92ea0da3b43204b5cfce3352524d0416a5a441e700af461503 + , private_dP = + 0x54494ca63eba0337e4e24023fcd69a5aeb07dddc0183a4d0ac9b54b051f2b13ed9490975eab77414ff59c1f7692e9a2e202b38fc910a474174adc93c1f67c981 + , private_dQ = + 0x471e0290ff0af0750351b7f878864ca961adbd3a8a7e991c5c0556a94c3146a7f9803f8f6f8ae342e931fd8ae47a220d1b99a495849807fe39f9245a9836da3d + , private_qinv = + 0xb06c4fdabb6301198d265bdbae9423b380f271f73453885093077fcd39e2119fc98632154f5883b167a967bf402b4e9e2e0f9656e698ea3666edfb25798039f7 + } + +rsaKey1 = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0xa8b3b284af8eb50b387034a860f146c4919f318763cd6c5598c8ae4811a1e0abc4c7e0b082d693a5e7fced675cf4668512772c0cbc64a742c6c630f533c8cc72f62ae833c40bf25842e984bb78bdbf97c0107d55bdb662f5c4e0fab9845cb5148ef7392dd3aaff93ae1e6b667bb3d4247616d4f5ba10d4cfd226de88d39f16fb + , public_e = 0x010001 + , public_size = 128 + } + , private_d = + 0x53339cfdb79fc8466a655c7316aca85c55fd8f6dd898fdaf119517ef4f52e8fd8e258df93fee180fa0e4ab29693cd83b152a553d4ac4d1812b8b9fa5af0e7f55fe7304df41570926f3311f15c4d65a732c483116ee3d3d2d0af3549ad9bf7cbfb78ad884f84d5beb04724dc7369b31def37d0cf539e9cfcdd3de653729ead5d1 + , private_p = + 0xd32737e7267ffe1341b2d5c0d150a81b586fb3132bed2f8d5262864a9cb9f30af38be448598d413a172efb802c21acf1c11c520c2f26a471dcad212eac7ca39d + , private_q = + 0xcc8853d1d54da630fac004f471f281c7b8982d8224a490edbeb33d3e3d5cc93c4765703d1dd791642f1f116a0dd852be2419b2af72bfe9a030e860b0288b5d77 + , private_dP = + 0x0e12bf1718e9cef5599ba1c3882fe8046a90874eefce8f2ccc20e4f2741fb0a33a3848aec9c9305fbecbd2d76819967d4671acc6431e4037968db37878e695c1 + , private_dQ = + 0x95297b0f95a2fa67d00707d609dfd4fc05c89dafc2ef6d6ea55bec771ea333734d9251e79082ecda866efef13c459e1a631386b7e354c899f5f112ca85d71583 + , private_qinv = + 0x4f456c502493bdc0ed2ab756a3a6ed4d67352a697d4216e93212b127a63d5411ce6fa98d5dbefd73263e3728142743818166ed7dd63687dd2a8ca1d2f4fbd8e1 + } + +data VectorOAEP = VectorOAEP + { seed :: ByteString + , message :: ByteString + , cipherText :: ByteString + } +vectorInt = + VectorOAEP + { message = "\xd4\x36\xe9\x95\x69\xfd\x32\xa7\xc8\xa0\x5b\xbc\x90\xd3\x2c\x49" + , seed = + "\xaa\xfd\x12\xf6\x59\xca\xe6\x34\x89\xb4\x79\xe5\x07\x6d\xde\xc2\xf0\x6c\xb5\x8f" + , cipherText = + "\x12\x53\xe0\x4d\xc0\xa5\x39\x7b\xb4\x4a\x7a\xb8\x7e\x9b\xf2\xa0\x39\xa3\x3d\x1e\x99\x6f\xc8\x2a\x94\xcc\xd3\x00\x74\xc9\x5d\xf7\x63\x72\x20\x17\x06\x9e\x52\x68\xda\x5d\x1c\x0b\x4f\x87\x2c\xf6\x53\xc1\x1d\xf8\x23\x14\xa6\x79\x68\xdf\xea\xe2\x8d\xef\x04\xbb\x6d\x84\xb1\xc3\x1d\x65\x4a\x19\x70\xe5\x78\x3b\xd6\xeb\x96\xa0\x24\xc2\xca\x2f\x4a\x90\xfe\x9f\x2e\xf5\xc9\xc1\x40\xe5\xbb\x48\xda\x95\x36\xad\x87\x00\xc8\x4f\xc9\x13\x0a\xde\xa7\x4e\x55\x8d\x51\xa7\x4d\xdf\x85\xd8\xb5\x0d\xe9\x68\x38\xd6\x06\x3e\x09\x55" + } + +vectorsKey1 = + [ VectorOAEP -- 1.1 + { message = + "\x66\x28\x19\x4e\x12\x07\x3d\xb0\x3b\xa9\x4c\xda\x9e\xf9\x53\x23\x97\xd5\x0d\xba\x79\xb9\x87\x00\x4a\xfe\xfe\x34" + , seed = + "\x18\xb7\x76\xea\x21\x06\x9d\x69\x77\x6a\x33\xe9\x6b\xad\x48\xe1\xdd\xa0\xa5\xef" + , cipherText = + "\x35\x4f\xe6\x7b\x4a\x12\x6d\x5d\x35\xfe\x36\xc7\x77\x79\x1a\x3f\x7b\xa1\x3d\xef\x48\x4e\x2d\x39\x08\xaf\xf7\x22\xfa\xd4\x68\xfb\x21\x69\x6d\xe9\x5d\x0b\xe9\x11\xc2\xd3\x17\x4f\x8a\xfc\xc2\x01\x03\x5f\x7b\x6d\x8e\x69\x40\x2d\xe5\x45\x16\x18\xc2\x1a\x53\x5f\xa9\xd7\xbf\xc5\xb8\xdd\x9f\xc2\x43\xf8\xcf\x92\x7d\xb3\x13\x22\xd6\xe8\x81\xea\xa9\x1a\x99\x61\x70\xe6\x57\xa0\x5a\x26\x64\x26\xd9\x8c\x88\x00\x3f\x84\x77\xc1\x22\x70\x94\xa0\xd9\xfa\x1e\x8c\x40\x24\x30\x9c\xe1\xec\xcc\xb5\x21\x00\x35\xd4\x7a\xc7\x2e\x8a" + } + , VectorOAEP -- 1.2 + { message = + "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" + , seed = + "\x0c\xc7\x42\xce\x4a\x9b\x7f\x32\xf9\x51\xbc\xb2\x51\xef\xd9\x25\xfe\x4f\xe3\x5f" + , cipherText = + "\x64\x0d\xb1\xac\xc5\x8e\x05\x68\xfe\x54\x07\xe5\xf9\xb7\x01\xdf\xf8\xc3\xc9\x1e\x71\x6c\x53\x6f\xc7\xfc\xec\x6c\xb5\xb7\x1c\x11\x65\x98\x8d\x4a\x27\x9e\x15\x77\xd7\x30\xfc\x7a\x29\x93\x2e\x3f\x00\xc8\x15\x15\x23\x6d\x8d\x8e\x31\x01\x7a\x7a\x09\xdf\x43\x52\xd9\x04\xcd\xeb\x79\xaa\x58\x3a\xdc\xc3\x1e\xa6\x98\xa4\xc0\x52\x83\xda\xba\x90\x89\xbe\x54\x91\xf6\x7c\x1a\x4e\xe4\x8d\xc7\x4b\xbb\xe6\x64\x3a\xef\x84\x66\x79\xb4\xcb\x39\x5a\x35\x2d\x5e\xd1\x15\x91\x2d\xf6\x96\xff\xe0\x70\x29\x32\x94\x6d\x71\x49\x2b\x44" + } + , VectorOAEP -- 1.3 + { message = + "\xd9\x4a\xe0\x83\x2e\x64\x45\xce\x42\x33\x1c\xb0\x6d\x53\x1a\x82\xb1\xdb\x4b\xaa\xd3\x0f\x74\x6d\xc9\x16\xdf\x24\xd4\xe3\xc2\x45\x1f\xff\x59\xa6\x42\x3e\xb0\xe1\xd0\x2d\x4f\xe6\x46\xcf\x69\x9d\xfd\x81\x8c\x6e\x97\xb0\x51" + , seed = + "\x25\x14\xdf\x46\x95\x75\x5a\x67\xb2\x88\xea\xf4\x90\x5c\x36\xee\xc6\x6f\xd2\xfd" + , cipherText = + "\x42\x37\x36\xed\x03\x5f\x60\x26\xaf\x27\x6c\x35\xc0\xb3\x74\x1b\x36\x5e\x5f\x76\xca\x09\x1b\x4e\x8c\x29\xe2\xf0\xbe\xfe\xe6\x03\x59\x5a\xa8\x32\x2d\x60\x2d\x2e\x62\x5e\x95\xeb\x81\xb2\xf1\xc9\x72\x4e\x82\x2e\xca\x76\xdb\x86\x18\xcf\x09\xc5\x34\x35\x03\xa4\x36\x08\x35\xb5\x90\x3b\xc6\x37\xe3\x87\x9f\xb0\x5e\x0e\xf3\x26\x85\xd5\xae\xc5\x06\x7c\xd7\xcc\x96\xfe\x4b\x26\x70\xb6\xea\xc3\x06\x6b\x1f\xcf\x56\x86\xb6\x85\x89\xaa\xfb\x7d\x62\x9b\x02\xd8\xf8\x62\x5c\xa3\x83\x36\x24\xd4\x80\x0f\xb0\x81\xb1\xcf\x94\xeb" + } + , VectorOAEP + { message = + "\x52\xe6\x50\xd9\x8e\x7f\x2a\x04\x8b\x4f\x86\x85\x21\x53\xb9\x7e\x01\xdd\x31\x6f\x34\x6a\x19\xf6\x7a\x85" + , seed = + "\xc4\x43\x5a\x3e\x1a\x18\xa6\x8b\x68\x20\x43\x62\x90\xa3\x7c\xef\xb8\x5d\xb3\xfb" + , cipherText = + "\x45\xea\xd4\xca\x55\x1e\x66\x2c\x98\x00\xf1\xac\xa8\x28\x3b\x05\x25\xe6\xab\xae\x30\xbe\x4b\x4a\xba\x76\x2f\xa4\x0f\xd3\xd3\x8e\x22\xab\xef\xc6\x97\x94\xf6\xeb\xbb\xc0\x5d\xdb\xb1\x12\x16\x24\x7d\x2f\x41\x2f\xd0\xfb\xa8\x7c\x6e\x3a\xcd\x88\x88\x13\x64\x6f\xd0\xe4\x8e\x78\x52\x04\xf9\xc3\xf7\x3d\x6d\x82\x39\x56\x27\x22\xdd\xdd\x87\x71\xfe\xc4\x8b\x83\xa3\x1e\xe6\xf5\x92\xc4\xcf\xd4\xbc\x88\x17\x4f\x3b\x13\xa1\x12\xaa\xe3\xb9\xf7\xb8\x0e\x0f\xc6\xf7\x25\x5b\xa8\x80\xdc\x7d\x80\x21\xe2\x2a\xd6\xa8\x5f\x07\x55" + } + , VectorOAEP + { message = + "\x8d\xa8\x9f\xd9\xe5\xf9\x74\xa2\x9f\xef\xfb\x46\x2b\x49\x18\x0f\x6c\xf9\xe8\x02" + , seed = + "\xb3\x18\xc4\x2d\xf3\xbe\x0f\x83\xfe\xa8\x23\xf5\xa7\xb4\x7e\xd5\xe4\x25\xa3\xb5" + , cipherText = + "\x36\xf6\xe3\x4d\x94\xa8\xd3\x4d\xaa\xcb\xa3\x3a\x21\x39\xd0\x0a\xd8\x5a\x93\x45\xa8\x60\x51\xe7\x30\x71\x62\x00\x56\xb9\x20\xe2\x19\x00\x58\x55\xa2\x13\xa0\xf2\x38\x97\xcd\xcd\x73\x1b\x45\x25\x7c\x77\x7f\xe9\x08\x20\x2b\xef\xdd\x0b\x58\x38\x6b\x12\x44\xea\x0c\xf5\x39\xa0\x5d\x5d\x10\x32\x9d\xa4\x4e\x13\x03\x0f\xd7\x60\xdc\xd6\x44\xcf\xef\x20\x94\xd1\x91\x0d\x3f\x43\x3e\x1c\x7c\x6d\xd1\x8b\xc1\xf2\xdf\x7f\x64\x3d\x66\x2f\xb9\xdd\x37\xea\xd9\x05\x91\x90\xf4\xfa\x66\xca\x39\xe8\x69\xc4\xeb\x44\x9c\xbd\xc4\x39" + } + , VectorOAEP -- 1.6 + { message = "\x26\x52\x10\x50\x84\x42\x71" + , seed = + "\xe4\xec\x09\x82\xc2\x33\x6f\x3a\x67\x7f\x6a\x35\x61\x74\xeb\x0c\xe8\x87\xab\xc2" + , cipherText = + "\x42\xce\xe2\x61\x7b\x1e\xce\xa4\xdb\x3f\x48\x29\x38\x6f\xbd\x61\xda\xfb\xf0\x38\xe1\x80\xd8\x37\xc9\x63\x66\xdf\x24\xc0\x97\xb4\xab\x0f\xac\x6b\xdf\x59\x0d\x82\x1c\x9f\x10\x64\x2e\x68\x1a\xd0\x5b\x8d\x78\xb3\x78\xc0\xf4\x6c\xe2\xfa\xd6\x3f\x74\xe0\xad\x3d\xf0\x6b\x07\x5d\x7e\xb5\xf5\x63\x6f\x8d\x40\x3b\x90\x59\xca\x76\x1b\x5c\x62\xbb\x52\xaa\x45\x00\x2e\xa7\x0b\xaa\xce\x08\xde\xd2\x43\xb9\xd8\xcb\xd6\x2a\x68\xad\xe2\x65\x83\x2b\x56\x56\x4e\x43\xa6\xfa\x42\xed\x19\x9a\x09\x97\x69\x74\x2d\xf1\x53\x9e\x82\x55" + } + ] + +doEncryptionTest key i vec = it (show i) (actual `shouldBe` Right (cipherText vec)) + where + actual = + OAEP.encryptWithSeed (seed vec) (OAEP.defaultOAEPParams SHA1) key (message vec) + +doDecryptionTest key i vec = it (show i) (actual `shouldBe` Right (message vec)) + where + actual = OAEP.decrypt Nothing (OAEP.defaultOAEPParams SHA1) key (cipherText vec) + +-- | EME-OAEP decoding rejects a block whose leading octet is not zero, whose +-- recovered label hash does not match, or which has no 01 separating the +-- padding from the message (RFC 8017 section 7.1.2). Reach those paths by +-- decrypting a known-good ciphertext to its encoded message, corrupting that, +-- and re-encrypting under the public key. +-- +-- Nothing exercised them before, and unpad is about to be rewritten, so pin +-- the behaviour down first. +oaepRejectTests :: Spec +oaepRejectTests = + describe "rejected blocks" $ do + it "the untouched block still decrypts" $ + decrypt' (reencrypt em) `shouldBe` Right (message vec) + rejects "a leading octet that is not 00" (poke 0 1 em) + rejects "a corrupted masked seed" (flipBit 3 em) + rejects "a corrupted masked db" (flipBit 60 em) + rejects "a corrupted final octet" (flipBit (B.length em - 1) em) + it "a ciphertext of the wrong length" $ + decrypt' (B.drop 1 (cipherText vec)) `shouldBe` Left MessageSizeIncorrect + where + key = rsaKey1 + vec = firstVector vectorsKey1 + em = dp Nothing key (cipherText vec) + reencrypt = ep (private_pub key) + decrypt' = OAEP.decrypt Nothing (OAEP.defaultOAEPParams SHA1) key + rejects name bad = + it name (decrypt' (reencrypt bad) `shouldBe` Left MessageNotRecognized) + poke i w bs = + B.concat [B.take i bs, B.singleton w, B.drop (i + 1) bs] + flipBit i bs = poke i (B.index bs i `xor` 1) bs + +-- | RSADP (RFC 8017 section 5.1.2 step 1) refuses a ciphertext representative +-- outside @[0, n-1]@, and section 7.1.2 step 1 passes the ciphertext to it +-- unchanged. The modular exponentiation normalises the range away, so without +-- the check @c@ and @c + n@ decrypt to the same message whenever @c + n@ still +-- fits in k octets. +oaepRangeTests :: Spec +oaepRangeTests = + describe "ciphertext range" $ do + it "the ciphertext itself decrypts" $ + decrypt' c `shouldBe` Right (message vec) + it "the same ciphertext plus n is refused" $ + decrypt' (i2ospOf_ k (os2ip c + modulus)) `shouldBe` Left MessageSizeIncorrect + it "a ciphertext representative equal to the modulus is refused" $ + decrypt' (i2ospOf_ k modulus) `shouldBe` Left MessageSizeIncorrect + where + key = rsaKey1 + k = public_size (private_pub key) + modulus = public_n (private_pub key) + decrypt' = OAEP.decrypt Nothing (OAEP.defaultOAEPParams SHA1) key + -- the first vector whose ciphertext can be shifted by n and still fit in k + -- octets + (vec, c) = + firstVector + [ (v, ct) + | v <- vectorsKey1 + , let ct = cipherText v + , os2ip ct + modulus < 2 ^ (8 * k) + ] + +spec :: Spec +spec = + describe "RSA-OAEP" $ do + describe "internal" $ do + doEncryptionTest (private_pub rsaKeyInt) (0 :: Int) vectorInt + doDecryptionTest rsaKeyInt (0 :: Int) vectorInt + describe "encryption key 1024 bits" $ + sequence_ $ + zipWith (doEncryptionTest $ private_pub rsaKey1) [katZero ..] vectorsKey1 + describe "decryption key 1024 bits" $ + sequence_ $ + zipWith (doDecryptionTest rsaKey1) [katZero ..] vectorsKey1 + oaepRejectTests + oaepRangeTests diff --git a/tests/PubKey/P256Spec.hs b/tests/PubKey/P256Spec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/P256Spec.hs @@ -0,0 +1,286 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE ScopedTypeVariables #-} + +module PubKey.P256Spec (spec) where + +import qualified Crypto.PubKey.ECC.P256 as P256 +import qualified Crypto.PubKey.ECC.Prim as ECC +import qualified Crypto.PubKey.ECC.Types as ECC + +import Crypto.Error +import Crypto.Number.ModArithmetic (inverseCoprimes) +import Crypto.Number.Serialize (i2ospOf, os2ip) +import Data.ByteArray (Bytes) + +import Imports + +newtype P256Scalar = P256Scalar Integer + deriving (Show, Eq, Ord) + +instance Arbitrary P256Scalar where + -- Cover the full range up to 2^256-1 except 0 and curveN. To test edge + -- cases with arithmetic functions, some values close to 0, curveN and + -- 2^256 are given higher frequency. + arbitrary = + P256Scalar + <$> oneof + [ choose (1, w) + , choose (w + 1, curveN - w - 1) + , choose (curveN - w, curveN - 1) + , choose (curveN + 1, curveN + w) + , choose (curveN + w + 1, high - w - 1) + , choose (high - w, high - 1) + ] + where + high = 2 ^ (256 :: Int) + w = 100 + +curve = ECC.getCurveByName ECC.SEC_p256r1 +curveN = ECC.ecc_n . ECC.common_curve $ curve +curveGen = ECC.ecc_g . ECC.common_curve $ curve + +pointP256ToECC :: P256.Point -> ECC.Point +pointP256ToECC p + | P256.pointIsAtInfinity p = ECC.PointO + | otherwise = uncurry ECC.Point (P256.pointToIntegers p) + +i2ospScalar :: Integer -> Bytes +i2ospScalar i = + case i2ospOf 32 i of + Nothing -> error "invalid size of P256 scalar" + Just b -> b + +unP256Scalar :: P256Scalar -> P256.Scalar +unP256Scalar (P256Scalar r) = + let rBytes = i2ospScalar r + in case P256.scalarFromBinary rBytes of + CryptoFailed err -> error ("cannot convert scalar: " ++ show err) + CryptoPassed scalar -> scalar + +unP256 :: P256Scalar -> Integer +unP256 (P256Scalar r) = r + +modP256Scalar :: P256Scalar -> P256Scalar +modP256Scalar (P256Scalar r) = P256Scalar (r `mod` curveN) + +p256ScalarToInteger :: P256.Scalar -> Integer +p256ScalarToInteger s = os2ip (P256.scalarToBinary s :: Bytes) + +xS = 0xde2444bebc8d36e682edd27e0f271508617519b3221a8fa0b77cab3989da97c9 +yS = 0xc093ae7ff36e5380fc01a5aad1e66659702de80f53cec576b6350b243042a256 +xT = 0x55a8b00f8da1d44e62f6b3b25316212e39540dc861c89575bb8cf92e35e0986b +yT = 0x5421c3209c2d6c704835d82ac4c3dd90f61a8a52598b9e7ab656e9d8c8b24316 +xR = 0x72b13dd4354b6b81745195e98cc5ba6970349191ac476bd4553cf35a545a067e +yR = 0x8d585cbb2e1327d75241a8a122d7620dc33b13315aa5c9d46d013011744ac264 + +-- Two points on the curve whose validation reduces a product whose top +-- digit has a zero low half: x = 2^96, and an x with a repeating bit +-- pattern. Wycheproof ecdh_secp256r1_ecpoint tcId 74 and 93. +xU = 0x0000000000000000000000000000000000000001000000000000000000000000 +yU = 0x7d12de58d54423eb85ae8d157ae416fb004a7eb522ac1b67047ef3cdf9acdc3f +xV = 0x8000003ffffff0000007fffffe000000ffffffc000001ffffff8000003fffffc +yV = 0x0c3527bd081c1c07b313bc1a0c3f845fb2fe22557699ccc8f1354e61a27b7f88 + +validPointEdgeCases :: [(String, (Integer, Integer))] +validPointEdgeCases = + [ + ( "x-zero-1" + , (0, 0x66485c780e2f83d72433bd5d84a06bb6541c2af31dae871728bf856a174f93f4) + ) + , + ( "x-zero-2" + , (0, 0x99b7a386f1d07c29dbcc42a27b5f9449abe3d50de25178e8d7407a95e8b06c0b) + ) + , + ( "y-one-1" + , (0x09e78d4ef60d05f750f6636209092bc43cbdd6b47e11a9de20a9feb2a50bb96c, 1) + ) + , + ( "y-one-2" + , (0x8d0177ebab9c6e9e10db6dd095dbac0d6375e8a97b70f611875d877f0069d2c7, 1) + ) + , + ( "y-one-3" + , (0x6916fac45e568b6b9e2e2ecd611b282e5fcc40a3067d601057f879ce5a8a73cc, 1) + ) + ] + +spec :: Spec +spec = do + describe "scalar" $ do + prop "marshalling" $ \(QAInteger r) -> + let rBytes = i2ospScalar r + in case P256.scalarFromBinary rBytes of + CryptoFailed err -> error (show err) + CryptoPassed scalar -> rBytes `propertyEq` P256.scalarToBinary scalar + prop "add" $ \r1 r2 -> + let r = (unP256 r1 + unP256 r2) `mod` curveN + r' = P256.scalarAdd (unP256Scalar r1) (unP256Scalar r2) + in r `propertyEq` p256ScalarToInteger r' + prop "add0" $ \r -> + let v = unP256 r `mod` curveN + v' = P256.scalarAdd (unP256Scalar r) P256.scalarZero + in v `propertyEq` p256ScalarToInteger v' + prop "sub" $ \r1 r2 -> + let r = (unP256 r1 - unP256 r2) `mod` curveN + r' = P256.scalarSub (unP256Scalar r1) (unP256Scalar r2) + v = (unP256 r2 - unP256 r1) `mod` curveN + v' = P256.scalarSub (unP256Scalar r2) (unP256Scalar r1) + in propertyHold + [ eqTest "r1-r2" r (p256ScalarToInteger r') + , eqTest "r2-r1" v (p256ScalarToInteger v') + ] + prop "sub0" $ \r -> + let v = unP256 r `mod` curveN + v' = P256.scalarSub (unP256Scalar r) P256.scalarZero + in v `propertyEq` p256ScalarToInteger v' + prop "mul" $ \r1 r2 -> + let r = (unP256 r1 * unP256 r2) `mod` curveN + r' = P256.scalarMul (unP256Scalar r1) (unP256Scalar r2) + in r `propertyEq` p256ScalarToInteger r' + prop "inv" $ \r' -> + let inv = inverseCoprimes (unP256 r') curveN + inv' = P256.scalarInv (unP256Scalar r') + in unP256 r' /= 0 ==> inv `propertyEq` p256ScalarToInteger inv' + prop "inv-safe" $ \r' -> + let inv = P256.scalarInv (unP256Scalar r') + inv' = P256.scalarInvSafe (unP256Scalar r') + in unP256 r' /= 0 ==> inv `propertyEq` inv' + prop "inv-safe-mul" $ \r' -> + let inv = P256.scalarInvSafe (unP256Scalar r') + res = P256.scalarMul (unP256Scalar r') inv + in unP256 r' /= 0 ==> 1 `propertyEq` p256ScalarToInteger res + prop "inv-safe-zero" $ + let inv0 = P256.scalarInvSafe P256.scalarZero + invN = P256.scalarInvSafe P256.scalarN + in propertyHold + [ eqTest "scalarZero" P256.scalarZero inv0 + , eqTest "scalarN" P256.scalarZero invN + ] + describe "point" $ do + prop "marshalling" $ \rx ry -> + let p = P256.pointFromIntegers (unP256 rx, unP256 ry) + b = P256.pointToBinary p :: Bytes + p' = P256.unsafePointFromBinary b + in propertyHold [eqTest "point" (CryptoPassed p) p'] + prop "marshalling-integer" $ \rx ry -> + let p = P256.pointFromIntegers (unP256 rx, unP256 ry) + (x, y) = P256.pointToIntegers p + in propertyHold [eqTest "x" (unP256 rx) x, eqTest "y" (unP256 ry) y] + it "valid-point-1" $ casePointIsValid (xS, yS) + it "valid-point-2" $ casePointIsValid (xR, yR) + it "valid-point-3" $ casePointIsValid (xT, yT) + -- The quotient estimate in crypton_p256_modmul can exceed the + -- true quotient, and the resulting borrow used to abort the + -- process on an assertion inside the reduction rather than + -- being corrected. Both points below are on the curve. + it "valid-point-reduction-1" $ casePointIsValid (xU, yU) + it "valid-point-reduction-2" $ casePointIsValid (xV, yV) + describe "valid-point-edge-cases" $ + sequence_ $ + map (\(name, point) -> it name $ casePointIsValid point) validPointEdgeCases + it "point-add-1" $ + let s = P256.pointFromIntegers (xS, yS) + t = P256.pointFromIntegers (xT, yT) + r = P256.pointFromIntegers (xR, yR) + in P256.pointAdd s t `shouldBe` r + prop "point-add-infinity" casePointAddInfinity + prop "lift-to-curve" propertyLiftToCurve + prop "point-add" propertyPointAdd + prop "point-add-infinity-identity" propertyPointAddInfinityIdentity + prop "point-add-inverse" propertyPointAddInverse + prop "point-negate" propertyPointNegate + prop "point-mul" propertyPointMul + prop "infinity" $ + let gN = P256.toPoint P256.scalarN + g1 = P256.pointBase + in propertyHold + [ eqTest "zero" True (P256.pointIsAtInfinity gN) + , eqTest "base" False (P256.pointIsAtInfinity g1) + ] + where + casePointIsValid pointTuple = + let s = P256.pointFromIntegers pointTuple in P256.pointIsValid s `shouldBe` True + + propertyLiftToCurve r = + let p = P256.toPoint (unP256Scalar r) + (x, y) = P256.pointToIntegers p + pEcc = ECC.pointMul curve (unP256 r) curveGen + in pEcc `propertyEq` ECC.Point x y + + propertyPointAdd r1 r2 = + let p1 = P256.toPoint (unP256Scalar r1) + p2 = P256.toPoint (unP256Scalar r2) + pe1 = ECC.pointMul curve (unP256 r1) curveGen + pe2 = ECC.pointMul curve (unP256 r2) curveGen + pR = P256.toPoint (P256.scalarAdd (unP256Scalar r1) (unP256Scalar r2)) + peR = ECC.pointAdd curve pe1 pe2 + in (unP256 r1 + unP256 r2) `mod` curveN + /= 0 + ==> propertyHold + [ eqTest "p256" pR (P256.pointAdd p1 p2) + , eqTest "ecc" peR (pointP256ToECC pR) + ] + + propertyPointNegate r = + let p = P256.toPoint (unP256Scalar r) + pe = ECC.pointMul curve (unP256 r) curveGen + pR = P256.pointNegate p + in ECC.pointNegate curve pe `propertyEq` pointP256ToECC pR + + propertyPointMul s' r' = + let s = modP256Scalar s' + r = modP256Scalar r' + p = P256.toPoint (unP256Scalar r) + pe = ECC.pointMul curve (unP256 r) curveGen + pR = P256.toPoint (P256.scalarMul (unP256Scalar s) (unP256Scalar r)) + peR = ECC.pointMul curve (unP256 s) pe + in propertyHold + [ eqTest "p256" pR (P256.pointMul (unP256Scalar s) p) + , eqTest "ecc" peR (pointP256ToECC pR) + ] + + pointInfinity :: P256.Point + pointInfinity = P256.pointFromIntegers (0, 0) + + casePointAddInfinity = + propertyHold + [ eqTest + "infinity + base" + P256.pointBase + (P256.pointAdd pointInfinity P256.pointBase) + , eqTest + "base + infinity" + P256.pointBase + (P256.pointAdd P256.pointBase pointInfinity) + , eqTest + "infinity + infinity" + pointInfinity + (P256.pointAdd pointInfinity pointInfinity) + ] + + propertyPointAddInfinityIdentity r = + let p = P256.toPoint (unP256Scalar r) + in propertyHold + [ eqTest + "infinity + p" + p + (P256.pointAdd pointInfinity p) + , eqTest + "p + infinity" + p + (P256.pointAdd p pointInfinity) + ] + + propertyPointAddInverse r = + let p = P256.toPoint (unP256Scalar r) + in propertyHold + [ eqTest + "p + negate p" + True + (P256.pointIsAtInfinity (P256.pointAdd p (P256.pointNegate p))) + , eqTest + "negate p + p" + True + (P256.pointIsAtInfinity (P256.pointAdd (P256.pointNegate p) p)) + ] diff --git a/tests/PubKey/PSSSpec.hs b/tests/PubKey/PSSSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/PSSSpec.hs @@ -0,0 +1,528 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.PSSSpec (spec) where + +import Crypto.Number.Serialize (i2ospOf_, os2ip) +import Crypto.PubKey.RSA +import qualified Crypto.PubKey.RSA.PSS as PSS + +import Imports + +-- Module contains one vector generated by the implementation itself and other +-- vectors from + +data VectorPSS = VectorPSS + { message :: ByteString + , salt :: ByteString + , signature :: ByteString + } + +rsaKeyInt = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0xa2ba40ee07e3b2bd2f02ce227f36a195024486e49c19cb41bbbdfbba98b22b0e577c2eeaffa20d883a76e65e394c69d4b3c05a1e8fadda27edb2a42bc000fe888b9b32c22d15add0cd76b3e7936e19955b220dd17d4ea904b1ec102b2e4de7751222aa99151024c7cb41cc5ea21d00eeb41f7c800834d2c6e06bce3bce7ea9a5 + , public_e = 0x010001 + , public_size = 128 + } + , private_d = + 0x50e2c3e38d886110288dfc68a9533e7e12e27d2aa56d2cdb3fb6efa990bcff29e1d2987fb711962860e7391b1ce01ebadb9e812d2fbdfaf25df4ae26110a6d7a26f0b810f54875e17dd5c9fb6d641761245b81e79f8c88f0e55a6dcd5f133abd35f8f4ec80adf1bf86277a582894cb6ebcd2162f1c7534f1f4947b129151b71 + , private_p = + 0xd17f655bf27c8b16d35462c905cc04a26f37e2a67fa9c0ce0dced472394a0df743fe7f929e378efdb368eddff453cf007af6d948e0ade757371f8a711e278f6b + , private_q = + 0xc6d92b6fee7414d1358ce1546fb62987530b90bd15e0f14963a5e2635adb69347ec0c01b2ab1763fd8ac1a592fb22757463a982425bb97a3a437c5bf86d03f2f + , private_dP = + 0x9d0dbf83e5ce9e4b1754dcd5cd05bcb7b55f1508330ea49f14d4e889550f8256cb5f806dff34b17ada44208853577d08e4262890acf752461cea05547601bc4f + , private_dQ = + 0x1291a524c6b7c059e90e46dc83b2171eb3fa98818fd179b6c8bf6cecaa476303abf283fe05769cfc495788fe5b1ddfde9e884a3cd5e936b7e955ebf97eb563b1 + , private_qinv = + 0xa63f1da38b950c9ad1c67ce0d677ec2914cd7d40062df42a67eb198a176f9742aac7c5fea14f2297662b84812c4defc49a8025ab4382286be4c03788dd01d69f + } + +rsaKey1 = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0xa56e4a0e701017589a5187dc7ea841d156f2ec0e36ad52a44dfeb1e61f7ad991d8c51056ffedb162b4c0f283a12a88a394dff526ab7291cbb307ceabfce0b1dfd5cd9508096d5b2b8b6df5d671ef6377c0921cb23c270a70e2598e6ff89d19f105acc2d3f0cb35f29280e1386b6f64c4ef22e1e1f20d0ce8cffb2249bd9a2137 + , public_e = 0x010001 + , public_size = 128 + } + , private_d = + 0x33a5042a90b27d4f5451ca9bbbd0b44771a101af884340aef9885f2a4bbe92e894a724ac3c568c8f97853ad07c0266c8c6a3ca0929f1e8f11231884429fc4d9ae55fee896a10ce707c3ed7e734e44727a39574501a532683109c2abacaba283c31b4bd2f53c3ee37e352cee34f9e503bd80c0622ad79c6dcee883547c6a3b325 + , private_p = + 0xe7e8942720a877517273a356053ea2a1bc0c94aa72d55c6e86296b2dfc967948c0a72cbccca7eacb35706e09a1df55a1535bd9b3cc34160b3b6dcd3eda8e6443 + , private_q = + 0xb69dca1cf7d4d7ec81e75b90fcca874abcde123fd2700180aa90479b6e48de8d67ed24f9f19d85ba275874f542cd20dc723e6963364a1f9425452b269a6799fd + , private_dP = + 0x28fa13938655be1f8a159cbaca5a72ea190c30089e19cd274a556f36c4f6e19f554b34c077790427bbdd8dd3ede2448328f385d81b30e8e43b2fffa027861979 + , private_dQ = + 0x1a8b38f398fa712049898d7fb79ee0a77668791299cdfa09efc0e507acb21ed74301ef5bfd48be455eaeb6e1678255827580a8e4e8e14151d1510a82a3f2e729 + , private_qinv = + 0x27156aba4126d24a81f3a528cbfb27f56886f840a9f6e86e17a44b94fe9319584b8e22fdde1e5a2e3bd8aa5ba8d8584194eb2190acf832b847f13a3d24a79f4d + } + +vectorInt = + VectorPSS + { message = + "\x85\x9e\xef\x2f\xd7\x8a\xca\x00\x30\x8b\xdc\x47\x11\x93\xbf\x55\xbf\x9d\x78\xdb\x8f\x8a\x67\x2b\x48\x46\x34\xf3\xc9\xc2\x6e\x64\x78\xae\x10\x26\x0f\xe0\xdd\x8c\x08\x2e\x53\xa5\x29\x3a\xf2\x17\x3c\xd5\x0c\x6d\x5d\x35\x4f\xeb\xf7\x8b\x26\x02\x1c\x25\xc0\x27\x12\xe7\x8c\xd4\x69\x4c\x9f\x46\x97\x77\xe4\x51\xe7\xf8\xe9\xe0\x4c\xd3\x73\x9c\x6b\xbf\xed\xae\x48\x7f\xb5\x56\x44\xe9\xca\x74\xff\x77\xa5\x3c\xb7\x29\x80\x2f\x6e\xd4\xa5\xff\xa8\xba\x15\x98\x90\xfc" + , salt = + "\xe3\xb5\xd5\xd0\x02\xc1\xbc\xe5\x0c\x2b\x65\xef\x88\xa1\x88\xd8\x3b\xce\x7e\x61" + , signature = + "\x8d\xaa\x62\x7d\x3d\xe7\x59\x5d\x63\x05\x6c\x7e\xc6\x59\xe5\x44\x06\xf1\x06\x10\x12\x8b\xaa\xe8\x21\xc8\xb2\xa0\xf3\x93\x6d\x54\xdc\x3b\xdc\xe4\x66\x89\xf6\xb7\x95\x1b\xb1\x8e\x84\x05\x42\x76\x97\x18\xd5\x71\x5d\x21\x0d\x85\xef\xbb\x59\x61\x92\x03\x2c\x42\xbe\x4c\x29\x97\x2c\x85\x62\x75\xeb\x6d\x5a\x45\xf0\x5f\x51\x87\x6f\xc6\x74\x3d\xed\xdd\x28\xca\xec\x9b\xb3\x0e\xa9\x9e\x02\xc3\x48\x82\x69\x60\x4f\xe4\x97\xf7\x4c\xcd\x7c\x7f\xca\x16\x71\x89\x71\x23\xcb\xd3\x0d\xef\x5d\x54\xa2\xb5\x53\x6a\xd9\x0a\x74\x7e" + } + +{- +# mHash = Hash(M) +# salt = random string of octets +# M' = Padding || mHash || salt +# H = Hash(M') +# DB = Padding || salt +# dbMask = MGF(H, length(DB)) +# maskedDB = DB xor dbMask (leftmost bit set to +# zero) +# EM = maskedDB || H || 0xbc + +# mHash: +37 b6 6a e0 44 58 43 35 3d 47 ec b0 b4 fd 14 c1 +10 e6 2d 6a + +# salt: + +# M': +00 00 00 00 00 00 00 00 37 b6 6a e0 44 58 43 35 +3d 47 ec b0 b4 fd 14 c1 10 e6 2d 6a e3 b5 d5 d0 +02 c1 bc e5 0c 2b 65 ef 88 a1 88 d8 3b ce 7e 61 + +# H: +df 1a 89 6f 9d 8b c8 16 d9 7c d7 a2 c4 3b ad 54 +6f be 8c fe + +# DB: +00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 +00 00 00 00 00 00 01 e3 b5 d5 d0 02 c1 bc e5 0c +2b 65 ef 88 a1 88 d8 3b ce 7e 61 + +# dbMask: +66 e4 67 2e 83 6a d1 21 ba 24 4b ed 65 76 b8 67 +d9 a4 47 c2 8a 6e 66 a5 b8 7d ee 7f bc 7e 65 af +50 57 f8 6f ae 89 84 d9 ba 7f 96 9a d6 fe 02 a4 +d7 5f 74 45 fe fd d8 5b 6d 3a 47 7c 28 d2 4b a1 +e3 75 6f 79 2d d1 dc e8 ca 94 44 0e cb 52 79 ec +d3 18 3a 31 1f c8 97 39 a9 66 43 13 6e 8b 0f 46 +5e 87 a4 53 5c d4 c5 9b 10 02 8d + +# maskedDB: +66 e4 67 2e 83 6a d1 21 ba 24 4b ed 65 76 b8 67 +d9 a4 47 c2 8a 6e 66 a5 b8 7d ee 7f bc 7e 65 af +50 57 f8 6f ae 89 84 d9 ba 7f 96 9a d6 fe 02 a4 +d7 5f 74 45 fe fd d8 5b 6d 3a 47 7c 28 d2 4b a1 +e3 75 6f 79 2d d1 dc e8 ca 94 44 0e cb 52 79 ec +d3 18 3a 31 1f c8 96 da 1c b3 93 11 af 37 ea 4a +75 e2 4b db fd 5c 1d a0 de 7c ec + +# Encoded message EM: +66 e4 67 2e 83 6a d1 21 ba 24 4b ed 65 76 b8 67 +d9 a4 47 c2 8a 6e 66 a5 b8 7d ee 7f bc 7e 65 af +50 57 f8 6f ae 89 84 d9 ba 7f 96 9a d6 fe 02 a4 +d7 5f 74 45 fe fd d8 5b 6d 3a 47 7c 28 d2 4b a1 +e3 75 6f 79 2d d1 dc e8 ca 94 44 0e cb 52 79 ec +d3 18 3a 31 1f c8 96 da 1c b3 93 11 af 37 ea 4a +75 e2 4b db fd 5c 1d a0 de 7c ec df 1a 89 6f 9d +8b c8 16 d9 7c d7 a2 c4 3b ad 54 6f be 8c fe bc +-} + +vectorsKey1 = + [ -- Example 1.1 + VectorPSS + { message = + "\xcd\xc8\x7d\xa2\x23\xd7\x86\xdf\x3b\x45\xe0\xbb\xbc\x72\x13\x26\xd1\xee\x2a\xf8\x06\xcc\x31\x54\x75\xcc\x6f\x0d\x9c\x66\xe1\xb6\x23\x71\xd4\x5c\xe2\x39\x2e\x1a\xc9\x28\x44\xc3\x10\x10\x2f\x15\x6a\x0d\x8d\x52\xc1\xf4\xc4\x0b\xa3\xaa\x65\x09\x57\x86\xcb\x76\x97\x57\xa6\x56\x3b\xa9\x58\xfe\xd0\xbc\xc9\x84\xe8\xb5\x17\xa3\xd5\xf5\x15\xb2\x3b\x8a\x41\xe7\x4a\xa8\x67\x69\x3f\x90\xdf\xb0\x61\xa6\xe8\x6d\xfa\xae\xe6\x44\x72\xc0\x0e\x5f\x20\x94\x57\x29\xcb\xeb\xe7\x7f\x06\xce\x78\xe0\x8f\x40\x98\xfb\xa4\x1f\x9d\x61\x93\xc0\x31\x7e\x8b\x60\xd4\xb6\x08\x4a\xcb\x42\xd2\x9e\x38\x08\xa3\xbc\x37\x2d\x85\xe3\x31\x17\x0f\xcb\xf7\xcc\x72\xd0\xb7\x1c\x29\x66\x48\xb3\xa4\xd1\x0f\x41\x62\x95\xd0\x80\x7a\xa6\x25\xca\xb2\x74\x4f\xd9\xea\x8f\xd2\x23\xc4\x25\x37\x02\x98\x28\xbd\x16\xbe\x02\x54\x6f\x13\x0f\xd2\xe3\x3b\x93\x6d\x26\x76\xe0\x8a\xed\x1b\x73\x31\x8b\x75\x0a\x01\x67\xd0" + , salt = + "\xde\xe9\x59\xc7\xe0\x64\x11\x36\x14\x20\xff\x80\x18\x5e\xd5\x7f\x3e\x67\x76\xaf" + , signature = + "\x90\x74\x30\x8f\xb5\x98\xe9\x70\x1b\x22\x94\x38\x8e\x52\xf9\x71\xfa\xac\x2b\x60\xa5\x14\x5a\xf1\x85\xdf\x52\x87\xb5\xed\x28\x87\xe5\x7c\xe7\xfd\x44\xdc\x86\x34\xe4\x07\xc8\xe0\xe4\x36\x0b\xc2\x26\xf3\xec\x22\x7f\x9d\x9e\x54\x63\x8e\x8d\x31\xf5\x05\x12\x15\xdf\x6e\xbb\x9c\x2f\x95\x79\xaa\x77\x59\x8a\x38\xf9\x14\xb5\xb9\xc1\xbd\x83\xc4\xe2\xf9\xf3\x82\xa0\xd0\xaa\x35\x42\xff\xee\x65\x98\x4a\x60\x1b\xc6\x9e\xb2\x8d\xeb\x27\xdc\xa1\x2c\x82\xc2\xd4\xc3\xf6\x6c\xd5\x00\xf1\xff\x2b\x99\x4d\x8a\x4e\x30\xcb\xb3\x3c" + } + , -- Example 1.2 + VectorPSS + { message = + "\x85\x13\x84\xcd\xfe\x81\x9c\x22\xed\x6c\x4c\xcb\x30\xda\xeb\x5c\xf0\x59\xbc\x8e\x11\x66\xb7\xe3\x53\x0c\x4c\x23\x3e\x2b\x5f\x8f\x71\xa1\xcc\xa5\x82\xd4\x3e\xcc\x72\xb1\xbc\xa1\x6d\xfc\x70\x13\x22\x6b\x9e" + , salt = + "\xef\x28\x69\xfa\x40\xc3\x46\xcb\x18\x3d\xab\x3d\x7b\xff\xc9\x8f\xd5\x6d\xf4\x2d" + , signature = + "\x3e\xf7\xf4\x6e\x83\x1b\xf9\x2b\x32\x27\x41\x42\xa5\x85\xff\xce\xfb\xdc\xa7\xb3\x2a\xe9\x0d\x10\xfb\x0f\x0c\x72\x99\x84\xf0\x4e\xf2\x9a\x9d\xf0\x78\x07\x75\xce\x43\x73\x9b\x97\x83\x83\x90\xdb\x0a\x55\x05\xe6\x3d\xe9\x27\x02\x8d\x9d\x29\xb2\x19\xca\x2c\x45\x17\x83\x25\x58\xa5\x5d\x69\x4a\x6d\x25\xb9\xda\xb6\x60\x03\xc4\xcc\xcd\x90\x78\x02\x19\x3b\xe5\x17\x0d\x26\x14\x7d\x37\xb9\x35\x90\x24\x1b\xe5\x1c\x25\x05\x5f\x47\xef\x62\x75\x2c\xfb\xe2\x14\x18\xfa\xfe\x98\xc2\x2c\x4d\x4d\x47\x72\x4f\xdb\x56\x69\xe8\x43" + } + , -- Example 1.3 + VectorPSS + { message = + "\xa4\xb1\x59\x94\x17\x61\xc4\x0c\x6a\x82\xf2\xb8\x0d\x1b\x94\xf5\xaa\x26\x54\xfd\x17\xe1\x2d\x58\x88\x64\x67\x9b\x54\xcd\x04\xef\x8b\xd0\x30\x12\xbe\x8d\xc3\x7f\x4b\x83\xaf\x79\x63\xfa\xff\x0d\xfa\x22\x54\x77\x43\x7c\x48\x01\x7f\xf2\xbe\x81\x91\xcf\x39\x55\xfc\x07\x35\x6e\xab\x3f\x32\x2f\x7f\x62\x0e\x21\xd2\x54\xe5\xdb\x43\x24\x27\x9f\xe0\x67\xe0\x91\x0e\x2e\x81\xca\x2c\xab\x31\xc7\x45\xe6\x7a\x54\x05\x8e\xb5\x0d\x99\x3c\xdb\x9e\xd0\xb4\xd0\x29\xc0\x6d\x21\xa9\x4c\xa6\x61\xc3\xce\x27\xfa\xe1\xd6\xcb\x20\xf4\x56\x4d\x66\xce\x47\x67\x58\x3d\x0e\x5f\x06\x02\x15\xb5\x90\x17\xbe\x85\xea\x84\x89\x39\x12\x7b\xd8\xc9\xc4\xd4\x7b\x51\x05\x6c\x03\x1c\xf3\x36\xf1\x7c\x99\x80\xf3\xb8\xf5\xb9\xb6\x87\x8e\x8b\x79\x7a\xa4\x3b\x88\x26\x84\x33\x3e\x17\x89\x3f\xe9\xca\xa6\xaa\x29\x9f\x7e\xd1\xa1\x8e\xe2\xc5\x48\x64\xb7\xb2\xb9\x9b\x72\x61\x8f\xb0\x25\x74\xd1\x39\xef\x50\xf0\x19\xc9\xee\xf4\x16\x97\x13\x38\xe7\xd4\x70" + , salt = + "\x71\x0b\x9c\x47\x47\xd8\x00\xd4\xde\x87\xf1\x2a\xfd\xce\x6d\xf1\x81\x07\xcc\x77" + , signature = + "\x66\x60\x26\xfb\xa7\x1b\xd3\xe7\xcf\x13\x15\x7c\xc2\xc5\x1a\x8e\x4a\xa6\x84\xaf\x97\x78\xf9\x18\x49\xf3\x43\x35\xd1\x41\xc0\x01\x54\xc4\x19\x76\x21\xf9\x62\x4a\x67\x5b\x5a\xbc\x22\xee\x7d\x5b\xaa\xff\xaa\xe1\xc9\xba\xca\x2c\xc3\x73\xb3\xf3\x3e\x78\xe6\x14\x3c\x39\x5a\x91\xaa\x7f\xac\xa6\x64\xeb\x73\x3a\xfd\x14\xd8\x82\x72\x59\xd9\x9a\x75\x50\xfa\xca\x50\x1e\xf2\xb0\x4e\x33\xc2\x3a\xa5\x1f\x4b\x9e\x82\x82\xef\xdb\x72\x8c\xc0\xab\x09\x40\x5a\x91\x60\x7c\x63\x69\x96\x1b\xc8\x27\x0d\x2d\x4f\x39\xfc\xe6\x12\xb1" + } + , -- Example 1.4 + VectorPSS + { message = "\xbc\x65\x67\x47\xfa\x9e\xaf\xb3\xf0" + , salt = + "\x05\x6f\x00\x98\x5d\xe1\x4d\x8e\xf5\xce\xa9\xe8\x2f\x8c\x27\xbe\xf7\x20\x33\x5e" + , signature = + "\x46\x09\x79\x3b\x23\xe9\xd0\x93\x62\xdc\x21\xbb\x47\xda\x0b\x4f\x3a\x76\x22\x64\x9a\x47\xd4\x64\x01\x9b\x9a\xea\xfe\x53\x35\x9c\x17\x8c\x91\xcd\x58\xba\x6b\xcb\x78\xbe\x03\x46\xa7\xbc\x63\x7f\x4b\x87\x3d\x4b\xab\x38\xee\x66\x1f\x19\x96\x34\xc5\x47\xa1\xad\x84\x42\xe0\x3d\xa0\x15\xb1\x36\xe5\x43\xf7\xab\x07\xc0\xc1\x3e\x42\x25\xb8\xde\x8c\xce\x25\xd4\xf6\xeb\x84\x00\xf8\x1f\x7e\x18\x33\xb7\xee\x6e\x33\x4d\x37\x09\x64\xca\x79\xfd\xb8\x72\xb4\xd7\x52\x23\xb5\xee\xb0\x81\x01\x59\x1f\xb5\x32\xd1\x55\xa6\xde\x87" + } + , -- Example 1.5 + VectorPSS + { message = + "\xb4\x55\x81\x54\x7e\x54\x27\x77\x0c\x76\x8e\x8b\x82\xb7\x55\x64\xe0\xea\x4e\x9c\x32\x59\x4d\x6b\xff\x70\x65\x44\xde\x0a\x87\x76\xc7\xa8\x0b\x45\x76\x55\x0e\xee\x1b\x2a\xca\xbc\x7e\x8b\x7d\x3e\xf7\xbb\x5b\x03\xe4\x62\xc1\x10\x47\xea\xdd\x00\x62\x9a\xe5\x75\x48\x0a\xc1\x47\x0f\xe0\x46\xf1\x3a\x2b\xf5\xaf\x17\x92\x1d\xc4\xb0\xaa\x8b\x02\xbe\xe6\x33\x49\x11\x65\x1d\x7f\x85\x25\xd1\x0f\x32\xb5\x1d\x33\xbe\x52\x0d\x3d\xdf\x5a\x70\x99\x55\xa3\xdf\xe7\x82\x83\xb9\xe0\xab\x54\x04\x6d\x15\x0c\x17\x7f\x03\x7f\xdc\xcc\x5b\xe4\xea\x5f\x68\xb5\xe5\xa3\x8c\x9d\x7e\xdc\xcc\xc4\x97\x5f\x45\x5a\x69\x09\xb4" + , salt = + "\x80\xe7\x0f\xf8\x6a\x08\xde\x3e\xc6\x09\x72\xb3\x9b\x4f\xbf\xdc\xea\x67\xae\x8e" + , signature = + "\x1d\x2a\xad\x22\x1c\xa4\xd3\x1d\xdf\x13\x50\x92\x39\x01\x93\x98\xe3\xd1\x4b\x32\xdc\x34\xdc\x5a\xf4\xae\xae\xa3\xc0\x95\xaf\x73\x47\x9c\xf0\xa4\x5e\x56\x29\x63\x5a\x53\xa0\x18\x37\x76\x15\xb1\x6c\xb9\xb1\x3b\x3e\x09\xd6\x71\xeb\x71\xe3\x87\xb8\x54\x5c\x59\x60\xda\x5a\x64\x77\x6e\x76\x8e\x82\xb2\xc9\x35\x83\xbf\x10\x4c\x3f\xdb\x23\x51\x2b\x7b\x4e\x89\xf6\x33\xdd\x00\x63\xa5\x30\xdb\x45\x24\xb0\x1c\x3f\x38\x4c\x09\x31\x0e\x31\x5a\x79\xdc\xd3\xd6\x84\x02\x2a\x7f\x31\xc8\x65\xa6\x64\xe3\x16\x97\x8b\x75\x9f\xad" + } + , -- Example 1.6 + VectorPSS + { message = + "\x10\xaa\xe9\xa0\xab\x0b\x59\x5d\x08\x41\x20\x7b\x70\x0d\x48\xd7\x5f\xae\xdd\xe3\xb7\x75\xcd\x6b\x4c\xc8\x8a\xe0\x6e\x46\x94\xec\x74\xba\x18\xf8\x52\x0d\x4f\x5e\xa6\x9c\xbb\xe7\xcc\x2b\xeb\xa4\x3e\xfd\xc1\x02\x15\xac\x4e\xb3\x2d\xc3\x02\xa1\xf5\x3d\xc6\xc4\x35\x22\x67\xe7\x93\x6c\xfe\xbf\x7c\x8d\x67\x03\x57\x84\xa3\x90\x9f\xa8\x59\xc7\xb7\xb5\x9b\x8e\x39\xc5\xc2\x34\x9f\x18\x86\xb7\x05\xa3\x02\x67\xd4\x02\xf7\x48\x6a\xb4\xf5\x8c\xad\x5d\x69\xad\xb1\x7a\xb8\xcd\x0c\xe1\xca\xf5\x02\x5a\xf4\xae\x24\xb1\xfb\x87\x94\xc6\x07\x0c\xc0\x9a\x51\xe2\xf9\x91\x13\x11\xe3\x87\x7d\x00\x44\xc7\x1c\x57\xa9\x93\x39\x50\x08\x80\x6b\x72\x3a\xc3\x83\x73\xd3\x95\x48\x18\x18\x52\x8c\x1e\x70\x53\x73\x92\x82\x05\x35\x29\x51\x0e\x93\x5c\xd0\xfa\x77\xb8\xfa\x53\xcc\x2d\x47\x4b\xd4\xfb\x3c\xc5\xc6\x72\xd6\xff\xdc\x90\xa0\x0f\x98\x48\x71\x2c\x4b\xcf\xe4\x6c\x60\x57\x36\x59\xb1\x1e\x64\x57\xe8\x61\xf0\xf6\x04\xb6\x13\x8d\x14\x4f\x8c\xe4\xe2\xda\x73" + , salt = + "\xa8\xab\x69\xdd\x80\x1f\x00\x74\xc2\xa1\xfc\x60\x64\x98\x36\xc6\x16\xd9\x96\x81" + , signature = + "\x2a\x34\xf6\x12\x5e\x1f\x6b\x0b\xf9\x71\xe8\x4f\xbd\x41\xc6\x32\xbe\x8f\x2c\x2a\xce\x7d\xe8\xb6\x92\x6e\x31\xff\x93\xe9\xaf\x98\x7f\xbc\x06\xe5\x1e\x9b\xe1\x4f\x51\x98\xf9\x1f\x3f\x95\x3b\xd6\x7d\xa6\x0a\x9d\xf5\x97\x64\xc3\xdc\x0f\xe0\x8e\x1c\xbe\xf0\xb7\x5f\x86\x8d\x10\xad\x3f\xba\x74\x9f\xef\x59\xfb\x6d\xac\x46\xa0\xd6\xe5\x04\x36\x93\x31\x58\x6f\x58\xe4\x62\x8f\x39\xaa\x27\x89\x82\x54\x3b\xc0\xee\xb5\x37\xdc\x61\x95\x80\x19\xb3\x94\xfb\x27\x3f\x21\x58\x58\xa0\xa0\x1a\xc4\xd6\x50\xb9\x55\xc6\x7f\x4c\x58" + } + ] + +-- ================================== +-- Example 2: A 1025-bit RSA Key Pair +-- ================================== + +rsaKey2 = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0x01d40c1bcf97a68ae7cdbd8a7bf3e34fa19dcca4ef75a47454375f94514d88fed006fb829f8419ff87d6315da68a1ff3a0938e9abb3464011c303ad99199cf0c7c7a8b477dce829e8844f625b115e5e9c4a59cf8f8113b6834336a2fd2689b472cbb5e5cabe674350c59b6c17e176874fb42f8fc3d176a017edc61fd326c4b33c9 + , public_e = 0x010001 + , public_size = 129 + } + , private_d = + 0x027d147e4673057377fd1ea201565772176a7dc38358d376045685a2e787c23c15576bc16b9f444402d6bfc5d98a3e88ea13ef67c353eca0c0ddba9255bd7b8bb50a644afdfd1dd51695b252d22e7318d1b6687a1c10ff75545f3db0fe602d5f2b7f294e3601eab7b9d1cecd767f64692e3e536ca2846cb0c2dd486a39fa75b1 + , private_p = + 0x016601e926a0f8c9e26ecab769ea65a5e7c52cc9e080ef519457c644da6891c5a104d3ea7955929a22e7c68a7af9fcad777c3ccc2b9e3d3650bce404399b7e59d1 + , private_q = + 0x014eafa1d4d0184da7e31f877d1281ddda625664869e8379e67ad3b75eae74a580e9827abd6eb7a002cb5411f5266797768fb8e95ae40e3e8a01f35ff89e56c079 + , private_dP = + 0xe247cce504939b8f0a36090de200938755e2444b29539a7da7a902f6056835c0db7b52559497cfe2c61a8086d0213c472c78851800b171f6401de2e9c2756f31 + , private_dQ = + 0xb12fba757855e586e46f64c38a70c68b3f548d93d787b399999d4c8f0bbd2581c21e19ed0018a6d5d3df86424b3abcad40199d31495b61309f27c1bf55d487c1 + , private_qinv = + 0x564b1e1fa003bda91e89090425aac05b91da9ee25061e7628d5f51304a84992fdc33762bd378a59f030a334d532bd0dae8f298ea9ed844636ad5fb8cbdc03cad + } + +vectorsKey2 = + [ -- Example 2.1 + VectorPSS + { message = + "\xda\xba\x03\x20\x66\x26\x3f\xae\xdb\x65\x98\x48\x11\x52\x78\xa5\x2c\x44\xfa\xa3\xa7\x6f\x37\x51\x5e\xd3\x36\x32\x10\x72\xc4\x0a\x9d\x9b\x53\xbc\x05\x01\x40\x78\xad\xf5\x20\x87\x51\x46\xaa\xe7\x0f\xf0\x60\x22\x6d\xcb\x7b\x1f\x1f\xc2\x7e\x93\x60" + , salt = + "\x57\xbf\x16\x0b\xcb\x02\xbb\x1d\xc7\x28\x0c\xf0\x45\x85\x30\xb7\xd2\x83\x2f\xf7" + , signature = + "\x01\x4c\x5b\xa5\x33\x83\x28\xcc\xc6\xe7\xa9\x0b\xf1\xc0\xab\x3f\xd6\x06\xff\x47\x96\xd3\xc1\x2e\x4b\x63\x9e\xd9\x13\x6a\x5f\xec\x6c\x16\xd8\x88\x4b\xdd\x99\xcf\xdc\x52\x14\x56\xb0\x74\x2b\x73\x68\x68\xcf\x90\xde\x09\x9a\xdb\x8d\x5f\xfd\x1d\xef\xf3\x9b\xa4\x00\x7a\xb7\x46\xce\xfd\xb2\x2d\x7d\xf0\xe2\x25\xf5\x46\x27\xdc\x65\x46\x61\x31\x72\x1b\x90\xaf\x44\x53\x63\xa8\x35\x8b\x9f\x60\x76\x42\xf7\x8f\xab\x0a\xb0\xf4\x3b\x71\x68\xd6\x4b\xae\x70\xd8\x82\x78\x48\xd8\xef\x1e\x42\x1c\x57\x54\xdd\xf4\x2c\x25\x89\xb5\xb3" + } + , -- Example 2.2 + VectorPSS + { message = + "\xe4\xf8\x60\x1a\x8a\x6d\xa1\xbe\x34\x44\x7c\x09\x59\xc0\x58\x57\x0c\x36\x68\xcf\xd5\x1d\xd5\xf9\xcc\xd6\xad\x44\x11\xfe\x82\x13\x48\x6d\x78\xa6\xc4\x9f\x93\xef\xc2\xca\x22\x88\xce\xbc\x2b\x9b\x60\xbd\x04\xb1\xe2\x20\xd8\x6e\x3d\x48\x48\xd7\x09\xd0\x32\xd1\xe8\xc6\xa0\x70\xc6\xaf\x9a\x49\x9f\xcf\x95\x35\x4b\x14\xba\x61\x27\xc7\x39\xde\x1b\xb0\xfd\x16\x43\x1e\x46\x93\x8a\xec\x0c\xf8\xad\x9e\xb7\x2e\x83\x2a\x70\x35\xde\x9b\x78\x07\xbd\xc0\xed\x8b\x68\xeb\x0f\x5a\xc2\x21\x6b\xe4\x0c\xe9\x20\xc0\xdb\x0e\xdd\xd3\x86\x0e\xd7\x88\xef\xac\xca\xca\x50\x2d\x8f\x2b\xd6\xd1\xa7\xc1\xf4\x1f\xf4\x6f\x16\x81\xc8\xf1\xf8\x18\xe9\xc4\xf6\xd9\x1a\x0c\x78\x03\xcc\xc6\x3d\x76\xa6\x54\x4d\x84\x3e\x08\x4e\x36\x3b\x8a\xcc\x55\xaa\x53\x17\x33\xed\xb5\xde\xe5\xb5\x19\x6e\x9f\x03\xe8\xb7\x31\xb3\x77\x64\x28\xd9\xe4\x57\xfe\x3f\xbc\xb3\xdb\x72\x74\x44\x2d\x78\x58\x90\xe9\xcb\x08\x54\xb6\x44\x4d\xac\xe7\x91\xd7\x27\x3d\xe1\x88\x97\x19\x33\x8a\x77\xfe" + , salt = + "\x7f\x6d\xd3\x59\xe6\x04\xe6\x08\x70\xe8\x98\xe4\x7b\x19\xbf\x2e\x5a\x7b\x2a\x90" + , signature = + "\x01\x09\x91\x65\x6c\xca\x18\x2b\x7f\x29\xd2\xdb\xc0\x07\xe7\xae\x0f\xec\x15\x8e\xb6\x75\x9c\xb9\xc4\x5c\x5f\xf8\x7c\x76\x35\xdd\x46\xd1\x50\x88\x2f\x4d\xe1\xe9\xae\x65\xe7\xf7\xd9\x01\x8f\x68\x36\x95\x4a\x47\xc0\xa8\x1a\x8a\x6b\x6f\x83\xf2\x94\x4d\x60\x81\xb1\xaa\x7c\x75\x9b\x25\x4b\x2c\x34\xb6\x91\xda\x67\xcc\x02\x26\xe2\x0b\x2f\x18\xb4\x22\x12\x76\x1d\xcd\x4b\x90\x8a\x62\xb3\x71\xb5\x91\x8c\x57\x42\xaf\x4b\x53\x7e\x29\x69\x17\x67\x4f\xb9\x14\x19\x47\x61\x62\x1c\xc1\x9a\x41\xf6\xfb\x95\x3f\xbc\xbb\x64\x9d\xea" + } + , -- Example 2.3 + VectorPSS + { message = + "\x52\xa1\xd9\x6c\x8a\xc3\x9e\x41\xe4\x55\x80\x98\x01\xb9\x27\xa5\xb4\x45\xc1\x0d\x90\x2a\x0d\xcd\x38\x50\xd2\x2a\x66\xd2\xbb\x07\x03\xe6\x7d\x58\x67\x11\x45\x95\xaa\xbf\x5a\x7a\xeb\x5a\x8f\x87\x03\x4b\xbb\x30\xe1\x3c\xfd\x48\x17\xa9\xbe\x76\x23\x00\x23\x60\x6d\x02\x86\xa3\xfa\xf8\xa4\xd2\x2b\x72\x8e\xc5\x18\x07\x9f\x9e\x64\x52\x6e\x3a\x0c\xc7\x94\x1a\xa3\x38\xc4\x37\x99\x7c\x68\x0c\xca\xc6\x7c\x66\xbf\xa1" + , salt = + "\xfc\xa8\x62\x06\x8b\xce\x22\x46\x72\x4b\x70\x8a\x05\x19\xda\x17\xe6\x48\x68\x8c" + , signature = + "\x00\x7f\x00\x30\x01\x8f\x53\xcd\xc7\x1f\x23\xd0\x36\x59\xfd\xe5\x4d\x42\x41\xf7\x58\xa7\x50\xb4\x2f\x18\x5f\x87\x57\x85\x20\xc3\x07\x42\xaf\xd8\x43\x59\xb6\xe6\xe8\xd3\xed\x95\x9d\xc6\xfe\x48\x6b\xed\xc8\xe2\xcf\x00\x1f\x63\xa7\xab\xe1\x62\x56\xa1\xb8\x4d\xf0\xd2\x49\xfc\x05\xd3\x19\x4c\xe5\xf0\x91\x27\x42\xdb\xbf\x80\xdd\x17\x4f\x6c\x51\xf6\xba\xd7\xf1\x6c\xf3\x36\x4e\xba\x09\x5a\x06\x26\x7d\xc3\x79\x38\x03\xac\x75\x26\xae\xbe\x0a\x47\x5d\x38\xb8\xc2\x24\x7a\xb5\x1c\x48\x98\xdf\x70\x47\xdc\x6a\xdf\x52\xc6\xc4" + } + , -- Example 2.4 + VectorPSS + { message = + "\xa7\x18\x2c\x83\xac\x18\xbe\x65\x70\xa1\x06\xaa\x9d\x5c\x4e\x3d\xbb\xd4\xaf\xae\xb0\xc6\x0c\x4a\x23\xe1\x96\x9d\x79\xff" + , salt = + "\x80\x70\xef\x2d\xe9\x45\xc0\x23\x87\x68\x4b\xa0\xd3\x30\x96\x73\x22\x35\xd4\x40" + , signature = + "\x00\x9c\xd2\xf4\xed\xbe\x23\xe1\x23\x46\xae\x8c\x76\xdd\x9a\xd3\x23\x0a\x62\x07\x61\x41\xf1\x6c\x15\x2b\xa1\x85\x13\xa4\x8e\xf6\xf0\x10\xe0\xe3\x7f\xd3\xdf\x10\xa1\xec\x62\x9a\x0c\xb5\xa3\xb5\xd2\x89\x30\x07\x29\x8c\x30\x93\x6a\x95\x90\x3b\x6b\xa8\x55\x55\xd9\xec\x36\x73\xa0\x61\x08\xfd\x62\xa2\xfd\xa5\x6d\x1c\xe2\xe8\x5c\x4d\xb6\xb2\x4a\x81\xca\x3b\x49\x6c\x36\xd4\xfd\x06\xeb\x7c\x91\x66\xd8\xe9\x48\x77\xc4\x2b\xea\x62\x2b\x3b\xfe\x92\x51\xfd\xc2\x1d\x8d\x53\x71\xba\xda\xd7\x8a\x48\x82\x14\x79\x63\x35\xb4\x0b" + } + , -- Example 2.5 + VectorPSS + { message = + "\x86\xa8\x3d\x4a\x72\xee\x93\x2a\x4f\x56\x30\xaf\x65\x79\xa3\x86\xb7\x8f\xe8\x89\x99\xe0\xab\xd2\xd4\x90\x34\xa4\xbf\xc8\x54\xdd\x94\xf1\x09\x4e\x2e\x8c\xd7\xa1\x79\xd1\x95\x88\xe4\xae\xfc\x1b\x1b\xd2\x5e\x95\xe3\xdd\x46\x1f" + , salt = + "\x17\x63\x9a\x4e\x88\xd7\x22\xc4\xfc\xa2\x4d\x07\x9a\x8b\x29\xc3\x24\x33\xb0\xc9" + , signature = + "\x00\xec\x43\x08\x24\x93\x1e\xbd\x3b\xaa\x43\x03\x4d\xae\x98\xba\x64\x6b\x8c\x36\x01\x3d\x16\x71\xc3\xcf\x1c\xf8\x26\x0c\x37\x4b\x19\xf8\xe1\xcc\x8d\x96\x50\x12\x40\x5e\x7e\x9b\xf7\x37\x86\x12\xdf\xcc\x85\xfc\xe1\x2c\xda\x11\xf9\x50\xbd\x0b\xa8\x87\x67\x40\x43\x6c\x1d\x25\x95\xa6\x4a\x1b\x32\xef\xcf\xb7\x4a\x21\xc8\x73\xb3\xcc\x33\xaa\xf4\xe3\xdc\x39\x53\xde\x67\xf0\x67\x4c\x04\x53\xb4\xfd\x9f\x60\x44\x06\xd4\x41\xb8\x16\x09\x8c\xb1\x06\xfe\x34\x72\xbc\x25\x1f\x81\x5f\x59\xdb\x2e\x43\x78\xa3\xad\xdc\x18\x1e\xcf" + } + , -- Example 2.6 + VectorPSS + { message = + "\x04\x9f\x91\x54\xd8\x71\xac\x4a\x7c\x7a\xb4\x53\x25\xba\x75\x45\xa1\xed\x08\xf7\x05\x25\xb2\x66\x7c\xf1" + , salt = + "\x37\x81\x0d\xef\x10\x55\xed\x92\x2b\x06\x3d\xf7\x98\xde\x5d\x0a\xab\xf8\x86\xee" + , signature = + "\x00\x47\x5b\x16\x48\xf8\x14\xa8\xdc\x0a\xbd\xc3\x7b\x55\x27\xf5\x43\xb6\x66\xbb\x6e\x39\xd3\x0e\x5b\x49\xd3\xb8\x76\xdc\xcc\x58\xea\xc1\x4e\x32\xa2\xd5\x5c\x26\x16\x01\x44\x56\xad\x2f\x24\x6f\xc8\xe3\xd5\x60\xda\x3d\xdf\x37\x9a\x1c\x0b\xd2\x00\xf1\x02\x21\xdf\x07\x8c\x21\x9a\x15\x1b\xc8\xd4\xec\x9d\x2f\xc2\x56\x44\x67\x81\x10\x14\xef\x15\xd8\xea\x01\xc2\xeb\xbf\xf8\xc2\xc8\xef\xab\x38\x09\x6e\x55\xfc\xbe\x32\x85\xc7\xaa\x55\x88\x51\x25\x4f\xaf\xfa\x92\xc1\xc7\x2b\x78\x75\x86\x63\xef\x45\x82\x84\x31\x39\xd7\xa6" + } + ] + +-- ================================== +-- Example 3: A 1026-bit RSA Key Pair +-- ================================== + +rsaKey3 = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0x02f246ef451ed3eebb9a310200cc25859c048e4be798302991112eb68ce6db674e280da21feded1ae74880ca522b18db249385012827c515f0e466a1ffa691d98170574e9d0eadb087586ca48933da3cc953d95bd0ed50de10ddcb6736107d6c831c7f663e833ca4c097e700ce0fb945f88fb85fe8e5a773172565b914a471a443 + , public_e = 0x010001 + , public_size = 129 + } + , private_d = + 0x651451733b56de5ac0a689a4aeb6e6894a69014e076c88dd7a667eab3232bbccd2fc44ba2fa9c31db46f21edd1fdb23c5c128a5da5bab91e7f952b67759c7cff705415ac9fa0907c7ca6178f668fb948d869da4cc3b7356f4008dfd5449d32ee02d9a477eb69fc29266e5d9070512375a50fbbcc27e238ad98425f6ebbf88991 + , private_p = + 0x01bd36e18ece4b0fdb2e9c9d548bd1a7d6e2c21c6fdc35074a1d05b1c6c8b3d558ea2639c9a9a421680169317252558bd148ad215aac550e2dcf12a82d0ebfe853 + , private_q = + 0x01b1b656ad86d8e19d5dc86292b3a192fdf6e0dd37877bad14822fa00190cab265f90d3f02057b6f54d6ecb14491e5adeacebc48bf0ebd2a2ad26d402e54f61651 + , private_dP = + 0x1f2779fd2e3e5e6bae05539518fba0cd0ead1aa4513a7cba18f1cf10e3f68195693d278a0f0ee72f89f9bc760d80e2f9d0261d516501c6ae39f14a476ce2ccf5 + , private_dQ = + 0x011a0d36794b04a854aab4b2462d439a5046c91d940b2bc6f75b62956fef35a2a6e63c5309817f307bbff9d59e7e331bd363f6d66849b18346adea169f0ae9aec1 + , private_qinv = + 0x0b30f0ecf558752fb3a6ce4ba2b8c675f659eba6c376585a1b39712d038ae3d2b46fcb418ae15d0905da6440e1513a30b9b7d6668fbc5e88e5ab7a175e73ba35 + } + +vectorsKey3 = + [ -- Example 3.1 + VectorPSS + { message = + "\x59\x4b\x37\x33\x3b\xbb\x2c\x84\x52\x4a\x87\xc1\xa0\x1f\x75\xfc\xec\x0e\x32\x56\xf1\x08\xe3\x8d\xca\x36\xd7\x0d\x00\x57" + , salt = + "\xf3\x1a\xd6\xc8\xcf\x89\xdf\x78\xed\x77\xfe\xac\xbc\xc2\xf8\xb0\xa8\xe4\xcf\xaa" + , signature = + "\x00\x88\xb1\x35\xfb\x17\x94\xb6\xb9\x6c\x4a\x3e\x67\x81\x97\xf8\xca\xc5\x2b\x64\xb2\xfe\x90\x7d\x6f\x27\xde\x76\x11\x24\x96\x4a\x99\xa0\x1a\x88\x27\x40\xec\xfa\xed\x6c\x01\xa4\x74\x64\xbb\x05\x18\x23\x13\xc0\x13\x38\xa8\xcd\x09\x72\x14\xcd\x68\xca\x10\x3b\xd5\x7d\x3b\xc9\xe8\x16\x21\x3e\x61\xd7\x84\xf1\x82\x46\x7a\xbf\x8a\x01\xcf\x25\x3e\x99\xa1\x56\xea\xa8\xe3\xe1\xf9\x0e\x3c\x6e\x4e\x3a\xa2\xd8\x3e\xd0\x34\x5b\x89\xfa\xfc\x9c\x26\x07\x7c\x14\xb6\xac\x51\x45\x4f\xa2\x6e\x44\x6e\x3a\x2f\x15\x3b\x2b\x16\x79\x7f" + } + , -- Example 3.2 + VectorPSS + { message = + "\x8b\x76\x95\x28\x88\x4a\x0d\x1f\xfd\x09\x0c\xf1\x02\x99\x3e\x79\x6d\xad\xcf\xbd\xdd\x38\xe4\x4f\xf6\x32\x4c\xa4\x51" + , salt = + "\xfc\xf9\xf0\xe1\xf1\x99\xa3\xd1\xd0\xda\x68\x1c\x5b\x86\x06\xfc\x64\x29\x39\xf7" + , signature = + "\x02\xa5\xf0\xa8\x58\xa0\x86\x4a\x4f\x65\x01\x7a\x7d\x69\x45\x4f\x3f\x97\x3a\x29\x99\x83\x9b\x7b\xbc\x48\xbf\x78\x64\x11\x69\x17\x95\x56\xf5\x95\xfa\x41\xf6\xff\x18\xe2\x86\xc2\x78\x30\x79\xbc\x09\x10\xee\x9c\xc3\x4f\x49\xba\x68\x11\x24\xf9\x23\xdf\xa8\x8f\x42\x61\x41\xa3\x68\xa5\xf5\xa9\x30\xc6\x28\xc2\xc3\xc2\x00\xe1\x8a\x76\x44\x72\x1a\x0c\xbe\xc6\xdd\x3f\x62\x79\xbd\xe3\xe8\xf2\xbe\x5e\x2d\x4e\xe5\x6f\x97\xe7\xce\xaf\x33\x05\x4b\xe7\x04\x2b\xd9\x1a\x63\xbb\x09\xf8\x97\xbd\x41\xe8\x11\x97\xde\xe9\x9b\x11\xaf" + } + , -- Example 3.3 + VectorPSS + { message = + "\x1a\xbd\xba\x48\x9c\x5a\xda\x2f\x99\x5e\xd1\x6f\x19\xd5\xa9\x4d\x9e\x6e\xc3\x4a\x8d\x84\xf8\x45\x57\xd2\x6e\x5e\xf9\xb0\x2b\x22\x88\x7e\x3f\x9a\x4b\x69\x0a\xd1\x14\x92\x09\xc2\x0c\x61\x43\x1f\x0c\x01\x7c\x36\xc2\x65\x7b\x35\xd7\xb0\x7d\x3f\x5a\xd8\x70\x85\x07\xa9\xc1\xb8\x31\xdf\x83\x5a\x56\xf8\x31\x07\x18\x14\xea\x5d\x3d\x8d\x8f\x6a\xde\x40\xcb\xa3\x8b\x42\xdb\x7a\x2d\x3d\x7a\x29\xc8\xf0\xa7\x9a\x78\x38\xcf\x58\xa9\x75\x7f\xa2\xfe\x4c\x40\xdf\x9b\xaa\x19\x3b\xfc\x6f\x92\xb1\x23\xad\x57\xb0\x7a\xce\x3e\x6a\xc0\x68\xc9\xf1\x06\xaf\xd9\xee\xb0\x3b\x4f\x37\xc2\x5d\xbf\xbc\xfb\x30\x71\xf6\xf9\x77\x17\x66\xd0\x72\xf3\xbb\x07\x0a\xf6\x60\x55\x32\x97\x3a\xe2\x50\x51" + , salt = + "\x98\x6e\x7c\x43\xdb\xb6\x71\xbd\x41\xb9\xa7\xf4\xb6\xaf\xc8\x0e\x80\x5f\x24\x23" + , signature = + "\x02\x44\xbc\xd1\xc8\xc1\x69\x55\x73\x6c\x80\x3b\xe4\x01\x27\x2e\x18\xcb\x99\x08\x11\xb1\x4f\x72\xdb\x96\x41\x24\xd5\xfa\x76\x06\x49\xcb\xb5\x7a\xfb\x87\x55\xdb\xb6\x2b\xf5\x1f\x46\x6c\xf2\x3a\x0a\x16\x07\x57\x6e\x98\x3d\x77\x8f\xce\xff\xa9\x2d\xf7\x54\x8a\xea\x8e\xa4\xec\xad\x2c\x29\xdd\x9f\x95\xbc\x07\xfe\x91\xec\xf8\xbe\xe2\x55\xbf\xe8\x76\x2f\xd7\x69\x0a\xa9\xbf\xa4\xfa\x08\x49\xef\x72\x8c\x2c\x42\xc4\x53\x23\x64\x52\x2d\xf2\xab\x7f\x9f\x8a\x03\xb6\x3f\x7a\x49\x91\x75\x82\x86\x68\xf5\xef\x5a\x29\xe3\x80\x2c" + } + , -- Example 3.4 + VectorPSS + { message = + "\x8f\xb4\x31\xf5\xee\x79\x2b\x6c\x2a\xc7\xdb\x53\xcc\x42\x86\x55\xae\xb3\x2d\x03\xf4\xe8\x89\xc5\xc2\x5d\xe6\x83\xc4\x61\xb5\x3a\xcf\x89\xf9\xf8\xd3\xaa\xbd\xf6\xb9\xf0\xc2\xa1\xde\x12\xe1\x5b\x49\xed\xb3\x91\x9a\x65\x2f\xe9\x49\x1c\x25\xa7\xfc\xe1\xf7\x22\xc2\x54\x36\x08\xb6\x9d\xc3\x75\xec" + , salt = + "\xf8\x31\x2d\x9c\x8e\xea\x13\xec\x0a\x4c\x7b\x98\x12\x0c\x87\x50\x90\x87\xc4\x78" + , signature = + "\x01\x96\xf1\x2a\x00\x5b\x98\x12\x9c\x8d\xf1\x3c\x4c\xb1\x6f\x8a\xa8\x87\xd3\xc4\x0d\x96\xdf\x3a\x88\xe7\x53\x2e\xf3\x9c\xd9\x92\xf2\x73\xab\xc3\x70\xbc\x1b\xe6\xf0\x97\xcf\xeb\xbf\x01\x18\xfd\x9e\xf4\xb9\x27\x15\x5f\x3d\xf2\x2b\x90\x4d\x90\x70\x2d\x1f\x7b\xa7\xa5\x2b\xed\x8b\x89\x42\xf4\x12\xcd\x7b\xd6\x76\xc9\xd1\x8e\x17\x03\x91\xdc\xd3\x45\xc0\x6a\x73\x09\x64\xb3\xf3\x0b\xcc\xe0\xbb\x20\xba\x10\x6f\x9a\xb0\xee\xb3\x9c\xf8\xa6\x60\x7f\x75\xc0\x34\x7f\x0a\xf7\x9f\x16\xaf\xa0\x81\xd2\xc9\x2d\x1e\xe6\xf8\x36\xb8" + } + , -- Example 3.5 + VectorPSS + { message = + "\xfe\xf4\x16\x1d\xfa\xaf\x9c\x52\x95\x05\x1d\xfc\x1f\xf3\x81\x0c\x8c\x9e\xc2\xe8\x66\xf7\x07\x54\x22\xc8\xec\x42\x16\xa9\xc4\xff\x49\x42\x7d\x48\x3c\xae\x10\xc8\x53\x4a\x41\xb2\xfd\x15\xfe\xe0\x69\x60\xec\x6f\xb3\xf7\xa7\xe9\x4a\x2f\x8a\x2e\x3e\x43\xdc\x4a\x40\x57\x6c\x30\x97\xac\x95\x3b\x1d\xe8\x6f\x0b\x4e\xd3\x6d\x64\x4f\x23\xae\x14\x42\x55\x29\x62\x24\x64\xca\x0c\xbf\x0b\x17\x41\x34\x72\x38\x15\x7f\xab\x59\xe4\xde\x55\x24\x09\x6d\x62\xba\xec\x63\xac\x64" + , salt = + "\x50\x32\x7e\xfe\xc6\x29\x2f\x98\x01\x9f\xc6\x7a\x2a\x66\x38\x56\x3e\x9b\x6e\x2d" + , signature = + "\x02\x1e\xca\x3a\xb4\x89\x22\x64\xec\x22\x41\x1a\x75\x2d\x92\x22\x10\x76\xd4\xe0\x1c\x0e\x6f\x0d\xde\x9a\xfd\x26\xba\x5a\xcf\x6d\x73\x9e\xf9\x87\x54\x5d\x16\x68\x3e\x56\x74\xc9\xe7\x0f\x1d\xe6\x49\xd7\xe6\x1d\x48\xd0\xca\xeb\x4f\xb4\xd8\xb2\x4f\xba\x84\xa6\xe3\x10\x8f\xee\x7d\x07\x05\x97\x32\x66\xac\x52\x4b\x4a\xd2\x80\xf7\xae\x17\xdc\x59\xd9\x6d\x33\x51\x58\x6b\x5a\x3b\xdb\x89\x5d\x1e\x1f\x78\x20\xac\x61\x35\xd8\x75\x34\x80\x99\x83\x82\xba\x32\xb7\x34\x95\x59\x60\x8c\x38\x74\x52\x90\xa8\x5e\xf4\xe9\xf9\xbd\x83" + } + , -- Example 3.6 + VectorPSS + { message = + "\xef\xd2\x37\xbb\x09\x8a\x44\x3a\xee\xb2\xbf\x6c\x3f\x8c\x81\xb8\xc0\x1b\x7f\xcb\x3f\xeb" + , salt = + "\xb0\xde\x3f\xc2\x5b\x65\xf5\xaf\x96\xb1\xd5\xcc\x3b\x27\xd0\xc6\x05\x30\x87\xb3" + , signature = + "\x01\x2f\xaf\xec\x86\x2f\x56\xe9\xe9\x2f\x60\xab\x0c\x77\x82\x4f\x42\x99\xa0\xca\x73\x4e\xd2\x6e\x06\x44\xd5\xd2\x22\xc7\xf0\xbd\xe0\x39\x64\xf8\xe7\x0a\x5c\xb6\x5e\xd4\x4e\x44\xd5\x6a\xe0\xed\xf1\xff\x86\xca\x03\x2c\xc5\xdd\x44\x04\xdb\xb7\x6a\xb8\x54\x58\x6c\x44\xee\xd8\x33\x6d\x08\xd4\x57\xce\x6c\x03\x69\x3b\x45\xc0\xf1\xef\xef\x93\x62\x4b\x95\xb8\xec\x16\x9c\x61\x6d\x20\xe5\x53\x8e\xbc\x0b\x67\x37\xa6\xf8\x2b\x4b\xc0\x57\x09\x24\xfc\x6b\x35\x75\x9a\x33\x48\x42\x62\x79\xf8\xb3\xd7\x74\x4e\x2d\x22\x24\x26\xce" + } + ] + +-- ================================== +-- Example 8: A 1031-bit RSA Key Pair +-- ================================== + +rsaKey8 = + PrivateKey + { private_pub = + PublicKey + { public_n = + 0x495370a1fb18543c16d3631e3163255df62be6eee890d5f25509e4f778a8ea6fbbbcdf85dff64e0d972003ab3681fbba6dd41fd541829b2e582de9f2a4a4e0a2d0900bef4753db3cee0ee06c7dfae8b1d53b5953218f9cceea695b08668edeaadced9463b1d790d5ebf27e9115b46cad4d9a2b8efab0561b0810344739ada0733f + , public_e = 0x010001 + , public_size = 129 + } + , private_d = + 0x6c66ffe98980c38fcdeab5159898836165f4b4b817c4f6a8d486ee4ea9130fe9b9092bd136d184f95f504a607eac565846d2fdd6597a8967c7396ef95a6eeebb4578a643966dca4d8ee3de842de63279c618159c1ab54a89437b6a6120e4930afb52a4ba6ced8a4947ac64b30a3497cbe701c2d6266d517219ad0ec6d347dbe9 + , private_p = + 0x08dad7f11363faa623d5d6d5e8a319328d82190d7127d2846c439b0ab72619b0a43a95320e4ec34fc3a9cea876422305bd76c5ba7be9e2f410c8060645a1d29edb + , private_q = + 0x0847e732376fc7900f898ea82eb2b0fc418565fdae62f7d9ec4ce2217b97990dd272db157f99f63c0dcbb9fbacdbd4c4dadb6df67756358ca4174825b48f49706d + , private_dP = + 0x05c2a83c124b3621a2aa57ea2c3efe035eff4560f33ddebb7adab81fce69a0c8c2edc16520dda83d59a23be867963ac65f2cc710bbcfb96ee103deb771d105fd85 + , private_dQ = + 0x04cae8aa0d9faa165c87b682ec140b8ed3b50b24594b7a3b2c220b3669bb819f984f55310a1ae7823651d4a02e99447972595139363434e5e30a7e7d241551e1b9 + , private_qinv = + 0x07d3e47bf686600b11ac283ce88dbb3f6051e8efd04680e44c171ef531b80b2b7c39fc766320e2cf15d8d99820e96ff30dc69691839c4b40d7b06e45307dc91f3f + } + +vectorsKey8 = + [ -- Example 8.1 + VectorPSS + { message = + "\x81\x33\x2f\x4b\xe6\x29\x48\x41\x5e\xa1\xd8\x99\x79\x2e\xea\xcf\x6c\x6e\x1d\xb1\xda\x8b\xe1\x3b\x5c\xea\x41\xdb\x2f\xed\x46\x70\x92\xe1\xff\x39\x89\x14\xc7\x14\x25\x97\x75\xf5\x95\xf8\x54\x7f\x73\x56\x92\xa5\x75\xe6\x92\x3a\xf7\x8f\x22\xc6\x99\x7d\xdb\x90\xfb\x6f\x72\xd7\xbb\x0d\xd5\x74\x4a\x31\xde\xcd\x3d\xc3\x68\x58\x49\x83\x6e\xd3\x4a\xec\x59\x63\x04\xad\x11\x84\x3c\x4f\x88\x48\x9f\x20\x97\x35\xf5\xfb\x7f\xda\xf7\xce\xc8\xad\xdc\x58\x18\x16\x8f\x88\x0a\xcb\xf4\x90\xd5\x10\x05\xb7\xa8\xe8\x4e\x43\xe5\x42\x87\x97\x75\x71\xdd\x99\xee\xa4\xb1\x61\xeb\x2d\xf1\xf5\x10\x8f\x12\xa4\x14\x2a\x83\x32\x2e\xdb\x05\xa7\x54\x87\xa3\x43\x5c\x9a\x78\xce\x53\xed\x93\xbc\x55\x08\x57\xd7\xa9\xfb" + , salt = + "\x1d\x65\x49\x1d\x79\xc8\x64\xb3\x73\x00\x9b\xe6\xf6\xf2\x46\x7b\xac\x4c\x78\xfa" + , signature = + "\x02\x62\xac\x25\x4b\xfa\x77\xf3\xc1\xac\xa2\x2c\x51\x79\xf8\xf0\x40\x42\x2b\x3c\x5b\xaf\xd4\x0a\x8f\x21\xcf\x0f\xa5\xa6\x67\xcc\xd5\x99\x3d\x42\xdb\xaf\xb4\x09\xc5\x20\xe2\x5f\xce\x2b\x1e\xe1\xe7\x16\x57\x7f\x1e\xfa\x17\xf3\xda\x28\x05\x2f\x40\xf0\x41\x9b\x23\x10\x6d\x78\x45\xaa\xf0\x11\x25\xb6\x98\xe7\xa4\xdf\xe9\x2d\x39\x67\xbb\x00\xc4\xd0\xd3\x5b\xa3\x55\x2a\xb9\xa8\xb3\xee\xf0\x7c\x7f\xec\xdb\xc5\x42\x4a\xc4\xdb\x1e\x20\xcb\x37\xd0\xb2\x74\x47\x69\x94\x0e\xa9\x07\xe1\x7f\xbb\xca\x67\x3b\x20\x52\x23\x80\xc5" + } + , -- Example 8.2 + VectorPSS + { message = + "\xe2\xf9\x6e\xaf\x0e\x05\xe7\xba\x32\x6e\xcc\xa0\xba\x7f\xd2\xf7\xc0\x23\x56\xf3\xce\xde\x9d\x0f\xaa\xbf\x4f\xcc\x8e\x60\xa9\x73\xe5\x59\x5f\xd9\xea\x08" + , salt = + "\x43\x5c\x09\x8a\xa9\x90\x9e\xb2\x37\x7f\x12\x48\xb0\x91\xb6\x89\x87\xff\x18\x38" + , signature = + "\x27\x07\xb9\xad\x51\x15\xc5\x8c\x94\xe9\x32\xe8\xec\x0a\x28\x0f\x56\x33\x9e\x44\xa1\xb5\x8d\x4d\xdc\xff\x2f\x31\x2e\x5f\x34\xdc\xfe\x39\xe8\x9c\x6a\x94\xdc\xee\x86\xdb\xbd\xae\x5b\x79\xba\x4e\x08\x19\xa9\xe7\xbf\xd9\xd9\x82\xe7\xee\x6c\x86\xee\x68\x39\x6e\x8b\x3a\x14\xc9\xc8\xf3\x4b\x17\x8e\xb7\x41\xf9\xd3\xf1\x21\x10\x9b\xf5\xc8\x17\x2f\xad\xa2\xe7\x68\xf9\xea\x14\x33\x03\x2c\x00\x4a\x8a\xa0\x7e\xb9\x90\x00\x0a\x48\xdc\x94\xc8\xba\xc8\xaa\xbe\x2b\x09\xb1\xaa\x46\xc0\xa2\xaa\x0e\x12\xf6\x3f\xbb\xa7\x75\xba\x7e" + } + , -- Example 8.3 + VectorPSS + { message = + "\xe3\x5c\x6e\xd9\x8f\x64\xa6\xd5\xa6\x48\xfc\xab\x8a\xdb\x16\x33\x1d\xb3\x2e\x5d\x15\xc7\x4a\x40\xed\xf9\x4c\x3d\xc4\xa4\xde\x79\x2d\x19\x08\x89\xf2\x0f\x1e\x24\xed\x12\x05\x4a\x6b\x28\x79\x8f\xcb\x42\xd1\xc5\x48\x76\x9b\x73\x4c\x96\x37\x31\x42\x09\x2a\xed\x27\x76\x03\xf4\x73\x8d\xf4\xdc\x14\x46\x58\x6d\x0e\xc6\x4d\xa4\xfb\x60\x53\x6d\xb2\xae\x17\xfc\x7e\x3c\x04\xbb\xfb\xbb\xd9\x07\xbf\x11\x7c\x08\x63\x6f\xa1\x6f\x95\xf5\x1a\x62\x16\x93\x4d\x3e\x34\xf8\x50\x30\xf1\x7b\xbb\xc5\xba\x69\x14\x40\x58\xaf\xf0\x81\xe0\xb1\x9c\xf0\x3c\x17\x19\x5c\x5e\x88\x8b\xa5\x8f\x6f\xe0\xa0\x2e\x5c\x3b\xda\x97\x19\xa7" + , salt = + "\xc6\xeb\xbe\x76\xdf\x0c\x4a\xea\x32\xc4\x74\x17\x5b\x2f\x13\x68\x62\xd0\x45\x29" + , signature = + "\x2a\xd2\x05\x09\xd7\x8c\xf2\x6d\x1b\x6c\x40\x61\x46\x08\x6e\x4b\x0c\x91\xa9\x1c\x2b\xd1\x64\xc8\x7b\x96\x6b\x8f\xaa\x42\xaa\x0c\xa4\x46\x02\x23\x23\xba\x4b\x1a\x1b\x89\x70\x6d\x7f\x4c\x3b\xe5\x7d\x7b\x69\x70\x2d\x16\x8a\xb5\x95\x5e\xe2\x90\x35\x6b\x8c\x4a\x29\xed\x46\x7d\x54\x7e\xc2\x3c\xba\xdf\x28\x6c\xcb\x58\x63\xc6\x67\x9d\xa4\x67\xfc\x93\x24\xa1\x51\xc7\xec\x55\xaa\xc6\xdb\x40\x84\xf8\x27\x26\x82\x5c\xfe\x1a\xa4\x21\xbc\x64\x04\x9f\xb4\x2f\x23\x14\x8f\x9c\x25\xb2\xdc\x30\x04\x37\xc3\x8d\x42\x8a\xa7\x5f\x96" + } + , -- Example 8.4 + VectorPSS + { message = + "\xdb\xc5\xf7\x50\xa7\xa1\x4b\xe2\xb9\x3e\x83\x8d\x18\xd1\x4a\x86\x95\xe5\x2e\x8a\xdd\x9c\x0a\xc7\x33\xb8\xf5\x6d\x27\x47\xe5\x29\xa0\xcc\xa5\x32\xdd\x49\xb9\x02\xae\xfe\xd5\x14\x44\x7f\x9e\x81\xd1\x61\x95\xc2\x85\x38\x68\xcb\x9b\x30\xf7\xd0\xd4\x95\xc6\x9d\x01\xb5\xc5\xd5\x0b\x27\x04\x5d\xb3\x86\x6c\x23\x24\xa4\x4a\x11\x0b\x17\x17\x74\x6d\xe4\x57\xd1\xc8\xc4\x5c\x3c\xd2\xa9\x29\x70\xc3\xd5\x96\x32\x05\x5d\x4c\x98\xa4\x1d\x6e\x99\xe2\xa3\xdd\xd5\xf7\xf9\x97\x9a\xb3\xcd\x18\xf3\x75\x05\xd2\x51\x41\xde\x2a\x1b\xff\x17\xb3\xa7\xdc\xe9\x41\x9e\xcc\x38\x5c\xf1\x1d\x72\x84\x0f\x19\x95\x3f\xd0\x50\x92\x51\xf6\xca\xfd\xe2\x89\x3d\x0e\x75\xc7\x81\xba\x7a\x50\x12\xca\x40\x1a\x4f\xa9\x9e\x04\xb3\xc3\x24\x9f\x92\x6d\x5a\xfe\x82\xcc\x87\xda\xb2\x2c\x3c\x1b\x10\x5d\xe4\x8e\x34\xac\xe9\xc9\x12\x4e\x59\x59\x7a\xc7\xeb\xf8" + , salt = + "\x02\x1f\xdc\xc6\xeb\xb5\xe1\x9b\x1c\xb1\x6e\x9c\x67\xf2\x76\x81\x65\x7f\xe2\x0a" + , signature = + "\x1e\x24\xe6\xe5\x86\x28\xe5\x17\x50\x44\xa9\xeb\x6d\x83\x7d\x48\xaf\x12\x60\xb0\x52\x0e\x87\x32\x7d\xe7\x89\x7e\xe4\xd5\xb9\xf0\xdf\x0b\xe3\xe0\x9e\xd4\xde\xa8\xc1\x45\x4f\xf3\x42\x3b\xb0\x8e\x17\x93\x24\x5a\x9d\xf8\xbf\x6a\xb3\x96\x8c\x8e\xdd\xc3\xb5\x32\x85\x71\xc7\x7f\x09\x1c\xc5\x78\x57\x69\x12\xdf\xeb\xd1\x64\xb9\xde\x54\x54\xfe\x0b\xe1\xc1\xf6\x38\x5b\x32\x83\x60\xce\x67\xec\x7a\x05\xf6\xe3\x0e\xb4\x5c\x17\xc4\x8a\xc7\x00\x41\xd2\xca\xb6\x7f\x0a\x2a\xe7\xaa\xfd\xcc\x8d\x24\x5e\xa3\x44\x2a\x63\x00\xcc\xc7" + } + , -- Example 8.5 + VectorPSS + { message = + "\x04\xdc\x25\x1b\xe7\x2e\x88\xe5\x72\x34\x85\xb6\x38\x3a\x63\x7e\x2f\xef\xe0\x76\x60\xc5\x19\xa5\x60\xb8\xbc\x18\xbd\xed\xb8\x6e\xae\x23\x64\xea\x53\xba\x9d\xca\x6e\xb3\xd2\xe7\xd6\xb8\x06\xaf\x42\xb3\xe8\x7f\x29\x1b\x4a\x88\x81\xd5\xbf\x57\x2c\xc9\xa8\x5e\x19\xc8\x6a\xcb\x28\xf0\x98\xf9\xda\x03\x83\xc5\x66\xd3\xc0\xf5\x8c\xfd\x8f\x39\x5d\xcf\x60\x2e\x5c\xd4\x0e\x8c\x71\x83\xf7\x14\x99\x6e\x22\x97\xef" + , salt = + "\xc5\x58\xd7\x16\x7c\xbb\x45\x08\xad\xa0\x42\x97\x1e\x71\xb1\x37\x7e\xea\x42\x69" + , signature = + "\x33\x34\x1b\xa3\x57\x6a\x13\x0a\x50\xe2\xa5\xcf\x86\x79\x22\x43\x88\xd5\x69\x3f\x5a\xcc\xc2\x35\xac\x95\xad\xd6\x8e\x5e\xb1\xee\xc3\x16\x66\xd0\xca\x7a\x1c\xda\x6f\x70\xa1\xaa\x76\x2c\x05\x75\x2a\x51\x95\x0c\xdb\x8a\xf3\xc5\x37\x9f\x18\xcf\xe6\xb5\xbc\x55\xa4\x64\x82\x26\xa1\x5e\x91\x2e\xf1\x9a\xd7\x7a\xde\xea\x91\x1d\x67\xcf\xef\xd6\x9b\xa4\x3f\xa4\x11\x91\x35\xff\x64\x21\x17\xba\x98\x5a\x7e\x01\x00\x32\x5e\x95\x19\xf1\xca\x6a\x92\x16\xbd\xa0\x55\xb5\x78\x50\x15\x29\x11\x25\xe9\x0d\xcd\x07\xa2\xca\x96\x73\xee" + } + , -- Example 8.6 + VectorPSS + { message = + "\x0e\xa3\x7d\xf9\xa6\xfe\xa4\xa8\xb6\x10\x37\x3c\x24\xcf\x39\x0c\x20\xfa\x6e\x21\x35\xc4\x00\xc8\xa3\x4f\x5c\x18\x3a\x7e\x8e\xa4\xc9\xae\x09\x0e\xd3\x17\x59\xf4\x2d\xc7\x77\x19\xcc\xa4\x00\xec\xdc\xc5\x17\xac\xfc\x7a\xc6\x90\x26\x75\xb2\xef\x30\xc5\x09\x66\x5f\x33\x21\x48\x2f\xc6\x9a\x9f\xb5\x70\xd1\x5e\x01\xc8\x45\xd0\xd8\xe5\x0d\x2a\x24\xcb\xf1\xcf\x0e\x71\x49\x75\xa5\xdb\x7b\x18\xd9\xe9\xe9\xcb\x91\xb5\xcb\x16\x86\x90\x60\xed\x18\xb7\xb5\x62\x45\x50\x3f\x0c\xaf\x90\x35\x2b\x8d\xe8\x1c\xb5\xa1\xd9\xc6\x33\x60\x92\xf0\xcd" + , salt = + "\x76\xfd\x4e\x64\xfd\xc9\x8e\xb9\x27\xa0\x40\x3e\x35\xa0\x84\xe7\x6b\xa9\xf9\x2a" + , signature = + "\x1e\xd1\xd8\x48\xfb\x1e\xdb\x44\x12\x9b\xd9\xb3\x54\x79\x5a\xf9\x7a\x06\x9a\x7a\x00\xd0\x15\x10\x48\x59\x3e\x0c\x72\xc3\x51\x7f\xf9\xff\x2a\x41\xd0\xcb\x5a\x0a\xc8\x60\xd7\x36\xa1\x99\x70\x4f\x7c\xb6\xa5\x39\x86\xa8\x8b\xbd\x8a\xbc\xc0\x07\x6a\x2c\xe8\x47\x88\x00\x31\x52\x5d\x44\x9d\xa2\xac\x78\x35\x63\x74\xc5\x36\xe3\x43\xfa\xa7\xcb\xa4\x2a\x5a\xaa\x65\x06\x08\x77\x91\xc0\x6a\x8e\x98\x93\x35\xae\xd1\x9b\xfa\xb2\xd5\xe6\x7e\x27\xfb\x0c\x28\x75\xaf\x89\x6c\x21\xb6\xe8\xe7\x30\x9d\x04\xe4\xf6\x72\x7e\x69\x46\x3e" + } + ] + +doSignTest key i vector = it (show i) (actual `shouldBe` Right (signature vector)) + where + actual = + PSS.signWithSalt + (salt vector) + Nothing + PSS.defaultPSSParamsSHA1 + key + (message vector) + +doVerifyTest key i vector = it (show i) (actual `shouldBe` True) + where + actual = + PSS.verify + PSS.defaultPSSParamsSHA1 + (private_pub key) + (message vector) + (signature vector) + +-- | RSAVP1 (RFC 8017 section 5.2.2 step 1) refuses a signature representative +-- outside @[0, n-1]@, and section 8.1.2 step 1 passes the signature to it +-- unchanged. The modular exponentiation normalises the range away, so without +-- the check @s + n@ verifies exactly as well as @s@ whenever it still fits in +-- k octets: a third party can turn one valid signature into another without +-- the private key, over the same message. +signatureRangeTests :: Spec +signatureRangeTests = + describe "signature range" $ do + it "the signature itself verifies" $ + verify' s `shouldBe` True + it "the same signature plus n is refused" $ + verify' (i2ospOf_ k (os2ip s + modulus)) `shouldBe` False + it "a signature representative equal to the modulus is refused" $ + verify' (i2ospOf_ k modulus) `shouldBe` False + where + key = rsaKey1 + k = public_size (private_pub key) + modulus = public_n (private_pub key) + verify' = PSS.verify PSS.defaultPSSParamsSHA1 (private_pub key) (message vec) + -- the first vector whose signature can be shifted by n and still fit in k + -- octets + (vec, s) = + firstVector + [ (v, sg) + | v <- vectorsKey1 + , let sg = signature v + , os2ip sg + modulus < 2 ^ (8 * k) + ] + +spec :: Spec +spec = + describe "RSA-PSS" $ do + signatureRangeTests + describe "signature internal" $ do + doSignTest rsaKeyInt katZero vectorInt + describe "verify internal" $ do + doVerifyTest rsaKeyInt katZero vectorInt + describe "signature key 1024" $ + sequence_ $ + zipWith (doSignTest rsaKey1) [katZero ..] vectorsKey1 + describe "verify key 1024" $ + sequence_ $ + zipWith (doVerifyTest rsaKey1) [katZero ..] vectorsKey1 + describe "signature key 1025" $ + sequence_ $ + zipWith (doSignTest rsaKey2) [katZero ..] vectorsKey2 + describe "verify key 1025" $ + sequence_ $ + zipWith (doVerifyTest rsaKey2) [katZero ..] vectorsKey2 + describe "signature key 1026" $ + sequence_ $ + zipWith (doSignTest rsaKey3) [katZero ..] vectorsKey3 + describe "verify key 1026" $ + sequence_ $ + zipWith (doVerifyTest rsaKey3) [katZero ..] vectorsKey3 + describe "signature key 1031" $ + sequence_ $ + zipWith (doSignTest rsaKey8) [katZero ..] vectorsKey8 + describe "verify key 1031" $ + sequence_ $ + zipWith (doVerifyTest rsaKey8) [katZero ..] vectorsKey8 diff --git a/tests/PubKey/RSASpec.hs b/tests/PubKey/RSASpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/RSASpec.hs @@ -0,0 +1,306 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.RSASpec (spec) where + +import Crypto.Hash +import Crypto.Number.ModArithmetic (inverse) +import Crypto.Number.Serialize (i2osp, i2ospOf_, os2ip) +import qualified Crypto.PubKey.RSA as RSA +import qualified Crypto.PubKey.RSA.PKCS15 as RSA +import Crypto.PubKey.RSA.Prim (ep) +import qualified Crypto.PubKey.RSA.Prim as Prim +import qualified Data.ByteString as B +import Data.Either + +import Imports + +data VectorRSA = VectorRSA + { size :: Int + , msg :: ByteString + , n :: Integer + , e :: Integer + , d :: Integer + , p :: Integer + , q :: Integer + , dP :: Integer + , dQ :: Integer + , qinv :: Integer + , sig :: Either RSA.Error ByteString + } + +vectorsSHA1 :: [VectorRSA] +vectorsSHA1 = + [ VectorRSA + { size = 2048 `div` 8 + , msg = "The quick brown fox jumps over the lazy dog" + , n = + 0x00c896c245fcca81775346c5f4f958229cab1aee08196dab4ee5959018b856aab93e4486f37a32da1a6804403c88473ecf9f1b9266fc682400d45329b6ec195710c98d9ba728bc09d767e7e9d9b8b102c3b7e7529b87f649a2a5ebe165da21863ec7842de600a834a8be2227bc989145b52f84ba685d45484a3d530745598a5d8a9e7551b3278bf139a770929f776aed5d43559205fe937df93eeb8ff3fb3f2ce22d4b8a5c17aeafd19758ac5e6251df09ef6a2858e8558a7f476dde4efe859ff2fcb97767614563033fd1d2d300196b1abf256f7badb16c17def3804946d1bf9cd51760176b41bbd506b44ff2bfe5bcd5052180da3cfbbc6cd6f662c06a8baed9 + , e = 0x10001 + , d = + 0x58aa533bae8f310536d95cdd796e5cf655a7f4b9bdcbbd62859743f7b95c0de10e462a44ebaa18c07d640ba4f6344fee648d427ca56bbf2662b45407187be70173a655bc6104257182eb7f720ef2a79f2de6619c804ffca299a7179df6fac4a57179daf4052c550295f0f111ab7ae38e406ff219f9c88b38cdbcaac51bdc4e961361b87e100d168fc08b298626a806b3bfeaa9579f400bbe6e3e6e4ae9b27446e1c5ce8c10c848b9ad7b6ed3a6b3871ad6a1a88af24e581da054845c197e8bae1582858410087c1180c4f0cc61689abfd0f61b8031910f3b3779e11a7fbe823d9a704c63c313f78c994975de834ee9ead5faf6c18b3e4248c51ba307776bf845 + , p = + 0x00f85bfcfe55af59445f21f67ab1d8617d1f84360556eeb660d5c466f29e4d2228f9cc3fde4c594ea97069a19c666b68b6d905b65738ae63de6c11f9181ee9262313e5165591651bb3abec192abbc8c3694550bcffa451a2e2d1976bf3ecbc4480354f8d8646133298156aaa626b8807c5295850f93686400835466b6a5ccec61b + , q = + 0x00cec28b22b1d37c6c60d25e9747cb1bebd1270f0306db56ed8533f392d6a0cfe6b3dde13789758cf89febac214ba96667e46599f89ca210dced550ca6092a854ff95dff80ea48ff1a83455f4bb93f2ececa782da03b85a789239e8be5264130628724ceab57c8f76e4c7e822bf4fbf334c7d32610bec65047433e0e3b636afe1b + , dP = + 0x52fe0a50c339514f33ab19be6e67ac4c2f97f2a55e236ef674f8a89e329ffbe64d731f749d76ca7e7c7e0fef3f9a6ce78d260784a600408736fdda8b60e8f0419088612a3ee7d695f7c171b78200d8abf8e9bdfe7f5e785beb45fa610c9eed151abb76c383ef2e5cfbeb24fcb68a426e741e7b108c53d859e5d39e5970a1f839 + , dQ = + 0x39ef91853b47038a6ae707d2642fa9b73e782f60adbf307085eeb4c5e496532b56234a4481a40ac870275da846c74506bf9d28b3dd501c618baf5548013185018fe2a301c0a48bb726297e367dc6129ba7685d8094ad32f0dea64295074f24fbb6dabd7e8daea686a5b09d512be89d91a09cae01eb332eb389480e3cddf2d119 + , qinv = + 0x09ce1fa29008ef4b9798e5b8ec213dbdfec4fab4403ebf4b8786ad401ef33bc880c40a990b0826f72415192a206a504b27d2ba45ca555706200ea8e7a9b42d4077e9e6e0d80d4144966c53a36d23d30d987322dcc0013efe8df3b6b5914a2ceefc22cc5de6d569731794e9894f18f11d36a79558dc4c3ae5db1ce9bd05e7bf2e + , sig = + Right + "\x56\x66\x99\x0f\xd4\xea\x2b\xe0\x6d\x46\x3b\x10\x99\x5b\x06\x32\x5e\xec\x29\xfe\xa4\x63\x4d\x54\xf6\x31\x74\x5d\x01\x5a\x67\x09\x2e\xa7\x02\x8a\x48\x00\x3c\x0d\xef\x04\xe7\x52\x46\xe0\xfa\xb1\x42\x26\x89\xe7\xec\x25\x44\x76\xa0\x86\x33\xb0\xbe\x22\x17\x88\x9b\x18\x4d\x3e\xc2\x9b\xd4\x61\x2b\x9e\xde\x08\x56\xf8\xd5\xee\xb8\x38\xf4\x3d\xda\x9a\xbb\x34\x58\x87\x71\x1d\x1a\x7e\xc7\x3d\x46\x39\x01\x79\x29\x8b\xa4\xcd\xce\xd7\xab\xcb\x2e\x94\x5c\xfd\x54\xcc\xef\x80\x31\xfc\x5e\x8f\xc2\x4d\x76\x1e\x4c\xbc\x50\x7a\x9b\x08\xae\x85\xeb\x6a\xe0\x80\xdc\xff\x60\x13\xb0\x31\x94\x14\x9d\x8f\x9f\x48\x38\xcf\x4c\x82\x9d\x3b\x68\xc6\xe4\xe9\x5d\x94\x74\xa2\xac\x1f\xb9\x84\x41\x86\x11\xeb\x2c\x50\x64\xd7\x00\xe0\x85\x21\x5a\xd7\xae\x9b\x4c\x8e\x6a\x92\x97\xac\xcc\xb8\x38\x4f\x41\xb9\x3d\xa9\xfe\x69\x8b\x04\x81\xad\xfb\x0f\x49\x74\xfe\x26\x9c\x86\x0c\xf3\xd1\x8e\xa1\xb5\xaf\xef\x85\x3d\xfe\xd0\x7c\xcf\x18\xe4\x0f\x14\x99\xea\x93\x61\x79\x16\xbf\x38\xac\xa2\xa2\xac\xac\x2d\xae\x21\x85\x71\x94\xda\x5d\xa1\x82\xa8\x76\x82\xe5\x2f" + } + , VectorRSA + { size = 360 `div` 8 + , msg = "The quick brown fox jumps over the lazy dog" + , n = + 0x00bc2d7481c83c8be55da4caeaf1a30dbf9a1226ba7443c0a66213180d3eb8e29c3162401b7be067dff8f571a8eb + , e = 0x10001 + , d = + 0x726fb62d82c707507a2d5055a6934136270d28ce350c3a36d89066e26fb54f5b33da0bc9a05c2084f2b39be4e1 + , p = 0x0e3ff89e1f95a461c9f5ee480fd7b13529a225f3ee07fb + , q = 0x0d349ebc89329b493c03451ad20155de9775df55c55fd1 + , dP = 0x00943adef9fb93a561967bab33f198c2c7414e777df997 + , dQ = 0x078de99ceb5392f7f327dfb97717a27ae2e4606dddaa71 + , qinv = 0x0c54d59eaa029844fb3fe33a180161590b1cb103cc668e + , sig = Left RSA.SignatureTooLong + } + , VectorRSA + { size = 368 `div` 8 + , msg = "The quick brown fox jumps over the lazy dog" + , n = + 0x009cff2fd20246e390d6860b48a3926e83086d1386f7147e9f195623cf8f18546ceb20d428b77e0748864c8f611cb7 + , e = 0x10001 + , d = + 0x0097706cbf6624dd448c3a36ce35c27d49762a4948ca33804178d2ff826f8d336aaed622801c8d76d442be371da841 + , p = 0x00d12519f81441069ab1a86c38e0065e9578a46e655d5a17 + , q = 0x00c02b485ac3ee241d57b6b282f830d7d5bf6f4de75c1661 + , dP = 0x00a1af4611444f34f4d88d7504cf23fd711e70382c42ec07 + , dQ = 0x04226a4219a90bf9dda33e9ff6bb0649c0fea20c723cc1 + , qinv = 0x5dd87bf3c1e295dcc8602859a7cd74f05a2fe91a9d5877 + , sig = + Right + "\x51\xe4\xdd\x98\xee\xd5\x06\xef\x7a\xa5\x3c\xaf\x29\x33\xa4\x91\xfa\x8b\xb8\x09\xcf\x3e\xa1\x64\x92\x71\xad\x7b\x3a\x83\xb2\xa0\x77\x94\x4e\x59\xdf\x69\x58\x2e\xc8\x8d\xa0\x70\xfe\x7d" + } + ] + +vectorToPrivate :: VectorRSA -> RSA.PrivateKey +vectorToPrivate vector = + RSA.PrivateKey + { RSA.private_pub = vectorToPublic vector + , RSA.private_d = d vector + , RSA.private_p = p vector + , RSA.private_q = q vector + , RSA.private_dP = dP vector + , RSA.private_dQ = dQ vector + , RSA.private_qinv = qinv vector + } + +vectorToPublic :: VectorRSA -> RSA.PublicKey +vectorToPublic vector = + RSA.PublicKey + { RSA.public_size = size vector + , RSA.public_n = n vector + , RSA.public_e = e vector + } + +vectorHasSignature :: VectorRSA -> Bool +vectorHasSignature = isRight . sig + +doSignatureTest :: Show a => a -> VectorRSA -> Spec +doSignatureTest i vector = it (show i) (actual `shouldBe` expected) + where + expected = sig vector + actual = RSA.sign Nothing (Just SHA1) (vectorToPrivate vector) (msg vector) + +doVerifyTest :: Show a => a -> VectorRSA -> Spec +doVerifyTest i vector = it (show i) (actual `shouldBe` True) + where + actual = RSA.verify (Just SHA1) (vectorToPublic vector) (msg vector) bs + bs = fromRight (error "doVerifyTest") $ sig vector + +-- | RFC 8017 section 8.2.2 step 1 requires a signature that is not exactly k +-- octets long, k being the modulus length, to be rejected, and RSAVP1 (section +-- 5.2.2 step 1) requires the same of a signature representative outside +-- [0, n-1]. Verification here re-encodes the expected signature and compares +-- it against the result of the public-key operation, which normalises both the +-- length and the range away: without those two checks a zero-padded signature +-- and @s + n@ verify just as well as @s@ itself. +doMalleabilityTest :: Show a => a -> VectorRSA -> Spec +doMalleabilityTest i vector = + describe (show i) $ do + it "the signature itself verifies" $ + verify' s `shouldBe` True + it "a leading zero octet is rejected" $ + verify' (B.cons 0 s) `shouldBe` False + it "a trailing zero octet is rejected" $ + verify' (B.snoc s 0) `shouldBe` False + it "s + n is rejected" $ + verify' (i2osp (os2ip s + n vector)) `shouldBe` False + it "an empty signature is rejected" $ + verify' B.empty `shouldBe` False + where + s = fromRight (error "doMalleabilityTest") $ sig vector + verify' = RSA.verify (Just SHA1) (vectorToPublic vector) (msg vector) + +-- | The checks RFC 8017 section 7.2.2 puts on an EME-PKCS1-v1_5 block: the +-- leading @00 02@, a padding string of at least eight nonzero octets, and the +-- @00@ that ends it. Nothing exercised unpad before, and the scan over the +-- padding is about to be rewritten, so pin the accepted and rejected shapes +-- down first. +unpadTests :: Spec +unpadTests = + describe "unpadding" $ do + accepts "the shortest permitted padding" (block 8 "hello") "hello" + accepts "a longer padding" (block 40 "hello") "hello" + accepts "an empty message" (block 8 "") "" + accepts "a message of one octet" (block 8 "x") "x" + rejects "a first octet that is not 00" $ + B.cons 1 (B.drop 1 (block 8 "hello")) + rejects "a second octet that is not 02" $ + B.concat [B.pack [0, 1], B.drop 2 (block 8 "hello")] + rejects "a padding string of seven octets" (block 7 "hello") + rejects "a padding string of no octets" (block 0 "hello") + rejects "a zero inside the first eight padding octets" $ + B.concat [B.pack [0, 2, 0xff, 0xff, 0], "hello"] + rejects "no octet ending the padding string" $ + B.concat [B.pack [0, 2], B.replicate 40 0xff] + rejects "an empty block" B.empty + rejects "a block of one octet" (B.singleton 0) + rejects "a block of two octets" (B.pack [0, 2]) + where + block padLen payload = + B.concat [B.pack [0, 2], B.replicate padLen 0xff, B.singleton 0, payload] + accepts name input expected = + it name (RSA.unpad input `shouldBe` Right expected) + rejects name input = + it + name + ( (RSA.unpad input :: Either RSA.Error ByteString) + `shouldBe` Left RSA.MessageNotRecognized + ) + +-- | RSADP (RFC 8017 section 5.1.2 step 1) refuses a ciphertext representative +-- outside @[0, n-1]@, and section 7.2.2 step 1 passes the ciphertext to it +-- unchanged. The modular exponentiation normalises the range away, so without +-- the check @c@ and @c + n@ decrypt to the same message whenever @c + n@ still +-- fits in k octets -- and then a ciphertext is not unique to its plaintext, +-- which is what a replay cache keyed on the ciphertext assumes. +ciphertextRangeTests :: Spec +ciphertextRangeTests = + describe "ciphertext range" $ do + it "the ciphertext itself decrypts" $ + decrypt' c `shouldBe` Right m + it "the same ciphertext plus n is refused" $ + decrypt' (i2ospOf_ k (os2ip c + modulus)) `shouldBe` sizeError + it "a ciphertext representative equal to the modulus is refused" $ + decrypt' (i2ospOf_ k modulus) `shouldBe` sizeError + where + vector = firstVector vectorsSHA1 + k = size vector + modulus = n vector + decrypt' ct = + RSA.decrypt Nothing (vectorToPrivate vector) ct :: Either RSA.Error ByteString + sizeError = Left RSA.MessageSizeIncorrect + + -- The padding string of an EME-PKCS1-v1_5 block is nonzero octets of the + -- encrypter's choosing, so the block can be built here and encrypted with + -- the public key. Whether c + n fits in k octets depends on the message; + -- with this modulus about a quarter of the candidates below do. + (m, c) = + firstVector + [ (msg', ct) + | i <- [1 .. 200 :: Int] + , let msg' = B.append "message " (B.replicate i 0x78) + , let block = + B.concat + [ B.pack [0, 2] + , B.replicate (k - 3 - B.length msg') 0xff + , B.pack [0] + , msg' + ] + , let ct = ep (vectorToPublic vector) block + , os2ip ct + modulus < 2 ^ (8 * k) + ] + +-- | Building a key from its two primes has to arrive at the key the vectors +-- carry -- the private exponent, both of its halves, and the inverse of one +-- prime modulo the other, which is the part worked out without the extended +-- Euclidean algorithm. +keyGenerationTests :: Spec +keyGenerationTests = + describe "generateWith" $ + zipWithM_ check [katZero ..] vectorsSHA1 + where + check i vector = + it (show i) $ + RSA.generateWith (p vector, q vector) (size vector) (e vector) + `shouldBe` Just (vectorToPublic vector, vectorToPrivate vector) + +-- | The blinder is a number and its inverse, and everything the blinding +-- does rests on that: the decryption multiplies by the one on the way in and +-- by the other on the way out, so an answer that comes back the same either +-- way is the pair being what it says it is. +blinderTests :: Spec +blinderTests = describe "blinder" $ do + prop "holds a number and its inverse" $ \testDRG -> + let key = vectorToPrivate (firstVector vectorsSHA1) + modulus = RSA.public_n (RSA.private_pub key) + RSA.Blinder r rm1 = withTestDRG testDRG $ RSA.generateBlinder modulus + in (r * rm1) `mod` modulus === 1 + prop "leaves the decryption where it was" $ \testDRG -> + let vector = firstVector vectorsSHA1 + key = vectorToPrivate vector + cipher = ep (vectorToPublic vector) (B.replicate 32 7) + blinder = + withTestDRG testDRG $ RSA.generateBlinder (RSA.public_n (RSA.private_pub key)) + in Prim.dp (Just blinder) key cipher === Prim.dp Nothing key cipher + +-- | The private exponent is the inverse of e modulo (p-1)(q-1), however it +-- is worked out. These are primes small enough to be quick and a spread of +-- exponents: prime ones, which have the arithmetic of e to themselves, a +-- composite one, which does not, and ones that share a factor with the +-- modulus and so have no inverse at all. +privateExponentTests :: Spec +privateExponentTests = describe "private exponent" $ do + it "is the inverse of e modulo phi" $ + [ (pr, qr, ex) + | (pr, qr) <- primePairs + , ex <- exponents + , let phi = (pr - 1) * (qr - 1) + , fmap (RSA.private_d . snd) (RSA.generateWith (pr, qr) 64 ex) + /= inverse ex phi + ] + `shouldBe` [] + where + primePairs = + [ (11, 13) + , (61, 53) + , (10007, 10009) + , (1000003, 1000033) + , + ( 0xfffffffffffffffffffffffffffffffeffffffffffffffff + , 0xffffffff00000001000000000000000000000000ffffffffffffffffffffffff + 4294967295 + ) + ] + exponents = [3, 5, 17, 257, 65537, 9, 15, 2] + +spec :: Spec +spec = do + keyGenerationTests + privateExponentTests + blinderTests + describe "SHA1" $ do + describe "signature" $ zipWithM_ doSignatureTest [katZero ..] vectorsSHA1 + describe "verify" $ + sequence_ $ + zipWith doVerifyTest [katZero ..] $ + filter vectorHasSignature vectorsSHA1 + describe "malleability" $ + sequence_ $ + zipWith doMalleabilityTest [katZero ..] $ + filter vectorHasSignature vectorsSHA1 + unpadTests + ciphertextRangeTests diff --git a/tests/PubKey/RabinSpec.hs b/tests/PubKey/RabinSpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/RabinSpec.hs @@ -0,0 +1,381 @@ +{-# LANGUAGE OverloadedStrings #-} + +module PubKey.RabinSpec (spec) where + +import qualified Data.ByteString as B + +import Crypto.Hash +import Crypto.Number.Serialize (i2osp, os2ip) +import qualified Crypto.PubKey.Rabin.Basic as BRabin +import qualified Crypto.PubKey.Rabin.Modified as MRabin +import qualified Crypto.PubKey.Rabin.OAEP as OAEP +import qualified Crypto.PubKey.Rabin.RW as RW +import Crypto.PubKey.Rabin.Types (Error (..)) +import Crypto.Random (drgNewTest, withDRG) +import Data.Bits (xor) + +import Imports + +basicRabinKey = + BRabin.PrivateKey + { BRabin.private_pub = + BRabin.PublicKey + { BRabin.public_n = + 0xc9c4b0df9db989d93df4137fc2de2a9cee2610523f7a450ecbbf252babe98fba2f8e389c3e420c081e18f584c5746ca43f77f6af1fc79161f8bf8fbcb9564779986ecbe656dd16740cb8e399c33ff1dcc679e73c9c98a58c65a8673b7de57290a2d3191cb27e29d627f7ec6e874b1406051ffe9181e4d90d1b487b100ad30685 + , BRabin.public_size = 128 + } + , BRabin.private_p = + 0xe071f231ab5912285a1f8db199795f5efdea4c32f646a3436eaec091ba853a3092216f26b539bbac1fe2ab2e4fbb20aad272a434a1e909bf6d3028aecae2a7b7 + , BRabin.private_q = + 0xe6229470dc7da58bfcd962f1b3ddcf52304efbfb91d31c8ed84dbae2380c1ad2e338a523b4250863a689b3f262f949bd7a9f1a603c36634bb932dd71bf5daba3 + , BRabin.private_a = + 0x65956653f711a63b776ce45862d4cd78f1ad7b1f8ed118bb8b5ea5fffd59762da5dc7c5298e236a8e45d5c93477cbc51f214b1cd1a4980eda859c1cb05e55666 + , BRabin.private_b = + -0x63126dd9c5d6b5215f62012885570e1306b6a47ec1c46553f3b13ceae869149d14544438dbb976800cd62fbb52266f9a6405bc91f192a462c974bc8a6f832e03 + } + +modifiedRabinKey = + MRabin.PrivateKey + { MRabin.private_pub = + MRabin.PublicKey + { MRabin.public_n = + 0x9461a6e7c55cb610f20fd9af5d642404a63332a8d7c4fe7aa559cbcaec691e7216eed5d9322cb6a8619c220a0241b44e0d0a7cefda01fb84e59722b4e842ab5e190d214424bbdfed6d523426fc57a28045dfbb6e8159123077c542c0278ee2daf2d8993e286bf709a10a948da6b13008441581a22233f0ad3d5ebc5858ff7be5 + , MRabin.public_size = 128 + } + , MRabin.private_p = + 0xc401e0ddbe565a8797292389bebb561c35eb019116ba25cc6c865a8d3d7bc599626ddf0bc4f575c22f89144fe99fc3300dd497ec2b7acc0221e729a61756b3f3 + , MRabin.private_q = + 0xc1cc0e35f23f5086691a18c755881e3fe6937581948b109f47605b45d055e7b352e19ff729dfb33fbecb1d28b115e590449e5e4e228ab1876d889d3d41d87ec7 + , MRabin.private_d = + 0x128c34dcf8ab96c21e41fb35ebac848094c666551af89fcf54ab39795d8d23ce42dddabb264596d50c33844140483689c1a14f9dfb403f709cb2e4569d08556b9267e6460e84c69beda1defabd0285c4852c288b7ac27b78987bd19da337a6b1c7b123476732d9c0f656cc62a17f70e8fe34516cfa85ce6475bddeae9ffa0926 + } + +rwKey = + RW.PrivateKey + { RW.private_pub = + RW.PublicKey + { RW.public_n = + 0x992db4c84564c68d4ee2fe0903d938b41e83bcac48dfe8f2219ccee2ccbdefda4cbeea9f1c98a515c5f39a458f5ea11bca97102aaa3d9ac69e000093024e7b968359287cdf57bdacff5df1893df3539c7e358f037d49b5c6ae7110ab8117220c73b6265987039c2c97078fccacdd3f5a560aff5076fdc3958c532db28ab9a855 + , RW.public_size = 128 + } + , RW.private_p = + 0xc144dd739c45397d61868ca944a9729a7ad34cf90466c8f5c98a88f5ab5e3288bcfd31d4af1d441d23a756a60abd4cf05c3e0b0053eb150166a327ae31e9347b + , RW.private_q = + 0xcae5a381f25a27ae2c359068753118fc384471cd6027e88b8b910306fb940781261089259a3c569546677aebd268704c767a071dbd4f50cb9f15fe448788856f + , RW.private_d = + 0x1325b69908ac98d1a9dc5fc1207b271683d07795891bfd1e443399dc5997bdfb4997dd53e39314a2b8be7348b1ebd4237952e2055547b358d3c000126049cf729ee5d4f0ea170b902e343a8ef0831900b963ba07a3176088ab2ab095db449d0052150d6be7b5402f459f17c759f6f043b06a5da64cb86bb910d340f7fa28fdce + } + +data EncryptionVector = EncryptionVector + { seed :: ByteString + , plainText :: ByteString + , cipherText :: ByteString + } + +data SignatureVector = SignatureVector + { message :: ByteString + , padding :: ByteString + , signature :: Integer + } + +basicRabinEncryptionVectors = + [ EncryptionVector + { plainText = + "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" + , seed = + "\x0c\xc7\x42\xce\x4a\x9b\x7f\x32\xf9\x51\xbc\xb2\x51\xef\xd9\x25\xfe\x4f\xe3\x5f" + , cipherText = + "\xaf\xc7\x03\xe3\x9d\x2f\x81\xc6\x3a\x80\x2a\xd1\x44\x26\x3f\x17\x0c\x0a\xe6\x48\x68\x98\x23\x14\x8f\x95\xd2\xce\xbb\xe7\x3f\x49\x34\x76\x1d\x99\x30\x7b\xeb\x84\xe5\x2a\x10\xd2\x1e\x11\x7e\x65\xe8\x88\x24\xc1\x12\xeb\x19\x0d\x97\xcd\x12\x25\x6b\x1f\x9b\x0c\x40\x40\xa3\x47\x00\xb7\x11\xf8\x50\x08\x51\x79\xe8\x1b\xd1\x77\xe0\x99\xa7\xe1\x5c\x63\xda\x29\xc7\xde\x28\x5d\x60\xed\x8e\xb2\x12\xd4\xfe\xb8\x1a\x5d\x17\x65\x80\x62\x6e\x65\x5c\x37\x07\x1c\xfa\xff\xe6\x21\xa5\x9f\xcd\x6a\x6a\xce\xa6\x96\xb2\xc5\x08\xe6" + } + ] + +basicRabinSignatureVectors = + [ SignatureVector + { message = + "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" + , padding = "\xe9\x87\x17\x15\xa2\xe4\x30\x15" + , signature = + 0xac95807bdd03ca975690151d39d23d75e5db2731c4ba30b83c3f3ea74709e4d4e340d7dab952356a76c9b8705b214e28d59f5bdc7c7fdff4e104569e30359b5c65c2dcd5b94db58505cd8b188267121700beebd7edbee492e374514646471b5c3fa252a2580dc7343f455683815d6d7c590dd3bcaa7df41d8b08197ccb183408 + } + ] + +modifiedRabinSignatureVectors = + [ SignatureVector + { message = + "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" + , padding = B.empty -- not used + , signature = + 0x278c7c269119218ab7f501ea53a97ab15a3a5a263c6daed8980abec78291e9729e0e3457731cdea8ec31a7566e93d10fc9b2615fe3e54f4533a5506ac24a3bd286e270324e538066f0ddf503f9b5e0c18e18379659834906ebd99c0d31588c66e70fc653bc8865b9239999cbd35704917d8647d1199286c533233e3e03582dd + } + ] + +rwEncryptionVectors = + [ EncryptionVector + { plainText = + "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" + , seed = + "\x0c\xc7\x42\xce\x4a\x9b\x7f\x32\xf9\x51\xbc\xb2\x51\xef\xd9\x25\xfe\x4f\xe3\x5f" + , cipherText = + "\x40\xc2\xe3\x36\xac\x46\x72\x8a\xaf\x33\x75\xe1\x27\xd0\x38\x40\xe2\x24\x4e\x20\xa7\x5d\x85\xd3\x74\x81\x21\xfd\xc9\x40\x90\x80\x8c\xed\x2d\xd3\x5b\xc4\xb7\xc9\x7c\x80\xa5\x2f\x63\x86\x34\x4e\x8c\x92\x07\x86\x9e\xda\xfd\xf8\x11\x83\x8a\x5a\x23\xc1\xe6\x77\x37\x5d\xf9\x5c\x60\xd1\x6d\xfd\x0c\x54\xd1\x00\xe9\xab\x97\x6d\x8e\x83\x8b\x6e\x1a\x38\x73\x43\xe2\x24\xc2\xe2\x4e\x74\x3f\xe4\x4d\xdd\x27\xed\xc7\x72\x88\xd3\x0f\x93\xb3\xdb\xa2\xb7\xaf\x6d\xe9\xab\x76\x53\x63\xf9\x62\xd7\x52\x44\x61\x60\x5d\x2e\x9b\xf7" + } + ] + +rwSignatureVectors = + [ SignatureVector + { message = + "\x75\x0c\x40\x47\xf5\x47\xe8\xe4\x14\x11\x85\x65\x23\x29\x8a\xc9\xba\xe2\x45\xef\xaf\x13\x97\xfb\xe5\x6f\x9d\xd5" + , padding = B.empty -- not used + , signature = + 0x1e57b554a8e83aacd9d4067f9535991e7db47803250cded5cc8af5458a6bb11fea852139e0afe143f9339dd94a518e354e702134d1ae222460127829d92e8bf6441336f5ae7044ec7b6c3ad8b9aeeb1ea02a49798e020cb5b558120bbb51f060eb1608ba68f90cac7edb1051c177d3bdbb99d1ad92e8d75d6f72f1d06f1d25be + } + ] + +doBasicRabinEncryptTest key i vector = it (show i) (actual `shouldBe` Right (cipherText vector)) + where + actual = + BRabin.encryptWithSeed + (seed vector) + (OAEP.defaultOAEPParams SHA1) + key + (plainText vector) + +doBasicRabinDecryptTest key i vector = it (show i) (actual `shouldBe` Just (plainText vector)) + where + actual = BRabin.decrypt (OAEP.defaultOAEPParams SHA1) key (cipherText vector) + +doBasicRabinSignTest key i vector = + it + (show i) + ( actual + `shouldBe` Right (BRabin.Signature ((os2ip $ padding vector), (signature vector))) + ) + where + actual = BRabin.signWith (padding vector) key SHA1 (message vector) + +doBasicRabinVerifyTest key i vector = it (show i) (actual `shouldBe` True) + where + actual = + BRabin.verify + key + SHA1 + (message vector) + (BRabin.Signature ((os2ip $ padding vector), (signature vector))) + +doModifiedRabinSignTest key i vector = it (show i) (actual `shouldBe` Right (signature vector)) + where + actual = MRabin.sign key SHA1 (message vector) + +doModifiedRabinVerifyTest key i vector = it (show i) (actual `shouldBe` True) + where + actual = MRabin.verify key SHA1 (message vector) (signature vector) + +doRwEncryptTest key i vector = it (show i) (actual `shouldBe` Right (cipherText vector)) + where + actual = + RW.encryptWithSeed + (seed vector) + (OAEP.defaultOAEPParams SHA1) + key + (plainText vector) + +doRwDecryptTest key i vector = it (show i) (actual `shouldBe` Just (plainText vector)) + where + actual = RW.decrypt (OAEP.defaultOAEPParams SHA1) key (cipherText vector) + +doRwSignTest key i vector = it (show i) (actual `shouldBe` Right (signature vector)) + where + actual = RW.sign key SHA1 (message vector) + +doRwVerifyTest key i vector = it (show i) (actual `shouldBe` True) + where + actual = RW.verify key SHA1 (message vector) (signature vector) + +-- | Squaring and the square roots that undo it both work modulo n, so a value +-- at or above the modulus behaves exactly like the value it reduces to, and so +-- does a negated one, @(-s)^2@ being @s^2@. Unless something checks the range, +-- @c + n@ decrypts to whatever @c@ decrypts to and @s + n@ verifies wherever +-- @s@ does -- a ciphertext is then not unique to its plaintext, and anyone can +-- turn one valid signature into another without the private key. A leading +-- zero octet is the same thing said in bytes. +rangeTests :: Spec +rangeTests = describe "value range" $ do + describe "Basic" $ do + it "decrypts a ciphertext it made" $ + basicDecrypt basicCipher `shouldBe` Just (plainText basicEnc) + it "refuses a ciphertext at or above the modulus" $ + basicDecrypt (i2osp (os2ip basicCipher + basicN)) `shouldBe` Nothing + it "refuses a ciphertext with a leading zero octet" $ + basicDecrypt (B.cons 0 basicCipher) `shouldBe` Nothing + it "verifies a signature it made" $ + basicVerify basicSig `shouldBe` True + it "refuses a signature at or above the modulus" $ + basicVerify (basicSig + basicN) `shouldBe` False + it "refuses a negated signature" $ + basicVerify (negate basicSig) `shouldBe` False + describe "Rabin-Williams" $ do + it "decrypts a ciphertext it made" $ + rwDecrypt rwCipher `shouldBe` Just (plainText rwEnc) + it "refuses a ciphertext at or above the modulus" $ + rwDecrypt (i2osp (os2ip rwCipher + rwN)) `shouldBe` Nothing + it "refuses a ciphertext with a leading zero octet" $ + rwDecrypt (B.cons 0 rwCipher) `shouldBe` Nothing + it "verifies a signature it made" $ + rwVerify rwSig `shouldBe` True + it "refuses a signature at or above the modulus" $ + rwVerify (rwSig + rwN) `shouldBe` False + it "refuses a negated signature" $ + rwVerify (negate rwSig) `shouldBe` False + describe "Modified" $ do + it "verifies a signature it made" $ + modVerify modSig `shouldBe` True + it "refuses a signature at or above the modulus" $ + modVerify (modSig + modN) `shouldBe` False + it "refuses a negated signature" $ + modVerify (negate modSig) `shouldBe` False + where + basicEnc = firstVector basicRabinEncryptionVectors + basicCipher = cipherText basicEnc + basicN = BRabin.public_n (BRabin.private_pub basicRabinKey) + basicDecrypt = BRabin.decrypt (OAEP.defaultOAEPParams SHA1) basicRabinKey + basicSigVec = firstVector basicRabinSignatureVectors + basicSig = signature basicSigVec + basicVerify s = + BRabin.verify + (BRabin.private_pub basicRabinKey) + SHA1 + (message basicSigVec) + (BRabin.Signature (os2ip (padding basicSigVec), s)) + + rwEnc = firstVector rwEncryptionVectors + rwCipher = cipherText rwEnc + rwN = RW.public_n (RW.private_pub rwKey) + rwDecrypt = RW.decrypt (OAEP.defaultOAEPParams SHA1) rwKey + rwSigVec = firstVector rwSignatureVectors + rwSig = signature rwSigVec + rwVerify = RW.verify (RW.private_pub rwKey) SHA1 (message rwSigVec) + + modN = MRabin.public_n (MRabin.private_pub modifiedRabinKey) + modSigVec = firstVector modifiedRabinSignatureVectors + modSig = signature modSigVec + modVerify = MRabin.verify (MRabin.private_pub modifiedRabinKey) SHA1 (message modSigVec) + +-- | Basic's signature carries the padding as an integer, so a padding whose +-- first octet is zero comes back one octet short and hashes to something else. +-- sign draws eight random octets, so about one signature in 256 was one its own +-- verify refused. +paddingTests :: Spec +paddingTests = describe "signature padding" $ do + it "refuses a padding that would not survive the signature" $ + BRabin.signWith + (B.cons 0 (B.drop 1 (padding sigVec))) + basicRabinKey + SHA1 + (message sigVec) + `shouldBe` Left InvalidParameters + it "verifies every signature it draws" $ + filter (not . verifies) signatures `shouldBe` [] + where + sigVec = firstVector basicRabinSignatureVectors + -- a fixed generator, so the same 400 paddings are drawn every run + signatures = + fst $ + withDRG (drgNewTest (1, 2, 3, 4, 5)) $ + replicateM 400 (BRabin.sign basicRabinKey SHA1 (message sigVec)) + verifies (Left _) = False + verifies (Right sig) = + BRabin.verify (BRabin.private_pub basicRabinKey) SHA1 (message sigVec) sig + +-- | EME-OAEP decoding accepts a block with the leading zero octet, the label +-- hash it expects, and an 01 octet ending the padding string; it refuses +-- everything else. The scan across that padding string and the comparison of +-- the label hash are about to be rewritten, so write down which blocks are +-- accepted and which are refused first. +oaepTests :: Spec +oaepTests = describe "OAEP" $ do + it "accepts a block it padded" $ + unpad' (block 43) `shouldBe` Right (msg 43) + it "accepts a message that fills the block" $ + unpad' (block 86) `shouldBe` Right (msg 86) + it "accepts a message of one octet, behind the longest padding" $ + unpad' (block 1) `shouldBe` Right (msg 1) + it "refuses a leading octet that is not zero" $ + unpad' (poke 0 1 (block 43)) `shouldBe` Left MessageNotRecognized + it "refuses a label hash that does not match" $ do + unpad' (flipBit 21 (block 43)) `shouldBe` Left MessageNotRecognized + unpad' (flipBit 40 (block 43)) `shouldBe` Left MessageNotRecognized + it "refuses a block with no octet ending the padding string" $ + -- every octet of db after the label hash is zero, so nothing separates + -- the padding from a message + unpad' (B.concat [B.take 21 (block 86), B.replicate 107 0]) + `shouldBe` Left MessageNotRecognized + it "refuses a corrupted masked seed" $ + unpad' (flipBit 3 (block 43)) `shouldBe` Left MessageNotRecognized + where + oaep = OAEP.defaultOAEPParams SHA1 + k = 128 + oaepSeed = B.replicate 20 0x5a + msg n = B.replicate n 0x41 + block n = case OAEP.pad oaepSeed oaep k (msg n) of + Right b -> b + Left e -> error (show e) + unpad' = OAEP.unpad oaep k + poke i w bs = B.concat [B.take i bs, B.singleton w, B.drop (i + 1) bs] + flipBit i bs = poke i (B.index bs i `xor` 1) bs + +spec :: Spec +spec = do + rangeTests + oaepTests + paddingTests + describe "Basic" $ do + describe "encrypt" $ + sequence_ $ + zipWith + (doBasicRabinEncryptTest $ BRabin.private_pub basicRabinKey) + [katZero ..] + basicRabinEncryptionVectors + describe "decrypt" $ + sequence_ $ + zipWith + (doBasicRabinDecryptTest basicRabinKey) + [katZero ..] + basicRabinEncryptionVectors + describe "sign" $ + sequence_ $ + zipWith + (doBasicRabinSignTest basicRabinKey) + [katZero ..] + basicRabinSignatureVectors + describe "verify" $ + sequence_ $ + zipWith + (doBasicRabinVerifyTest $ BRabin.private_pub basicRabinKey) + [katZero ..] + basicRabinSignatureVectors + describe "Modified" $ do + describe "sign" $ + sequence_ $ + zipWith + (doModifiedRabinSignTest modifiedRabinKey) + [katZero ..] + modifiedRabinSignatureVectors + describe "verify" $ + sequence_ $ + zipWith + (doModifiedRabinVerifyTest $ MRabin.private_pub modifiedRabinKey) + [katZero ..] + modifiedRabinSignatureVectors + describe "RW" $ do + describe "encrypt" $ + sequence_ $ + zipWith + (doRwEncryptTest $ RW.private_pub rwKey) + [katZero ..] + rwEncryptionVectors + describe "decrypt" $ + sequence_ $ + zipWith (doRwDecryptTest rwKey) [katZero ..] rwEncryptionVectors + describe "sign" $ zipWithM_ (doRwSignTest rwKey) [katZero ..] rwSignatureVectors + describe "verify" $ + sequence_ $ + zipWith (doRwVerifyTest $ RW.private_pub rwKey) [katZero ..] rwSignatureVectors diff --git a/tests/PubKey/SecrecySpec.hs b/tests/PubKey/SecrecySpec.hs new file mode 100644 --- /dev/null +++ b/tests/PubKey/SecrecySpec.hs @@ -0,0 +1,162 @@ +{-# LANGUAGE ExistentialQuantification #-} +-- | A key's 'Show' instance is what a log, a crash report and a test +-- failure all reach for, and none of those is a place to put a private +-- key. So the types that hold one do not print it, and the module below +-- holds them to that mechanically: the secret is rendered, and the +-- rendering must not appear in what 'show' returns. +module PubKey.SecrecySpec (spec) where + +import Data.List (isInfixOf) + +import Crypto.Debug (DebugShow, debugShow) +import Crypto.Error (CryptoFailable, throwCryptoError) +import qualified Crypto.PubKey.Curve448 as X448 +import qualified Crypto.PubKey.Curve25519 as X25519 +import qualified Crypto.PubKey.DH as DH +import qualified Crypto.PubKey.DSA as DSA +import qualified Crypto.PubKey.ECC.ECDSA as ECDSA +import qualified Crypto.PubKey.ECC.Types as ECC +import qualified Crypto.PubKey.Ed448 as Ed448 +import qualified Crypto.PubKey.Ed25519 as Ed25519 +import qualified Crypto.PubKey.RSA.Types as RSA +import qualified Crypto.PubKey.Rabin.Basic as Basic +import qualified Crypto.PubKey.Rabin.Modified as Modified +import qualified Crypto.PubKey.Rabin.RW as RW + +import qualified Data.ByteString as BS +import Data.Word (Word8) + +import Imports + +-- | Distinctive values, so that finding one in a rendering means it came +-- from the field it was put in and not from a coincidence of digits. +d1, d2, d3, d4, d5, d6 :: Integer +d1 = 0xd1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1d1 +d2 = 0xd2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2d2 +d3 = 0xd3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3d3 +d4 = 0xd4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4d4 +d5 = 0xd5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5d5 +d6 = 0xd6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6d6 + +rsaPub :: RSA.PublicKey +rsaPub = RSA.PublicKey{RSA.public_size = 32, RSA.public_n = 0xabc1, RSA.public_e = 0x10001} + +rsaPriv :: RSA.PrivateKey +rsaPriv = + RSA.PrivateKey + { RSA.private_pub = rsaPub + , RSA.private_d = d1 + , RSA.private_p = d2 + , RSA.private_q = d3 + , RSA.private_dP = d4 + , RSA.private_dQ = d5 + , RSA.private_qinv = d6 + } + +dsaParams :: DSA.Params +dsaParams = DSA.Params{DSA.params_p = 0xabc2, DSA.params_g = 2, DSA.params_q = 0xabc3} + +ecdsaCurve :: ECC.Curve +ecdsaCurve = ECC.getCurveByName ECC.SEC_p256r1 + +-- | Each entry names a value, what it renders to, and the secrets that +-- must not be findable in that rendering. +cases :: [(String, String, [Integer])] +cases = + [ ("RSA.PrivateKey", show rsaPriv, [d1, d2, d3, d4, d5, d6]) + , ("RSA.KeyPair", show (RSA.KeyPair rsaPriv), [d1, d2, d3, d4, d5, d6]) + , ("DSA.PrivateKey", show (DSA.PrivateKey dsaParams d1), [d1]) + , ("DSA.KeyPair", show (DSA.KeyPair dsaParams 0xabc4 d1), [d1]) + , ("ECDSA.PrivateKey", show (ECDSA.PrivateKey ecdsaCurve d1), [d1]) + , ("ECDSA.KeyPair", show (ECDSA.KeyPair ecdsaCurve ECC.PointO d1), [d1]) + , ("DH.PrivateNumber", show (DH.PrivateNumber d1), [d1]) + , + ( "Rabin.Basic.PrivateKey" + , show (Basic.PrivateKey (Basic.PublicKey 32 0xabc5) d1 d2 d3 d4) + , [d1, d2, d3, d4] + ) + , + ( "Rabin.Modified.PrivateKey" + , show (Modified.PrivateKey (Modified.PublicKey 32 0xabc6) d1 d2 d3) + , [d1, d2, d3] + ) + , + ( "Rabin.RW.PrivateKey" + , show (RW.PrivateKey (RW.PublicKey 32 0xabc7) d1 d2 d3) + , [d1, d2, d3] + ) + ] + +-- | The values above again, paired with what 'debugShow' makes of them and +-- with a reading of that back, which has to give the value returned. +data Reveal = forall a. (Show a, Read a, Eq a, DebugShow a) => Reveal a + +reveals :: [(String, Reveal, [Integer])] +reveals = + [ ("RSA.PrivateKey", Reveal rsaPriv, [d1, d2, d3, d4, d5, d6]) + , ("RSA.KeyPair", Reveal (RSA.KeyPair rsaPriv), [d1, d2, d3, d4, d5, d6]) + , ("DSA.PrivateKey", Reveal (DSA.PrivateKey dsaParams d1), [d1]) + , ("DSA.KeyPair", Reveal (DSA.KeyPair dsaParams 0xabc4 d1), [d1]) + , ("ECDSA.PrivateKey", Reveal (ECDSA.PrivateKey ecdsaCurve d1), [d1]) + , ("ECDSA.KeyPair", Reveal (ECDSA.KeyPair ecdsaCurve ECC.PointO d1), [d1]) + , ("DH.PrivateNumber", Reveal (DH.PrivateNumber d1), [d1]) + , + ( "Rabin.Basic.PrivateKey" + , Reveal (Basic.PrivateKey (Basic.PublicKey 32 0xabc5) d1 d2 d3 d4) + , [d1, d2, d3, d4] + ) + , + ( "Rabin.Modified.PrivateKey" + , Reveal (Modified.PrivateKey (Modified.PublicKey 32 0xabc6) d1 d2 d3) + , [d1, d2, d3] + ) + , + ( "Rabin.RW.PrivateKey" + , Reveal (RW.PrivateKey (RW.PublicKey 32 0xabc7) d1 d2 d3) + , [d1, d2, d3] + ) + ] + +-- | The keys that keep their secret in a @ScrubbedBytes@. Their 'Show' was +-- already silent; what is new is that 'debugShow' can speak. The bytes are +-- distinct and not 0 or 255, so finding the hexadecimal of one in a rendering +-- means it came from the key. +scrubbed :: [(String, String, String, String)] +scrubbed = + [ entry "Curve25519.SecretKey" 0x5a (X25519.secretKey . BS.replicate 32) + , entry "Curve448.SecretKey" 0x5b (X448.secretKey . BS.replicate 56) + , entry "Ed25519.SecretKey" 0x5c (Ed25519.secretKey . BS.replicate 32) + , entry "Ed448.SecretKey" 0x5d (Ed448.secretKey . BS.replicate 57) + ] + where + entry + :: (Show k, DebugShow k) + => String + -> Word8 + -> (Word8 -> CryptoFailable k) + -> (String, String, String, String) + entry name b mk = + let k = throwCryptoError (mk b) + in (name, show k, debugShow k, hex b ++ hex b) + hex :: Word8 -> String + hex b = [digit (b `div` 16), digit (b `mod` 16)] + digit n = "0123456789abcdef" !! fromIntegral n + +spec :: Spec +spec = do + describe "show does not print the secret" $ mapM_ check cases + describe "debugShow does print the secret" $ mapM_ reveal reveals + describe "debugShow round-trips through read" $ mapM_ roundTrip reveals + describe "a scrubbed secret key" $ mapM_ scrub scrubbed + where + check (name, rendered, secrets) = + it name $ + [s | s <- secrets, show s `isInfixOf` rendered] `shouldBe` [] + reveal (name, Reveal v, secrets) = + it name $ + [s | s <- secrets, not (show s `isInfixOf` debugShow v)] `shouldBe` [] + roundTrip (name, Reveal v, _) = + it name $ read (debugShow v) `shouldBe` v + scrub (name, shown, revealed, h) = describe name $ do + it "is not printed by show" $ (h `isInfixOf` shown) `shouldBe` False + it "is printed by debugShow" $ (h `isInfixOf` revealed) `shouldBe` True diff --git a/tests/RuntimeSpec.hs b/tests/RuntimeSpec.hs new file mode 100644 --- /dev/null +++ b/tests/RuntimeSpec.hs @@ -0,0 +1,7 @@ +module RuntimeSpec (spec) where + +import Crypto.System.CPU +import Test.Hspec + +spec :: Spec +spec = it "CPU" $ putStrLn (show processorOptions) diff --git a/tests/Salsa.hs b/tests/Salsa.hs deleted file mode 100644 --- a/tests/Salsa.hs +++ /dev/null @@ -1,134 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module Salsa (tests) where - -import qualified Crypto.Cipher.Salsa as Salsa -import qualified Data.ByteString as B - -import Imports - -type Vector = (Int, B.ByteString, B.ByteString, [(Int, B.ByteString)]) - -vectors :: [Vector] -vectors = - [ - ( 20 - , key - , iv - , - [ - ( 0 - , "\x99\xA8\xCC\xEC\x6C\x5B\x2A\x0B\x6E\x33\x6C\xB2\x06\x52\x24\x1C\x32\xB2\x4D\x34\xAC\xC0\x45\x7E\xF6\x79\x17\x8E\xDE\x7C\xF8\x05\x80\x5A\x93\x05\xC7\xC4\x99\x09\x68\x3B\xD1\xA8\x03\x32\x78\x17\x62\x7C\xA4\x6F\xE8\xB9\x29\xB6\xDF\x00\x12\xBD\x86\x41\x83\xBE" - ) - , - ( 192 - , "\x2D\x22\x6C\x11\xF4\x7B\x3C\x0C\xCD\x09\x59\xB6\x1F\x59\xD5\xCC\x30\xFC\xEF\x6D\xBB\x8C\xBB\x3D\xCC\x1C\xC2\x52\x04\xFC\xD4\x49\x8C\x37\x42\x6A\x63\xBE\xA3\x28\x2B\x1A\x8A\x0D\x60\xE1\x3E\xB2\xFE\x59\x24\x1A\x9F\x6A\xF4\x26\x68\x98\x66\xED\xC7\x69\xE1\xE6\x48\x2F\xE1\xC1\x28\xA1\x5C\x11\x23\xB5\x65\x5E\xD5\x46\xDF\x01\x4C\xE0\xC4\x55\xDB\xF5\xD3\xA1\x3D\x9C\xD4\xF0\xE2\xD1\xDA\xB9\xF1\x2F\xB6\x8C\x54\x42\x61\xD7\xF8\x8E\xAC\x1C\x6C\xBF\x99\x3F\xBB\xB8\xE0\xAA\x85\x10\xBF\xF8\xE7\x38\x35\xA1\xE8\x6E\xAD\xBB" - ) - , - ( 448 - , "\x05\x97\x18\x8A\x1C\x19\x25\x57\x69\xBE\x1C\x21\x03\x99\xAD\x17\x2E\xB4\x6C\x52\xF9\x2F\xD5\x41\xDF\x2E\xAD\x71\xB1\xFF\x8E\xA7\xAD\xD3\x80\xEC\x71\xA5\xFD\x7A\xDB\x51\x81\xEA\xDD\x18\x25\xEC\x02\x77\x9A\x45\x09\xBE\x58\x32\x70\x8C\xA2\x83\x6C\x16\x93\xA5" - ) - ] - ) - , - ( 20 - , "\x00\x53\xA6\xF9\x4C\x9F\xF2\x45\x98\xEB\x3E\x91\xE4\x37\x8A\xDD\x30\x83\xD6\x29\x7C\xCF\x22\x75\xC8\x1B\x6E\xC1\x14\x67\xBA\x0D" - , "\x0D\x74\xDB\x42\xA9\x10\x77\xDE" - , - [ - ( 0 - , "\xF5\xFA\xD5\x3F\x79\xF9\xDF\x58\xC4\xAE\xA0\xD0\xED\x9A\x96\x01\xF2\x78\x11\x2C\xA7\x18\x0D\x56\x5B\x42\x0A\x48\x01\x96\x70\xEA\xF2\x4C\xE4\x93\xA8\x62\x63\xF6\x77\xB4\x6A\xCE\x19\x24\x77\x3D\x2B\xB2\x55\x71\xE1\xAA\x85\x93\x75\x8F\xC3\x82\xB1\x28\x0B\x71" - ) - , - ( 65472 - , "\xB7\x0C\x50\x13\x9C\x63\x33\x2E\xF6\xE7\x7A\xC5\x43\x38\xA4\x07\x9B\x82\xBE\xC9\xF9\xA4\x03\xDF\xEA\x82\x1B\x83\xF7\x86\x07\x91\x65\x0E\xF1\xB2\x48\x9D\x05\x90\xB1\xDE\x77\x2E\xED\xA4\xE3\xBC\xD6\x0F\xA7\xCE\x9C\xD6\x23\xD9\xD2\xFD\x57\x58\xB8\x65\x3E\x70\x81\x58\x2C\x65\xD7\x56\x2B\x80\xAE\xC2\xF1\xA6\x73\xA9\xD0\x1C\x9F\x89\x2A\x23\xD4\x91\x9F\x6A\xB4\x7B\x91\x54\xE0\x8E\x69\x9B\x41\x17\xD7\xC6\x66\x47\x7B\x60\xF8\x39\x14\x81\x68\x2F\x5D\x95\xD9\x66\x23\xDB\xC4\x89\xD8\x8D\xAA\x69\x56\xB9\xF0\x64\x6B\x6E" - ) - , - ( 131008 - , "\xA1\x3F\xFA\x12\x08\xF8\xBF\x50\x90\x08\x86\xFA\xAB\x40\xFD\x10\xE8\xCA\xA3\x06\xE6\x3D\xF3\x95\x36\xA1\x56\x4F\xB7\x60\xB2\x42\xA9\xD6\xA4\x62\x8C\xDC\x87\x87\x62\x83\x4E\x27\xA5\x41\xDA\x2A\x5E\x3B\x34\x45\x98\x9C\x76\xF6\x11\xE0\xFE\xC6\xD9\x1A\xCA\xCC" - ) - ] - ) - ] - where - key :: B.ByteString - key = - "\xEA\xEB\xEC\xED\xEE\xEF\xF0\xF1\xF2\xF3\xF4\xF5\xF6\xF7\xF8\xF9\xFA\xFB\xFC\xFD\xFE\xFF\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09" - - iv = B.replicate 8 0 - -newtype RandomVector = RandomVector Vector - deriving (Show, Eq) - -instance Arbitrary RandomVector where - arbitrary = RandomVector <$> elements vectors - -tests = - testGroup - "Salsa" - [ testGroup "KAT" $ - zipWith - (\i (r, k, n, e) -> testCase (show (i :: Int)) $ salsaRunSimple e r k n) - [1 ..] - vectors - , testProperty "generate-combine" salsaGenerateCombine - , testProperty "chunking-generate" salsaGenerateChunks - , testProperty "chunking-combine" salsaCombineChunks - ] - where - salsaRunSimple expected rounds key nonce = - let salsa = Salsa.initialize rounds key nonce - in map snd expected @=? salsaLoop 0 salsa expected - - salsaLoop _ _ [] = [] - salsaLoop current salsa (r@(ofs, expectBs) : rs) - | current < ofs = - let (_, salsaNext) = Salsa.generate salsa (ofs - current) :: (ByteString, Salsa.State) - in salsaLoop ofs salsaNext (r : rs) - | current == ofs = - let (e, salsaNext) = Salsa.generate salsa (B.length expectBs) - in e : salsaLoop (current + B.length expectBs) salsaNext rs - | otherwise = error "internal error in salsaLoop" - - salsaGenerateCombine :: ChunkingLen0_127 -> RandomVector -> Int0_2901 -> Bool - salsaGenerateCombine (ChunkingLen0_127 ckLen) (RandomVector (rounds, key, iv, _)) (Int0_2901 nbBytes) = - let initSalsa = Salsa.initialize rounds key iv - in loop nbBytes ckLen initSalsa - where - loop n [] salsa = loop n ckLen salsa - loop 0 _ _ = True - loop n (x : xs) salsa = - let len = min x n - (c1, next) = Salsa.generate salsa len - (c2, _) = Salsa.combine salsa (B.replicate len 0) - in if c1 == c2 then loop (n - len) xs next else False - - salsaGenerateChunks :: ChunkingLen -> RandomVector -> Bool - salsaGenerateChunks (ChunkingLen ckLen) (RandomVector (rounds, key, iv, _)) = - let initSalsa = Salsa.initialize rounds key iv - nbBytes = 1048 - (expected, _) = Salsa.generate initSalsa nbBytes - chunks = loop nbBytes ckLen (Salsa.initialize rounds key iv) - in expected == B.concat chunks - where - loop n [] salsa = loop n ckLen salsa - loop 0 _ _ = [] - loop n (x : xs) salsa = - let len = min x n - (c, next) = Salsa.generate salsa len - in c : loop (n - len) xs next - - salsaCombineChunks :: ChunkingLen -> RandomVector -> ArbitraryBS0_2901 -> Bool - salsaCombineChunks (ChunkingLen ckLen) (RandomVector (rounds, key, iv, _)) (ArbitraryBS0_2901 wholebs) = - let initSalsa = Salsa.initialize rounds key iv - (expected, _) = Salsa.combine initSalsa wholebs - chunks = loop wholebs ckLen initSalsa - in expected `propertyEq` B.concat chunks - where - loop bs [] salsa = loop bs ckLen salsa - loop bs (x : xs) salsa - | B.null bs = [] - | otherwise = - let (bs1, bs2) = B.splitAt (min x (B.length bs)) bs - (c, next) = Salsa.combine salsa bs1 - in c : loop bs2 xs next diff --git a/tests/Spec.hs b/tests/Spec.hs new file mode 100644 --- /dev/null +++ b/tests/Spec.hs @@ -0,0 +1,1 @@ +{-# OPTIONS_GHC -F -pgmF hspec-discover #-} diff --git a/tests/StreamCipher/ChaChaPoly1305Spec.hs b/tests/StreamCipher/ChaChaPoly1305Spec.hs new file mode 100644 --- /dev/null +++ b/tests/StreamCipher/ChaChaPoly1305Spec.hs @@ -0,0 +1,167 @@ +{-# LANGUAGE OverloadedStrings #-} + +module StreamCipher.ChaChaPoly1305Spec where + +import qualified Crypto.Cipher.ChaChaPoly1305 as CP +import Crypto.Cipher.Types +import Crypto.Error +import Imports +import MAC.Poly1305Spec () + +import qualified Data.ByteArray as B (convert) +import qualified Data.ByteString as B + +plaintext + , aad + , key + , iv + , ivX + , ciphertext + , ciphertextX + , tag + , tagX + , nonce1 + , nonce2 + , nonce3 + , nonce4 + , nonce5 + , nonce6 + , nonce7 + , nonce8 + , nonce9 + , nonce10 + :: B.ByteString +plaintext = + "Ladies and Gentlemen of the class of '99: If I could offer you only one tip for the future, sunscreen would be it." +aad = "\x50\x51\x52\x53\xc0\xc1\xc2\xc3\xc4\xc5\xc6\xc7" +key = + "\x80\x81\x82\x83\x84\x85\x86\x87\x88\x89\x8a\x8b\x8c\x8d\x8e\x8f\x90\x91\x92\x93\x94\x95\x96\x97\x98\x99\x9a\x9b\x9c\x9d\x9e\x9f" +iv = "\x40\x41\x42\x43\x44\x45\x46\x47" +ivX = B.pack [0x40 .. 0x57] +constant = "\x07\x00\x00\x00" +ciphertext = + "\xd3\x1a\x8d\x34\x64\x8e\x60\xdb\x7b\x86\xaf\xbc\x53\xef\x7e\xc2\xa4\xad\xed\x51\x29\x6e\x08\xfe\xa9\xe2\xb5\xa7\x36\xee\x62\xd6\x3d\xbe\xa4\x5e\x8c\xa9\x67\x12\x82\xfa\xfb\x69\xda\x92\x72\x8b\x1a\x71\xde\x0a\x9e\x06\x0b\x29\x05\xd6\xa5\xb6\x7e\xcd\x3b\x36\x92\xdd\xbd\x7f\x2d\x77\x8b\x8c\x98\x03\xae\xe3\x28\x09\x1b\x58\xfa\xb3\x24\xe4\xfa\xd6\x75\x94\x55\x85\x80\x8b\x48\x31\xd7\xbc\x3f\xf4\xde\xf0\x8e\x4b\x7a\x9d\xe5\x76\xd2\x65\x86\xce\xc6\x4b\x61\x16" +ciphertextX = + "\xbd\x6d\x17\x9d\x3e\x83\xd4\x3b\x95\x76\x57\x94\x93\xc0\xe9\x39\x57\x2a\x17\x00\x25\x2b\xfa\xcc\xbe\xd2\x90\x2c\x21\x39\x6c\xbb\x73\x1c\x7f\x1b\x0b\x4a\xa6\x44\x0b\xf3\xa8\x2f\x4e\xda\x7e\x39\xae\x64\xc6\x70\x8c\x54\xc2\x16\xcb\x96\xb7\x2e\x12\x13\xb4\x52\x2f\x8c\x9b\xa4\x0d\xb5\xd9\x45\xb1\x1b\x69\xb9\x82\xc1\xbb\x9e\x3f\x3f\xac\x2b\xc3\x69\x48\x8f\x76\xb2\x38\x35\x65\xd3\xff\xf9\x21\xf9\x66\x4c\x97\x63\x7d\xa9\x76\x88\x12\xf6\x15\xc6\x8b\x13\xb5\x2e" +tag = "\x1a\xe1\x0b\x59\x4f\x09\xe2\x6a\x7e\x90\x2e\xcb\xd0\x60\x06\x91" +tagX = "\xc0\x87\x59\x24\xc1\xc7\x98\x79\x47\xde\xaf\xd8\x78\x0a\xcf\x49" +nonce1 = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" +nonce2 = "\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" +nonce3 = "\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" +nonce4 = "\x00\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" +nonce5 = "\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff" +nonce6 = "\x00\x00\x00\x00\x00\x00\x00\x00" +nonce7 = "\x01\x00\x00\x00\x00\x00\x00\x00" +nonce8 = "\xff\x00\x00\x00\x00\x00\x00\x00" +nonce9 = "\x00\x01\x00\x00\x00\x00\x00\x00" +nonce10 = "\xff\xff\xff\xff\xff\xff\xff\xff" + +a5key :: ByteString +a5key = + "\x1c\x92\x40\xa5\xeb\x55\xd3\x8a\xf3\x33\x88\x86\x04\xf6\xb5\xf0\x47\x39\x17\xc1\x40\x2b\x80\x09\x9d\xca\x5c\xbc\x20\x70\x75\xc0" + +a5nonce :: ByteString +a5nonce = "\x00\x00\x00\x00\x01\x02\x03\x04\x05\x06\x07\x08" + +a5aad :: ByteString +a5aad = "\xf3\x33\x88\x86\x00\x00\x00\x00\x00\x00\x4e\x91" + +a5cipher :: ByteString +a5cipher = + "\x64\xa0\x86\x15\x75\x86\x1a\xf4\x60\xf0\x62\xc7\x9b\xe6\x43\xbd\x5e\x80\x5c\xfd\x34\x5c\xf3\x89\xf1\x08\x67\x0a\xc7\x6c\x8c\xb2\x4c\x6c\xfc\x18\x75\x5d\x43\xee\xa0\x9e\xe9\x4e\x38\x2d\x26\xb0\xbd\xb7\xb7\x3c\x32\x1b\x01\x00\xd4\xf0\x3b\x7f\x35\x58\x94\xcf\x33\x2f\x83\x0e\x71\x0b\x97\xce\x98\xc8\xa8\x4a\xbd\x0b\x94\x81\x14\xad\x17\x6e\x00\x8d\x33\xbd\x60\xf9\x82\xb1\xff\x37\xc8\x55\x97\x97\xa0\x6e\xf4\xf0\xef\x61\xc1\x86\x32\x4e\x2b\x35\x06\x38\x36\x06\x90\x7b\x6a\x7c\x02\xb0\xf9\xf6\x15\x7b\x53\xc8\x67\xe4\xb9\x16\x6c\x76\x7b\x80\x4d\x46\xa5\x9b\x52\x16\xcd\xe7\xa4\xe9\x90\x40\xc5\xa4\x04\x33\x22\x5e\xe2\x82\xa1\xb0\xa0\x6c\x52\x3e\xaf\x45\x34\xd7\xf8\x3f\xa1\x15\x5b\x00\x47\x71\x8c\xbc\x54\x6a\x0d\x07\x2b\x04\xb3\x56\x4e\xea\x1b\x42\x22\x73\xf5\x48\x27\x1a\x0b\xb2\x31\x60\x53\xfa\x76\x99\x19\x55\xeb\xd6\x31\x59\x43\x4e\xce\xbb\x4e\x46\x6d\xae\x5a\x10\x73\xa6\x72\x76\x27\x09\x7a\x10\x49\xe6\x17\xd9\x1d\x36\x10\x94\xfa\x68\xf0\xff\x77\x98\x71\x30\x30\x5b\xea\xba\x2e\xda\x04\xdf\x99\x7b\x71\x4d\x6c\x6f\x2c\x29\xa6\xad\x5c\xb4\x02\x2b\x02\x70\x9b" + +a5plain :: ByteString +a5plain = + "\x49\x6e\x74\x65\x72\x6e\x65\x74\x2d\x44\x72\x61\x66\x74\x73\x20\x61\x72\x65\x20\x64\x72\x61\x66\x74\x20\x64\x6f\x63\x75\x6d\x65\x6e\x74\x73\x20\x76\x61\x6c\x69\x64\x20\x66\x6f\x72\x20\x61\x20\x6d\x61\x78\x69\x6d\x75\x6d\x20\x6f\x66\x20\x73\x69\x78\x20\x6d\x6f\x6e\x74\x68\x73\x20\x61\x6e\x64\x20\x6d\x61\x79\x20\x62\x65\x20\x75\x70\x64\x61\x74\x65\x64\x2c\x20\x72\x65\x70\x6c\x61\x63\x65\x64\x2c\x20\x6f\x72\x20\x6f\x62\x73\x6f\x6c\x65\x74\x65\x64\x20\x62\x79\x20\x6f\x74\x68\x65\x72\x20\x64\x6f\x63\x75\x6d\x65\x6e\x74\x73\x20\x61\x74\x20\x61\x6e\x79\x20\x74\x69\x6d\x65\x2e\x20\x49\x74\x20\x69\x73\x20\x69\x6e\x61\x70\x70\x72\x6f\x70\x72\x69\x61\x74\x65\x20\x74\x6f\x20\x75\x73\x65\x20\x49\x6e\x74\x65\x72\x6e\x65\x74\x2d\x44\x72\x61\x66\x74\x73\x20\x61\x73\x20\x72\x65\x66\x65\x72\x65\x6e\x63\x65\x20\x6d\x61\x74\x65\x72\x69\x61\x6c\x20\x6f\x72\x20\x74\x6f\x20\x63\x69\x74\x65\x20\x74\x68\x65\x6d\x20\x6f\x74\x68\x65\x72\x20\x74\x68\x61\x6e\x20\x61\x73\x20\x2f\xe2\x80\x9c\x77\x6f\x72\x6b\x20\x69\x6e\x20\x70\x72\x6f\x67\x72\x65\x73\x73\x2e\x2f\xe2\x80\x9d" + +a5tag :: ByteString +a5tag = "\xee\xad\x9d\x67\x89\x0c\xbb\x22\x39\x23\x36\xfe\xa1\x85\x1f\x38" + +rfc8439encrypt = ct `shouldBe` a5cipher + where + ct = case CP.aeadChacha20poly1305Init a5key a5nonce of + CryptoPassed st -> snd $ aeadSimpleEncrypt st a5aad a5plain 16 + _ -> "dummy" + +rfc8439decrypt = mpt `shouldBe` Just a5plain + where + mpt = case CP.aeadChacha20poly1305Init a5key a5nonce of + CryptoPassed st -> aeadSimpleDecrypt st a5aad a5cipher (AuthTag $ B.convert a5tag) + _ -> Nothing + +spec :: Spec +spec = do + it "V1" runEncrypt + it "V1-decrypt" runDecrypt + it "V1-extended" runEncryptX + it "V1-extended-decrypt" runDecryptX + it "nonce increment" runNonceInc + it "RFC8439 A5 enc" rfc8439encrypt + it "RFC8439 A5 dec" rfc8439decrypt + where + runEncrypt = + let ini = + throwCryptoError $ + CP.initialize key (throwCryptoError $ CP.nonce8 constant iv) + afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) + (out, afterEncrypt) = CP.encrypt plaintext afterAAD + outtag = CP.finalize afterEncrypt + in propertyHoldCase + [ eqTest "ciphertext" ciphertext out + , eqTest "tag" tag (B.convert outtag) + ] + runEncryptX = + let ini = + throwCryptoError $ CP.initializeX key (throwCryptoError $ CP.nonce24 ivX) + afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) + (out, afterEncrypt) = CP.encrypt plaintext afterAAD + outtag = CP.finalize afterEncrypt + in propertyHoldCase + [ eqTest "ciphertext" ciphertextX out + , eqTest "tag" tagX (B.convert outtag) + ] + + runDecrypt = + let ini = + throwCryptoError $ + CP.initialize key (throwCryptoError $ CP.nonce8 constant iv) + afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) + (out, afterDecrypt) = CP.decrypt ciphertext afterAAD + outtag = CP.finalize afterDecrypt + in propertyHoldCase + [ eqTest "plaintext" plaintext out + , eqTest "tag" tag (B.convert outtag) + ] + + runDecryptX = + let ini = + throwCryptoError $ CP.initializeX key (throwCryptoError $ CP.nonce24 ivX) + afterAAD = CP.finalizeAAD (CP.appendAAD aad ini) + (out, afterDecrypt) = CP.decrypt ciphertextX afterAAD + outtag = CP.finalize afterDecrypt + in propertyHoldCase + [ eqTest "plaintext" plaintext out + , eqTest "tag" tagX (B.convert outtag) + ] + + runNonceInc = + let n1 = throwCryptoError . CP.nonce12 $ nonce1 + n3 = throwCryptoError . CP.nonce12 $ nonce3 + n5 = throwCryptoError . CP.nonce12 $ nonce5 + n6 = throwCryptoError . CP.nonce8 constant $ nonce6 + n8 = throwCryptoError . CP.nonce8 constant $ nonce8 + n10 = throwCryptoError . CP.nonce8 constant $ nonce10 + in propertyHoldCase + [ eqTest "nonce12a" nonce2 $ B.convert . CP.incrementNonce $ n1 + , eqTest "nonce12b" nonce4 $ B.convert . CP.incrementNonce $ n3 + , eqTest "nonce12c" nonce1 $ B.convert . CP.incrementNonce $ n5 + , eqTest "nonce8a" (B.concat [constant, nonce7]) $ + B.convert . CP.incrementNonce $ + n6 + , eqTest "nonce8b" (B.concat [constant, nonce9]) $ + B.convert . CP.incrementNonce $ + n8 + , eqTest "nonce8c" (B.concat [constant, nonce6]) $ + B.convert . CP.incrementNonce $ + n10 + ] diff --git a/tests/StreamCipher/ChaChaSpec.hs b/tests/StreamCipher/ChaChaSpec.hs new file mode 100644 --- /dev/null +++ b/tests/StreamCipher/ChaChaSpec.hs @@ -0,0 +1,539 @@ +{-# LANGUAGE OverloadedStrings #-} + +module StreamCipher.ChaChaSpec (spec) where + +import qualified Crypto.Cipher.ChaCha as ChaCha +import Crypto.Hash (Digest, SHA256, hash) +import qualified Data.ByteArray as BA +import Imports + +import qualified Data.ByteString as B + +b8_128_k0_i0 = + "\xe2\x8a\x5f\xa4\xa6\x7f\x8c\x5d\xef\xed\x3e\x6f\xb7\x30\x34\x86\xaa\x84\x27\xd3\x14\x19\xa7\x29\x57\x2d\x77\x79\x53\x49\x11\x20\xb6\x4a\xb8\xe7\x2b\x8d\xeb\x85\xcd\x6a\xea\x7c\xb6\x08\x9a\x10\x18\x24\xbe\xeb\x08\x81\x4a\x42\x8a\xab\x1f\xa2\xc8\x16\x08\x1b\x8a\x26\xaf\x44\x8a\x1b\xa9\x06\x36\x8f\xd8\xc8\x38\x31\xc1\x8c\xec\x8c\xed\x81\x1a\x02\x8e\x67\x5b\x8d\x2b\xe8\xfc\xe0\x81\x16\x5c\xea\xe9\xf1\xd1\xb7\xa9\x75\x49\x77\x49\x48\x05\x69\xce\xb8\x3d\xe6\xa0\xa5\x87\xd4\x98\x4f\x19\x92\x5f\x5d\x33\x8e\x43\x0d" + +b12_128_k0_i0 = + "\xe1\x04\x7b\xa9\x47\x6b\xf8\xff\x31\x2c\x01\xb4\x34\x5a\x7d\x8c\xa5\x79\x2b\x0a\xd4\x67\x31\x3f\x1d\xc4\x12\xb5\xfd\xce\x32\x41\x0d\xea\x8b\x68\xbd\x77\x4c\x36\xa9\x20\xf0\x92\xa0\x4d\x3f\x95\x27\x4f\xbe\xff\x97\xbc\x84\x91\xfc\xef\x37\xf8\x59\x70\xb4\x50\x1d\x43\xb6\x1a\x8f\x7e\x19\xfc\xed\xde\xf3\x68\xae\x6b\xfb\x11\x10\x1b\xd9\xfd\x3e\x4d\x12\x7d\xe3\x0d\xb2\xdb\x1b\x47\x2e\x76\x42\x68\x03\xa4\x5e\x15\xb9\x62\x75\x19\x86\xef\x1d\x9d\x50\xf5\x98\xa5\xdc\xdc\x9f\xa5\x29\xa2\x83\x57\x99\x1e\x78\x4e\xa2\x0f" + +b20_128_k0_i0 = + "\x89\x67\x09\x52\x60\x83\x64\xfd\x00\xb2\xf9\x09\x36\xf0\x31\xc8\xe7\x56\xe1\x5d\xba\x04\xb8\x49\x3d\x00\x42\x92\x59\xb2\x0f\x46\xcc\x04\xf1\x11\x24\x6b\x6c\x2c\xe0\x66\xbe\x3b\xfb\x32\xd9\xaa\x0f\xdd\xfb\xc1\x21\x23\xd4\xb9\xe4\x4f\x34\xdc\xa0\x5a\x10\x3f\x6c\xd1\x35\xc2\x87\x8c\x83\x2b\x58\x96\xb1\x34\xf6\x14\x2a\x9d\x4d\x8d\x0d\x8f\x10\x26\xd2\x0a\x0a\x81\x51\x2c\xbc\xe6\xe9\x75\x8a\x71\x43\xd0\x21\x97\x80\x22\xa3\x84\x14\x1a\x80\xce\xa3\x06\x2f\x41\xf6\x7a\x75\x2e\x66\xad\x34\x11\x98\x4c\x78\x7e\x30\xad" + +b8_256_k0_i0 = + "\x3e\x00\xef\x2f\x89\x5f\x40\xd6\x7f\x5b\xb8\xe8\x1f\x09\xa5\xa1\x2c\x84\x0e\xc3\xce\x9a\x7f\x3b\x18\x1b\xe1\x88\xef\x71\x1a\x1e\x98\x4c\xe1\x72\xb9\x21\x6f\x41\x9f\x44\x53\x67\x45\x6d\x56\x19\x31\x4a\x42\xa3\xda\x86\xb0\x01\x38\x7b\xfd\xb8\x0e\x0c\xfe\x42\xd2\xae\xfa\x0d\xea\xa5\xc1\x51\xbf\x0a\xdb\x6c\x01\xf2\xa5\xad\xc0\xfd\x58\x12\x59\xf9\xa2\xaa\xdc\xf2\x0f\x8f\xd5\x66\xa2\x6b\x50\x32\xec\x38\xbb\xc5\xda\x98\xee\x0c\x6f\x56\x8b\x87\x2a\x65\xa0\x8a\xbf\x25\x1d\xeb\x21\xbb\x4b\x56\xe5\xd8\x82\x1e\x68\xaa" + +b12_256_k0_i0 = + "\x9b\xf4\x9a\x6a\x07\x55\xf9\x53\x81\x1f\xce\x12\x5f\x26\x83\xd5\x04\x29\xc3\xbb\x49\xe0\x74\x14\x7e\x00\x89\xa5\x2e\xae\x15\x5f\x05\x64\xf8\x79\xd2\x7a\xe3\xc0\x2c\xe8\x28\x34\xac\xfa\x8c\x79\x3a\x62\x9f\x2c\xa0\xde\x69\x19\x61\x0b\xe8\x2f\x41\x13\x26\xbe\x0b\xd5\x88\x41\x20\x3e\x74\xfe\x86\xfc\x71\x33\x8c\xe0\x17\x3d\xc6\x28\xeb\xb7\x19\xbd\xcb\xcc\x15\x15\x85\x21\x4c\xc0\x89\xb4\x42\x25\x8d\xcd\xa1\x4c\xf1\x11\xc6\x02\xb8\x97\x1b\x8c\xc8\x43\xe9\x1e\x46\xca\x90\x51\x51\xc0\x27\x44\xa6\xb0\x17\xe6\x93\x16" + +b20_256_k0_i0 = + "\x76\xb8\xe0\xad\xa0\xf1\x3d\x90\x40\x5d\x6a\xe5\x53\x86\xbd\x28\xbd\xd2\x19\xb8\xa0\x8d\xed\x1a\xa8\x36\xef\xcc\x8b\x77\x0d\xc7\xda\x41\x59\x7c\x51\x57\x48\x8d\x77\x24\xe0\x3f\xb8\xd8\x4a\x37\x6a\x43\xb8\xf4\x15\x18\xa1\x1c\xc3\x87\xb6\x69\xb2\xee\x65\x86\x9f\x07\xe7\xbe\x55\x51\x38\x7a\x98\xba\x97\x7c\x73\x2d\x08\x0d\xcb\x0f\x29\xa0\x48\xe3\x65\x69\x12\xc6\x53\x3e\x32\xee\x7a\xed\x29\xb7\x21\x76\x9c\xe6\x4e\x43\xd5\x71\x33\xb0\x74\xd8\x39\xd5\x31\xed\x1f\x28\x51\x0a\xfb\x45\xac\xe1\x0a\x1f\x4b\x79\x4d\x6f" + +-- XChaCha20 test vector from RFC draft: https://datatracker.ietf.org/doc/html/draft-arciszewski-xchacha + +xChaCha20_ExampleKAT = fst (ChaCha.combine initState plaintext) `shouldBe` expected + where + iv = B.pack $ [0x40 .. 0x56] ++ [0x58] + key = B.pack [0x80 .. 0x9f] + initState = ChaCha.initializeX 20 key iv + plaintext :: B.ByteString + plaintext = + "The dhole (pronounced \"dole\") is also known as the Asiatic wild dog, red dog, and whistling dog. It is about the size of a German shepherd but looks more like a long-legged fox. This highly elusive and skilled jumper is classified with wolves, coyotes, jackals, and foxes in the taxonomic family Canidae." + expected :: B.ByteString + expected = + "\x45\x59\xab\xba\x4e\x48\xc1\x61\x02\xe8\xbb\x2c\x05\xe6\x94\x7f\x50\xa7\x86\xde\x16\x2f\x9b\x0b\x7e\x59\x2a\x9b\x53\xd0\xd4\xe9\x8d\x8d\x64\x10\xd5\x40\xa1\xa6\x37\x5b\x26\xd8\x0d\xac\xe4\xfa\xb5\x23\x84\xc7\x31\xac\xbf\x16\xa5\x92\x3c\x0c\x48\xd3\x57\x5d\x4d\x0d\x2c\x67\x3b\x66\x6f\xaa\x73\x10\x61\x27\x77\x01\x09\x3a\x6b\xf7\xa1\x58\xa8\x86\x42\x92\xa4\x1c\x48\xe3\xa9\xb4\xc0\xda\xec\xe0\xf8\xd9\x8d\x0d\x7e\x05\xb3\x7a\x30\x7b\xbb\x66\x33\x31\x64\xec\x9e\x1b\x24\xea\x0d\x6c\x3f\xfd\xdc\xec\x4f\x68\xe7\x44\x30\x56\x19\x3a\x03\xc8\x10\xe1\x13\x44\xca\x06\xd8\xed\x8a\x2b\xfb\x1e\x8d\x48\xcf\xa6\xbc\x0e\xb4\xe2\x46\x4b\x74\x81\x42\x40\x7c\x9f\x43\x1a\xee\x76\x99\x60\xe1\x5b\xa8\xb9\x68\x90\x46\x6e\xf2\x45\x75\x99\x85\x23\x85\xc6\x61\xf7\x52\xce\x20\xf9\xda\x0c\x09\xab\x6b\x19\xdf\x74\xe7\x6a\x95\x96\x74\x46\xf8\xd0\xfd\x41\x5e\x7b\xee\x2a\x12\xa1\x14\xc2\x0e\xb5\x29\x2a\xe7\xa3\x49\xae\x57\x78\x20\xd5\x52\x0a\x1f\x3f\xb6\x2a\x17\xce\x6a\x7e\x68\xfa\x7c\x79\x11\x1d\x88\x60\x92\x0b\xc0\x48\xef\x43\xfe\x84\x48\x6c\xcb\x87\xc2\x5f\x0a\xe0\x45\xf0\xcc\xe1\xe7\x98\x9a\x9a\xa2\x20\xa2\x8b\xdd\x48\x27\xe7\x51\xa2\x4a\x6d\x5c\x62\xd7\x90\xa6\x63\x93\xb9\x31\x11\xc1\xa5\x5d\xd7\x42\x1a\x10\x18\x49\x74\xc7\xc5" + +rfc8439A2_1 = cipher' `shouldBe` cipher + where + key :: ByteString + key = + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + nonce :: ByteString + nonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + plain :: ByteString + plain = + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00" + cipher :: ByteString + cipher = + "\x76\xb8\xe0\xad\xa0\xf1\x3d\x90\x40\x5d\x6a\xe5\x53\x86\xbd\x28\xbd\xd2\x19\xb8\xa0\x8d\xed\x1a\xa8\x36\xef\xcc\x8b\x77\x0d\xc7\xda\x41\x59\x7c\x51\x57\x48\x8d\x77\x24\xe0\x3f\xb8\xd8\x4a\x37\x6a\x43\xb8\xf4\x15\x18\xa1\x1c\xc3\x87\xb6\x69\xb2\xee\x65\x86" + cipher' = fst $ ChaCha.combine (ChaCha.initialize 20 key nonce) plain + +rfc8439A2_2 = cipher' `shouldBe` cipher + where + key :: ByteString + key = + "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01" + nonce :: ByteString + nonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02" + plain :: ByteString + plain = + "\x41\x6e\x79\x20\x73\x75\x62\x6d\x69\x73\x73\x69\x6f\x6e\x20\x74\x6f\x20\x74\x68\x65\x20\x49\x45\x54\x46\x20\x69\x6e\x74\x65\x6e\x64\x65\x64\x20\x62\x79\x20\x74\x68\x65\x20\x43\x6f\x6e\x74\x72\x69\x62\x75\x74\x6f\x72\x20\x66\x6f\x72\x20\x70\x75\x62\x6c\x69\x63\x61\x74\x69\x6f\x6e\x20\x61\x73\x20\x61\x6c\x6c\x20\x6f\x72\x20\x70\x61\x72\x74\x20\x6f\x66\x20\x61\x6e\x20\x49\x45\x54\x46\x20\x49\x6e\x74\x65\x72\x6e\x65\x74\x2d\x44\x72\x61\x66\x74\x20\x6f\x72\x20\x52\x46\x43\x20\x61\x6e\x64\x20\x61\x6e\x79\x20\x73\x74\x61\x74\x65\x6d\x65\x6e\x74\x20\x6d\x61\x64\x65\x20\x77\x69\x74\x68\x69\x6e\x20\x74\x68\x65\x20\x63\x6f\x6e\x74\x65\x78\x74\x20\x6f\x66\x20\x61\x6e\x20\x49\x45\x54\x46\x20\x61\x63\x74\x69\x76\x69\x74\x79\x20\x69\x73\x20\x63\x6f\x6e\x73\x69\x64\x65\x72\x65\x64\x20\x61\x6e\x20\x22\x49\x45\x54\x46\x20\x43\x6f\x6e\x74\x72\x69\x62\x75\x74\x69\x6f\x6e\x22\x2e\x20\x53\x75\x63\x68\x20\x73\x74\x61\x74\x65\x6d\x65\x6e\x74\x73\x20\x69\x6e\x63\x6c\x75\x64\x65\x20\x6f\x72\x61\x6c\x20\x73\x74\x61\x74\x65\x6d\x65\x6e\x74\x73\x20\x69\x6e\x20\x49\x45\x54\x46\x20\x73\x65\x73\x73\x69\x6f\x6e\x73\x2c\x20\x61\x73\x20\x77\x65\x6c\x6c\x20\x61\x73\x20\x77\x72\x69\x74\x74\x65\x6e\x20\x61\x6e\x64\x20\x65\x6c\x65\x63\x74\x72\x6f\x6e\x69\x63\x20\x63\x6f\x6d\x6d\x75\x6e\x69\x63\x61\x74\x69\x6f\x6e\x73\x20\x6d\x61\x64\x65\x20\x61\x74\x20\x61\x6e\x79\x20\x74\x69\x6d\x65\x20\x6f\x72\x20\x70\x6c\x61\x63\x65\x2c\x20\x77\x68\x69\x63\x68\x20\x61\x72\x65\x20\x61\x64\x64\x72\x65\x73\x73\x65\x64\x20\x74\x6f" + cipher :: ByteString + cipher = + "\xa3\xfb\xf0\x7d\xf3\xfa\x2f\xde\x4f\x37\x6c\xa2\x3e\x82\x73\x70\x41\x60\x5d\x9f\x4f\x4f\x57\xbd\x8c\xff\x2c\x1d\x4b\x79\x55\xec\x2a\x97\x94\x8b\xd3\x72\x29\x15\xc8\xf3\xd3\x37\xf7\xd3\x70\x05\x0e\x9e\x96\xd6\x47\xb7\xc3\x9f\x56\xe0\x31\xca\x5e\xb6\x25\x0d\x40\x42\xe0\x27\x85\xec\xec\xfa\x4b\x4b\xb5\xe8\xea\xd0\x44\x0e\x20\xb6\xe8\xdb\x09\xd8\x81\xa7\xc6\x13\x2f\x42\x0e\x52\x79\x50\x42\xbd\xfa\x77\x73\xd8\xa9\x05\x14\x47\xb3\x29\x1c\xe1\x41\x1c\x68\x04\x65\x55\x2a\xa6\xc4\x05\xb7\x76\x4d\x5e\x87\xbe\xa8\x5a\xd0\x0f\x84\x49\xed\x8f\x72\xd0\xd6\x62\xab\x05\x26\x91\xca\x66\x42\x4b\xc8\x6d\x2d\xf8\x0e\xa4\x1f\x43\xab\xf9\x37\xd3\x25\x9d\xc4\xb2\xd0\xdf\xb4\x8a\x6c\x91\x39\xdd\xd7\xf7\x69\x66\xe9\x28\xe6\x35\x55\x3b\xa7\x6c\x5c\x87\x9d\x7b\x35\xd4\x9e\xb2\xe6\x2b\x08\x71\xcd\xac\x63\x89\x39\xe2\x5e\x8a\x1e\x0e\xf9\xd5\x28\x0f\xa8\xca\x32\x8b\x35\x1c\x3c\x76\x59\x89\xcb\xcf\x3d\xaa\x8b\x6c\xcc\x3a\xaf\x9f\x39\x79\xc9\x2b\x37\x20\xfc\x88\xdc\x95\xed\x84\xa1\xbe\x05\x9c\x64\x99\xb9\xfd\xa2\x36\xe7\xe8\x18\xb0\x4b\x0b\xc3\x9c\x1e\x87\x6b\x19\x3b\xfe\x55\x69\x75\x3f\x88\x12\x8c\xc0\x8a\xaa\x9b\x63\xd1\xa1\x6f\x80\xef\x25\x54\xd7\x18\x9c\x41\x1f\x58\x69\xca\x52\xc5\xb8\x3f\xa3\x6f\xf2\x16\xb9\xc1\xd3\x00\x62\xbe\xbc\xfd\x2d\xc5\xbc\xe0\x91\x19\x34\xfd\xa7\x9a\x86\xf6\xe6\x98\xce\xd7\x59\xc3\xff\x9b\x64\x77\x33\x8f\x3d\xa4\xf9\xcd\x85\x14\xea\x99\x82\xcc\xaf\xb3\x41\xb2\x38\x4d\xd9\x02\xf3\xd1\xab\x7a\xc6\x1d\xd2\x9c\x6f\x21\xba\x5b\x86\x2f\x37\x30\xe3\x7c\xfd\xc4\xfd\x80\x6c\x22\xf2\x21" + cipher' = + fst $ + ChaCha.combine (ChaCha.setCounter32 1 (ChaCha.initialize 20 key nonce)) plain + +rfc8439A2_3 = cipher' `shouldBe` cipher + where + key :: ByteString + key = + "\x1c\x92\x40\xa5\xeb\x55\xd3\x8a\xf3\x33\x88\x86\x04\xf6\xb5\xf0\x47\x39\x17\xc1\x40\x2b\x80\x09\x9d\xca\x5c\xbc\x20\x70\x75\xc0" + nonce :: ByteString + nonce = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02" + plain :: ByteString + plain = + "\x27\x54\x77\x61\x73\x20\x62\x72\x69\x6c\x6c\x69\x67\x2c\x20\x61\x6e\x64\x20\x74\x68\x65\x20\x73\x6c\x69\x74\x68\x79\x20\x74\x6f\x76\x65\x73\x0a\x44\x69\x64\x20\x67\x79\x72\x65\x20\x61\x6e\x64\x20\x67\x69\x6d\x62\x6c\x65\x20\x69\x6e\x20\x74\x68\x65\x20\x77\x61\x62\x65\x3a\x0a\x41\x6c\x6c\x20\x6d\x69\x6d\x73\x79\x20\x77\x65\x72\x65\x20\x74\x68\x65\x20\x62\x6f\x72\x6f\x67\x6f\x76\x65\x73\x2c\x0a\x41\x6e\x64\x20\x74\x68\x65\x20\x6d\x6f\x6d\x65\x20\x72\x61\x74\x68\x73\x20\x6f\x75\x74\x67\x72\x61\x62\x65\x2e" + cipher :: ByteString + cipher = + "\x62\xe6\x34\x7f\x95\xed\x87\xa4\x5f\xfa\xe7\x42\x6f\x27\xa1\xdf\x5f\xb6\x91\x10\x04\x4c\x0d\x73\x11\x8e\xff\xa9\x5b\x01\xe5\xcf\x16\x6d\x3d\xf2\xd7\x21\xca\xf9\xb2\x1e\x5f\xb1\x4c\x61\x68\x71\xfd\x84\xc5\x4f\x9d\x65\xb2\x83\x19\x6c\x7f\xe4\xf6\x05\x53\xeb\xf3\x9c\x64\x02\xc4\x22\x34\xe3\x2a\x35\x6b\x3e\x76\x43\x12\xa6\x1a\x55\x32\x05\x57\x16\xea\xd6\x96\x25\x68\xf8\x7d\x3f\x3f\x77\x04\xc6\xa8\xd1\xbc\xd1\xbf\x4d\x50\xd6\x15\x4b\x6d\xa7\x31\xb1\x87\xb5\x8d\xfd\x72\x8a\xfa\x36\x75\x7a\x79\x7a\xc1\x88\xd1" + cipher' = + fst $ + ChaCha.combine (ChaCha.setCounter32 42 (ChaCha.initialize 20 key nonce)) plain + +data Vector + = Vector + Int -- rounds + ByteString -- key + ByteString -- nonce + deriving (Show, Eq) + +-- The key length decides which of the two sets of constants the state is +-- built from, the nonce length whether the counter is 32 or 64 bits wide, +-- and the vector code paths are entered only at 20 rounds, so the +-- properties below are worth running over all of them rather than over one +-- corner. +instance Arbitrary Vector where + arbitrary = + Vector + <$> elements [8, 12, 20] + <*> (arbitraryBS =<< elements [16, 32]) + <*> (arbitraryBS =<< elements [8, 12]) + +-- | The keystream at lengths either side of the boundaries the bulk loops +-- have -- 192 bytes, where the vector code starts, 320 where it takes five +-- blocks at a time, and 512 where it takes six -- and at counters that are +-- not zero. The expected values are the SHA-256 of the keystream, from +-- OpenSSL 3.6.4 through EVP. +longVectors :: [(Word32, Int, ByteString)] +longVectors = + [ + ( 0 + , 191 + , "\x00\xd8\x21\x8c\x32\x59\xc5\x2e\xce\xc5\x72\xbf\x80\x73\x10\x5e\xdd\x01\x9f\xce\x8f\xc0\x0c\x31\x92\xd1\x1c\x97\x88\x5e\xf3\x6a" + ) + , + ( 0 + , 192 + , "\x21\x3f\x43\x21\x3a\x5d\xf2\x19\xf3\x25\x4c\x50\x7e\x09\x91\x78\x1d\xa7\x3e\x36\xe0\x40\x56\x9c\x9a\x88\x94\x8d\x80\xf8\x82\xb0" + ) + , + ( 0 + , 193 + , "\xf9\xf3\x5b\xb4\xe7\x20\x6c\xd9\xb7\xd3\x9d\xbb\x73\x1d\x0a\xbf\x98\x4d\xbe\x20\x20\x69\x77\x30\xe9\x33\x5e\xf2\x47\x9e\x16\x27" + ) + , + ( 0 + , 255 + , "\x2b\x14\xaa\x0e\xba\xf8\x28\x96\xc0\x58\x9a\x7f\x96\x82\x30\x9c\x6c\x58\x16\x56\xc1\xcc\xb5\xcc\xd1\x3e\x3b\x41\xd4\x0c\xe0\x62" + ) + , + ( 0 + , 256 + , "\x0e\x92\x3a\x76\xc5\x25\x4f\x1e\xb3\x53\x29\xa7\x93\x79\x44\x6b\x72\x5c\x1b\x68\xc0\xf4\x23\x7a\xf5\x2f\xfd\x82\x1f\x72\xda\xe2" + ) + , + ( 0 + , 320 + , "\xf4\xd3\xef\xfe\x43\xec\x46\x78\xa0\x66\x80\x5a\x3e\xb7\x12\x74\x04\x6c\x5e\x30\x40\xf9\xaf\xc2\xe0\xc1\xe0\x52\x50\x36\x6d\xa5" + ) + , + ( 0 + , 383 + , "\x30\xab\x6f\x23\xd3\x55\xbc\x2b\x3b\xc3\x9f\x19\x0f\x1a\x58\x19\x19\x4a\x0e\x58\xbc\x50\xd3\x05\x22\x47\x9c\x42\x2e\x23\x8d\x67" + ) + , + ( 0 + , 384 + , "\x9f\x4c\x9e\x6f\x16\xf2\x2c\xce\xb1\x1c\x0c\x98\xa9\x64\xf1\x53\x47\xc0\x64\x67\xf6\xb4\x55\x80\x64\x35\xf1\x34\xa9\x77\x99\x90" + ) + , + ( 0 + , 447 + , "\x25\x40\xa9\xc0\x66\x18\x9e\x62\xb6\x14\xad\xf5\x00\x2c\x12\xdc\x03\x8a\x38\xb5\xe9\x43\x50\x10\x19\xf0\x16\xa7\x0b\xb4\x04\x7a" + ) + , + ( 0 + , 448 + , "\x39\xcc\x80\x48\xb9\x2b\x18\x88\xf5\xa3\x70\x06\xd9\x54\xa1\x33\xb0\x82\x9a\xff\x80\x34\xb2\x63\xfa\xf5\x9c\x18\xda\x4d\xed\x9c" + ) + , + ( 0 + , 511 + , "\x6d\xf5\xc5\x31\x38\xc8\x8e\x52\x5b\xe0\x26\x70\xfb\xb3\xef\x77\x50\xe5\x32\x7d\x5a\x21\xde\x3f\x79\x8e\x95\x0b\x23\x7f\x10\x12" + ) + , + ( 0 + , 512 + , "\xf0\x38\x11\xa6\x94\x7d\xe9\x42\x63\x49\x0b\x11\x32\x24\x0c\x7d\x46\xab\xd4\x64\x51\x13\x0d\x86\x04\x20\x29\xd0\xda\xe6\x83\x04" + ) + , + ( 0 + , 513 + , "\x6a\x4b\x18\xb8\x4e\x67\xca\xac\x64\xe7\x9d\xd0\x0a\x01\x97\x9a\x64\x95\xc4\xe8\x7f\xbd\xe9\xb4\x4a\x24\x8d\xd4\x12\x4b\x24\xf0" + ) + , + ( 0 + , 576 + , "\xe3\x17\xfc\x20\x12\x1d\x70\x36\x29\xec\xef\x79\x0e\xba\xf9\xdf\x9b\x09\x35\xc7\x7d\x6c\x88\x8f\x81\xcf\x98\xc7\x42\x88\xe5\x48" + ) + , + ( 0 + , 640 + , "\x73\x38\xfe\x4f\x7f\x36\xbb\x15\xb1\x40\x46\xa8\xab\x66\x66\x9c\x6c\x2e\x2a\x7e\x37\xa1\x58\xa0\x39\x27\xe3\xf9\x79\x9a\xd4\xe6" + ) + , + ( 0 + , 704 + , "\x8a\x38\x79\x32\xfd\x3f\xcb\xb1\xec\x0e\x4f\xe3\xdb\xd3\x31\x75\x74\xdb\x69\x20\xaa\xd3\x4b\x69\xa2\xe6\x4e\xcb\xfd\x12\xfb\x8b" + ) + , + ( 0 + , 1024 + , "\x45\x20\xf2\xe8\xee\x19\xee\x25\xcc\x50\x45\x03\x51\x65\x7b\x81\x25\xbe\x47\x27\x5d\x8e\x88\x5b\x6f\x76\xc5\x6d\x62\x68\x9d\x63" + ) + , + ( 0 + , 2048 + , "\xd0\x5c\xe9\x7f\x18\x46\x93\x0b\xd3\xa4\x5c\x15\xf2\xdf\xf7\x7f\x29\xcc\x01\xb5\x48\x55\x10\x41\x94\x1d\xc5\x1e\x2f\xc2\x26\x88" + ) + , + ( 0 + , 4096 + , "\x71\x05\xec\x3f\x33\xb9\xe9\x07\x05\x9c\xc5\x30\xec\x41\xb0\x09\xcd\x3d\x8a\xe6\x7b\x88\x3a\xeb\xf4\xea\x53\xee\x42\xea\x2d\x3c" + ) + , + ( 0 + , 8192 + , "\x05\x62\x0c\x5d\xc9\xc4\x23\xd2\xb8\x14\x46\xc3\xad\xbf\x17\x5a\xfb\x7c\xd0\xa5\xcc\xc2\x8e\x60\x68\x6b\x77\x97\x47\x44\x8d\x96" + ) + , + ( 0 + , 12288 + , "\x09\x67\xf7\x03\x07\x6f\xb6\xb5\xe7\xcd\xaf\x4b\x8c\xad\x66\x39\x41\x99\x72\xd3\x06\xae\x20\x88\xd6\x0c\xd6\x52\x1e\x7e\x5f\xc3" + ) + , + ( 1 + , 191 + , "\xf2\x7e\x48\x8b\x09\xfd\xf6\xb8\xbc\x30\xe1\xba\x49\xa0\xff\x51\x29\xe0\x8f\xf0\xfb\x47\x5f\xa4\xbf\x74\x08\x9d\x49\x0e\x04\x3c" + ) + , + ( 1 + , 192 + , "\x14\x36\x48\xb5\xbe\xa7\xf8\xb1\x5b\xfb\xaf\xc1\x61\x8e\x02\x19\x80\x4c\xfa\xdc\xa1\xc0\xde\xf4\x81\xc6\xdd\xfe\x95\x87\x12\x94" + ) + , + ( 1 + , 193 + , "\x31\x64\xcd\xc8\x37\x13\xe5\xbc\x3e\x47\x03\xb2\xa0\xac\x78\x79\xb7\x67\x06\xfd\x63\x70\xa8\x26\xb5\xeb\xfc\xb8\x2c\xb4\x30\x46" + ) + , + ( 1 + , 255 + , "\xff\xdb\x93\x0d\x04\x5d\xfb\xd0\xce\x68\x8a\xb9\x71\x11\xbc\xdd\x2e\xf0\x8e\xeb\xbf\xd1\x29\x4e\x86\x3f\x67\x3f\x7d\x4d\xd5\xd0" + ) + , + ( 1 + , 256 + , "\xa6\x91\xf4\x64\x38\x29\x5c\x67\x7a\xd3\x59\x54\x2c\x77\x89\x23\x13\x9b\x38\xda\x9d\x2c\x04\x83\x34\x38\x6d\x0e\x84\x43\x10\xae" + ) + , + ( 1 + , 320 + , "\xff\xf0\x79\x6f\x40\x95\xa6\x30\xfa\x3d\x93\x8c\x1a\xe1\x43\x4c\xfe\x70\x75\x80\xe5\x1e\x2b\x58\xf6\x20\xd0\x22\xa9\xbd\xfa\x0a" + ) + , + ( 1 + , 383 + , "\xed\xf0\x30\xf2\x34\x53\xb6\x41\x6d\xd1\xdf\x76\xce\xee\xda\x2a\x13\xf2\xb4\xb4\xee\x1f\xde\xc5\x3f\x05\xd2\x57\xe6\xed\x99\xd0" + ) + , + ( 1 + , 384 + , "\xb3\x87\x2d\x6b\xa8\x30\xd5\x3f\x08\xc8\x53\x89\x06\xea\xb1\xb2\x42\x90\xbd\x2d\x15\xa1\x64\xbd\xcf\xe2\xa9\x5e\xa4\x28\x98\xfc" + ) + , + ( 1 + , 447 + , "\x7c\x73\x0d\xdf\xae\x6d\x29\x16\x24\xdd\x99\x8a\xc1\x33\x7d\xaf\x13\x7b\x89\x6f\xad\x9f\x72\xeb\x7d\x1a\x49\xde\x5b\xee\x41\x82" + ) + , + ( 1 + , 448 + , "\x83\x8a\x55\x1e\x18\x0e\x39\x7e\x65\xee\x91\x1e\x30\xfb\x95\x74\xcc\x49\x45\xbd\x38\x2f\xb6\xc0\xb1\x0e\x0b\x79\x83\x73\xf9\x30" + ) + , + ( 1 + , 511 + , "\x63\x7a\x92\x5b\x5a\x98\x2d\xa9\xa6\xb6\x91\x6e\x79\x9e\x28\x6a\xc9\x4f\x3e\xe8\x50\x2f\xf7\xdb\xef\xe2\x02\x6f\x65\xd5\x0a\x84" + ) + , + ( 1 + , 512 + , "\x37\xa0\xb2\x08\x93\xb3\xd3\xf8\xcd\xfa\x76\xcb\x0a\x77\x99\x19\xb6\x80\x92\x89\x1a\x28\x32\x6b\x42\x96\x9e\xff\xbc\xbf\x80\x11" + ) + , + ( 1 + , 513 + , "\x60\x31\x68\x0c\x03\x89\xdd\x65\xb0\xfc\x1b\x8c\xa4\xec\x4f\x90\x7f\x25\xc0\x03\xf1\xde\x85\x77\xf4\x68\xc0\x4a\xe1\xf9\x00\x9b" + ) + , + ( 1 + , 576 + , "\xb0\xae\xd0\x02\xce\xa7\x61\xef\xf4\x48\x7d\x1f\x6f\x07\x17\x64\x0f\x72\x5b\xad\x2e\x1b\xc7\x6d\xf7\xee\x2b\xcf\xf4\xb4\x72\x5f" + ) + , + ( 1 + , 640 + , "\x82\xfc\x19\x96\x59\x37\x41\x8b\x6f\x12\x0d\xae\xfc\xa8\xce\xc7\x02\xbb\x7e\x6a\x66\x37\xf3\x8d\xc8\x86\xe6\x1d\x90\xb4\x4d\x67" + ) + , + ( 1 + , 704 + , "\xf5\x3c\x7c\xfa\x24\xee\x6f\x49\x79\x16\xdb\x8f\x72\x4a\x3d\xc8\x6c\x78\x42\x8b\xf0\x8d\xdd\x0d\x71\xff\x36\x74\xa6\xb7\x8a\xe6" + ) + , + ( 1 + , 1024 + , "\x23\xa1\xce\x21\x96\x72\x54\xa0\xeb\x3a\xbd\xbc\x24\xa2\xa9\x65\x93\x98\x74\x92\x6d\xe5\x56\xe7\xed\xa4\x17\x2b\xdc\x1c\x3f\x02" + ) + , + ( 1 + , 2048 + , "\x7d\x86\xa4\xb2\x82\x57\x78\x16\x38\x6f\x44\xed\x4a\x13\x50\x9c\x94\x12\x5d\x84\xf6\x3c\x09\x86\xf1\x94\x29\xe7\x10\x75\x54\xe5" + ) + , + ( 1 + , 4096 + , "\x4b\xd4\x83\xc4\x45\x2d\x54\x6a\x5b\x4e\xf8\xae\xcb\x87\x10\xfb\xda\xcd\x68\xbe\x95\xd0\x73\x44\xdb\xce\x25\x83\xd9\x6c\x72\xaf" + ) + , + ( 1 + , 8192 + , "\x43\x22\x12\x02\xd5\x94\x93\x26\x2f\xea\xc8\x82\xe2\xc2\x8a\x4c\x36\x10\xd3\x5c\x59\x82\x92\x27\x3a\x4f\x3a\x6c\x2a\x9a\x6f\xa2" + ) + , + ( 1 + , 12288 + , "\x00\x75\xab\x26\xfe\x8e\x9c\x2e\xcb\xa2\xcd\x16\x26\xd8\x88\x89\xe3\x9a\xeb\x2b\xb8\xbe\xb1\x7f\x46\x8a\x5a\x72\x38\x57\x88\xe4" + ) + , + ( 305419896 + , 191 + , "\x19\x27\x86\x98\xbf\xe8\x52\x42\x4a\x72\x76\xb8\xcf\x7f\x40\x2b\xf2\x41\x97\x39\xf4\x13\x7d\x62\x25\x8d\x5e\x32\x7b\x3e\x19\xe0" + ) + , + ( 305419896 + , 192 + , "\x6c\x17\x58\xd2\x31\xf9\x5f\x02\x82\x8a\x76\x32\x13\xd6\x9c\x32\xf0\x81\x6a\x6a\xe7\x43\x78\x19\xd9\x89\x8e\xda\x55\xa4\x7e\x06" + ) + , + ( 305419896 + , 193 + , "\x85\x4d\x53\xe9\x9a\x06\x20\x4e\x6e\x30\x5d\xa1\xab\x66\xc0\xb1\x85\xe7\x1b\xd6\xed\x1e\xa3\xd7\x13\x91\x28\x69\xb1\xf8\x6a\x32" + ) + , + ( 305419896 + , 255 + , "\x34\xb9\x16\x61\xca\xe0\xe7\x75\x55\x44\x54\x3d\x1f\xff\x93\xcb\x02\x89\xd1\x32\x48\xc1\x70\xa5\xa1\x6d\xfb\x07\xf4\xc2\x73\x6b" + ) + , + ( 305419896 + , 256 + , "\x36\x3b\x26\x10\x5c\xdc\x9d\xba\x59\xae\x8d\xe0\x92\xe5\xd9\xdc\x99\xd8\xa4\xa1\x68\xec\xfc\x1e\xce\x2a\x18\x8c\xb8\x56\xfb\xa9" + ) + , + ( 305419896 + , 320 + , "\x41\xb1\x29\x5e\x42\xe2\x3a\xfd\xdc\xeb\x09\xd2\x22\x62\xc0\xff\x44\x17\xa0\x54\xd7\x6e\x93\x77\xb4\xa1\xec\xe1\x62\x31\x5e\x43" + ) + , + ( 305419896 + , 383 + , "\xbc\xec\xe8\x44\x2d\x0d\x7c\x68\x67\x30\xa6\xbf\x42\x39\x75\x39\xda\x5c\x3b\xd0\x82\x98\xdf\xf1\xa6\x94\x3a\x1e\x45\xd0\x5e\x4b" + ) + , + ( 305419896 + , 384 + , "\xc4\xa0\x5c\x22\x3c\xed\xdf\x3c\xc8\x40\x9f\x15\x6e\x54\x7b\xaa\x46\x51\xa0\xf5\xbd\x5d\xba\x73\x86\xba\x41\x2f\x88\x21\x8c\x6a" + ) + , + ( 305419896 + , 447 + , "\xa6\x16\x3c\x57\xe3\xfc\x78\xe2\x1c\xaf\xef\x51\x81\xa3\x00\x2b\xfa\xe3\x3e\x97\x4c\xaf\x37\x6a\x3a\xb0\xe3\x31\x74\x97\x4d\x3b" + ) + , + ( 305419896 + , 448 + , "\xbf\xf8\xea\x06\x20\x99\xca\x63\x97\x88\xe3\xfe\xed\x9d\x59\x98\x52\xf2\xc0\x9e\xda\xb3\x91\xbd\x00\x05\x31\x0e\xc9\xa3\x5f\x7b" + ) + , + ( 305419896 + , 511 + , "\x38\x2c\x89\x65\x0f\x15\x33\x28\xb7\x07\xbc\xe0\x40\x28\x77\x75\x81\x3c\x77\x58\xe0\xa0\xe2\xb4\xf4\xd8\xb0\xce\xc6\x10\x45\x08" + ) + , + ( 305419896 + , 512 + , "\x87\x6f\xcc\xc1\xd9\x61\x56\xc5\x3c\x9d\xf0\x06\x8b\x12\xbf\x7c\xd5\x60\x2f\xa3\x20\x7b\x71\x3d\xdc\x30\x57\x74\x24\xa8\x2e\x18" + ) + , + ( 305419896 + , 513 + , "\x73\x1c\x5b\x67\x29\xb5\xcf\x59\xd1\x6e\x28\x2c\x8d\x99\x6f\xad\xbc\x42\xd7\xf7\xbd\x1b\x05\xc2\x04\x4d\x4e\x16\x52\x04\x27\x88" + ) + , + ( 305419896 + , 576 + , "\x1c\x63\x4a\x40\xcc\x65\x66\x57\x62\x52\xa3\xdf\x2a\x5a\x24\x01\xaa\x12\xf9\xf7\xa7\xf1\x13\xd5\xd4\x57\x26\x9b\xae\xf0\xc2\xc2" + ) + , + ( 305419896 + , 640 + , "\x31\x36\x49\xcc\x53\xba\x02\x7c\x1f\x73\xf8\x80\x75\xb9\x82\xef\xc2\x90\xaa\xb3\xba\x5a\x93\x1f\x59\x02\xf3\x1a\x86\xac\xe2\x7a" + ) + , + ( 305419896 + , 704 + , "\x02\xcb\x50\x33\xb9\x6d\x14\x20\x7f\xdb\xb9\x6a\x9c\x32\x75\x0f\x57\x4b\x3c\x8b\x77\xc7\x7e\x36\x2d\x86\x54\x6c\x3d\x34\x9c\xd9" + ) + , + ( 305419896 + , 1024 + , "\x47\x2e\x03\xd7\xec\x96\x19\xc5\x4e\x49\xa7\x9f\x4f\xb6\x1c\x3c\x33\xd9\x6b\x23\xb2\x61\x34\xf5\x24\x23\xcf\xe8\x00\x71\x89\xd6" + ) + , + ( 305419896 + , 2048 + , "\x55\xef\xa5\x8b\xb9\xbb\x9e\x94\xa3\x75\xb1\x40\x84\x4f\x55\x41\x12\xd0\x5f\x4f\x2d\xa3\xc8\xc7\x36\x78\x91\x4e\xba\x68\x91\xcb" + ) + , + ( 305419896 + , 4096 + , "\x2d\x31\xc3\x49\x9f\xc7\x6b\x99\x46\xf7\x8a\x64\xff\x7e\x70\x92\x83\x5f\x06\x03\xa7\xe3\x88\x83\x76\x2d\x3a\x78\x88\x4a\x11\x48" + ) + , + ( 305419896 + , 8192 + , "\x4c\xe7\x49\xa6\xd5\x80\x77\xd4\x9e\x9b\x77\xcd\x46\x33\x8c\x15\xae\x97\xc8\x63\x8e\x45\xcd\x40\x74\xe3\x92\x96\x7e\xab\xbb\xd0" + ) + , + ( 305419896 + , 12288 + , "\x47\x90\x36\xe5\x3d\xd1\x06\x37\x0e\x9d\xd6\x52\x7e\x78\x85\xd7\x4d\x9a\xb0\x4d\xc3\x13\xec\x8c\xc8\x08\xc6\xd2\x5a\xea\xc6\x3d" + ) + ] + +longKey :: ByteString +longKey = B.pack [fromIntegral (0x40 + i) | i <- [0 .. 31 :: Int]] + +longNonce :: ByteString +longNonce = B.pack [fromIntegral (0xf0 - i) | i <- [0 .. 11 :: Int]] + +longTests :: Spec +longTests = describe "long keystream" $ mapM_ test longVectors + where + test (counter, len, expected) = + it (show len ++ " bytes from counter " ++ show counter) $ do + digest (fst (ChaCha.generate st len) :: ByteString) `shouldBe` expected + digest (fst (ChaCha.combine st (B.replicate len 0)) :: ByteString) + `shouldBe` expected + where + st = + ChaCha.setCounter32 counter $ + ChaCha.initialize 20 longKey longNonce + digest bs = BA.convert (hash bs :: Digest SHA256) :: ByteString + +-- The 32-bit counter carries into the word above it, which the bulk loops +-- must not do on their own account, so a message that runs over the carry +-- has to come out the same as the same message taken a block at a time. +counterCarry :: Spec +counterCarry = + describe "counter carry" $ + mapM_ test [0xffffff00, 0xfffffff0, 0xfffffffe] + where + test counter = + it ("crossing 2^32 from " ++ show (counter :: Word32)) $ + fst (ChaCha.combine st (B.replicate len 0)) + `shouldBe` B.concat (blockAtATime len st) + where + len = 8192 + st = ChaCha.setCounter32 counter (ChaCha.initialize 20 longKey longNonce) + blockAtATime 0 _ = [] + blockAtATime n s = + let (c, next) = ChaCha.combine s (B.replicate (min 64 n) 0) + in c : blockAtATime (n - min 64 n) next + +spec :: Spec +spec = do + it "8-128-K0-I0" (chachaRunSimple b8_128_k0_i0 8 16 8) + it "12-128-K0-I0" (chachaRunSimple b12_128_k0_i0 12 16 8) + it "20-128-K0-I0" (chachaRunSimple b20_128_k0_i0 20 16 8) + it "8-256-K0-I0" (chachaRunSimple b8_256_k0_i0 8 32 8) + it "12-256-K0-I0" (chachaRunSimple b12_256_k0_i0 12 32 8) + it "20-256-K0-I0" (chachaRunSimple b20_256_k0_i0 20 32 8) + it "XChaCha20 example KAT" xChaCha20_ExampleKAT + it "RFC 8439 A2 #1 ChaCha20" rfc8439A2_1 + it "RFC 8439 A2 #2 ChaCha20" rfc8439A2_2 + it "RFC 8439 A2 #3 ChaCha20" rfc8439A2_3 + longTests + counterCarry + prop "generate-combine" chachaGenerateCombine + prop "chunking-generate" chachaGenerateChunks + prop "chunking-combine" chachaCombineChunks + where + chachaRunSimple expected rounds klen nonceLen = + let chacha = ChaCha.initialize rounds (B.replicate klen 0) (B.replicate nonceLen 0) + in fst (ChaCha.generate chacha (B.length expected)) `shouldBe` expected + + chachaGenerateChunks :: ChunkingLen -> Vector -> Bool + chachaGenerateChunks (ChunkingLen ckLen) (Vector rounds key iv) = + let initChaCha = ChaCha.initialize rounds key iv + nbBytes = 1048 + (expected, _) = ChaCha.generate initChaCha nbBytes + chunks = loop nbBytes ckLen initChaCha + in expected `propertyEq` B.concat chunks + where + loop n [] chacha = loop n ckLen chacha + loop 0 _ _ = [] + loop n (x : xs) chacha = + let len = min x n + (c, next) = ChaCha.generate chacha len + in c : loop (n - len) xs next + + chachaGenerateCombine :: ChunkingLen0_127 -> Vector -> Int0_2901 -> Bool + chachaGenerateCombine (ChunkingLen0_127 ckLen) (Vector rounds key iv) (Int0_2901 nbBytes) = + let initChaCha = ChaCha.initialize rounds key iv + in loop nbBytes ckLen initChaCha + where + loop n [] chacha = loop n ckLen chacha + loop 0 _ _ = True + loop n (x : xs) chacha = + let len = min x n + (c1, next) = ChaCha.generate chacha len + (c2, _) = ChaCha.combine chacha (B.replicate len 0) + in if c1 == c2 then loop (n - len) xs next else False + + chachaCombineChunks :: ChunkingLen0_127 -> Vector -> ArbitraryBS0_2901 -> Bool + chachaCombineChunks (ChunkingLen0_127 ckLen) (Vector rounds key iv) (ArbitraryBS0_2901 wholebs) = + let initChaCha = ChaCha.initialize rounds key iv + (expected, _) = ChaCha.combine initChaCha wholebs + chunks = loop wholebs ckLen initChaCha + in expected `propertyEq` B.concat chunks + where + loop bs [] chacha = loop bs ckLen chacha + loop bs (x : xs) chacha + | B.null bs = [] + | otherwise = + let (bs1, bs2) = B.splitAt (min x (B.length bs)) bs + (c, next) = ChaCha.combine chacha bs1 + in c : loop bs2 xs next diff --git a/tests/StreamCipher/RC4Spec.hs b/tests/StreamCipher/RC4Spec.hs new file mode 100644 --- /dev/null +++ b/tests/StreamCipher/RC4Spec.hs @@ -0,0 +1,42 @@ +{-# LANGUAGE OverloadedStrings #-} +{-# LANGUAGE ViewPatterns #-} + +module StreamCipher.RC4Spec where + +import Test.Hspec + +import qualified Crypto.Cipher.RC4 as RC4 +import Data.ByteString (ByteString) +import Data.ByteString.Char8 () + +-- taken from wikipedia pages +vectors :: [(ByteString, ByteString, ByteString)] +vectors = + [ + ( "Key" + , "Plaintext" + , "\xBB\xF3\x16\xE8\xD9\x40\xAF\x0A\xD3" + ) + , + ( "Wiki" + , "pedia" + , "\x10\x21\xBF\x04\x20" + ) + , + ( "Secret" + , "Attack at dawn" + , "\x45\xA0\x1F\x64\x5F\xC3\x5B\x38\x35\x52\x54\x4B\x9B\xF5" + ) + ] + +spec :: Spec +spec = + sequence_ $ + zipWith toKatTest is vectors + where + toKatTest i (key, plainText, cipherText) = + it + (show i) + (snd (RC4.combine (RC4.initialize key) plainText) `shouldBe` cipherText) + is :: [Int] + is = [1 ..] diff --git a/tests/StreamCipher/SalsaSpec.hs b/tests/StreamCipher/SalsaSpec.hs new file mode 100644 --- /dev/null +++ b/tests/StreamCipher/SalsaSpec.hs @@ -0,0 +1,133 @@ +{-# LANGUAGE OverloadedStrings #-} + +module StreamCipher.SalsaSpec (spec) where + +import qualified Crypto.Cipher.Salsa as Salsa +import qualified Data.ByteString as B + +import Imports + +type Vector = (Int, B.ByteString, B.ByteString, [(Int, B.ByteString)]) + +vectors :: [Vector] +vectors = + [ + ( 20 + , key + , iv + , + [ + ( 0 + , "\x99\xA8\xCC\xEC\x6C\x5B\x2A\x0B\x6E\x33\x6C\xB2\x06\x52\x24\x1C\x32\xB2\x4D\x34\xAC\xC0\x45\x7E\xF6\x79\x17\x8E\xDE\x7C\xF8\x05\x80\x5A\x93\x05\xC7\xC4\x99\x09\x68\x3B\xD1\xA8\x03\x32\x78\x17\x62\x7C\xA4\x6F\xE8\xB9\x29\xB6\xDF\x00\x12\xBD\x86\x41\x83\xBE" + ) + , + ( 192 + , "\x2D\x22\x6C\x11\xF4\x7B\x3C\x0C\xCD\x09\x59\xB6\x1F\x59\xD5\xCC\x30\xFC\xEF\x6D\xBB\x8C\xBB\x3D\xCC\x1C\xC2\x52\x04\xFC\xD4\x49\x8C\x37\x42\x6A\x63\xBE\xA3\x28\x2B\x1A\x8A\x0D\x60\xE1\x3E\xB2\xFE\x59\x24\x1A\x9F\x6A\xF4\x26\x68\x98\x66\xED\xC7\x69\xE1\xE6\x48\x2F\xE1\xC1\x28\xA1\x5C\x11\x23\xB5\x65\x5E\xD5\x46\xDF\x01\x4C\xE0\xC4\x55\xDB\xF5\xD3\xA1\x3D\x9C\xD4\xF0\xE2\xD1\xDA\xB9\xF1\x2F\xB6\x8C\x54\x42\x61\xD7\xF8\x8E\xAC\x1C\x6C\xBF\x99\x3F\xBB\xB8\xE0\xAA\x85\x10\xBF\xF8\xE7\x38\x35\xA1\xE8\x6E\xAD\xBB" + ) + , + ( 448 + , "\x05\x97\x18\x8A\x1C\x19\x25\x57\x69\xBE\x1C\x21\x03\x99\xAD\x17\x2E\xB4\x6C\x52\xF9\x2F\xD5\x41\xDF\x2E\xAD\x71\xB1\xFF\x8E\xA7\xAD\xD3\x80\xEC\x71\xA5\xFD\x7A\xDB\x51\x81\xEA\xDD\x18\x25\xEC\x02\x77\x9A\x45\x09\xBE\x58\x32\x70\x8C\xA2\x83\x6C\x16\x93\xA5" + ) + ] + ) + , + ( 20 + , "\x00\x53\xA6\xF9\x4C\x9F\xF2\x45\x98\xEB\x3E\x91\xE4\x37\x8A\xDD\x30\x83\xD6\x29\x7C\xCF\x22\x75\xC8\x1B\x6E\xC1\x14\x67\xBA\x0D" + , "\x0D\x74\xDB\x42\xA9\x10\x77\xDE" + , + [ + ( 0 + , "\xF5\xFA\xD5\x3F\x79\xF9\xDF\x58\xC4\xAE\xA0\xD0\xED\x9A\x96\x01\xF2\x78\x11\x2C\xA7\x18\x0D\x56\x5B\x42\x0A\x48\x01\x96\x70\xEA\xF2\x4C\xE4\x93\xA8\x62\x63\xF6\x77\xB4\x6A\xCE\x19\x24\x77\x3D\x2B\xB2\x55\x71\xE1\xAA\x85\x93\x75\x8F\xC3\x82\xB1\x28\x0B\x71" + ) + , + ( 65472 + , "\xB7\x0C\x50\x13\x9C\x63\x33\x2E\xF6\xE7\x7A\xC5\x43\x38\xA4\x07\x9B\x82\xBE\xC9\xF9\xA4\x03\xDF\xEA\x82\x1B\x83\xF7\x86\x07\x91\x65\x0E\xF1\xB2\x48\x9D\x05\x90\xB1\xDE\x77\x2E\xED\xA4\xE3\xBC\xD6\x0F\xA7\xCE\x9C\xD6\x23\xD9\xD2\xFD\x57\x58\xB8\x65\x3E\x70\x81\x58\x2C\x65\xD7\x56\x2B\x80\xAE\xC2\xF1\xA6\x73\xA9\xD0\x1C\x9F\x89\x2A\x23\xD4\x91\x9F\x6A\xB4\x7B\x91\x54\xE0\x8E\x69\x9B\x41\x17\xD7\xC6\x66\x47\x7B\x60\xF8\x39\x14\x81\x68\x2F\x5D\x95\xD9\x66\x23\xDB\xC4\x89\xD8\x8D\xAA\x69\x56\xB9\xF0\x64\x6B\x6E" + ) + , + ( 131008 + , "\xA1\x3F\xFA\x12\x08\xF8\xBF\x50\x90\x08\x86\xFA\xAB\x40\xFD\x10\xE8\xCA\xA3\x06\xE6\x3D\xF3\x95\x36\xA1\x56\x4F\xB7\x60\xB2\x42\xA9\xD6\xA4\x62\x8C\xDC\x87\x87\x62\x83\x4E\x27\xA5\x41\xDA\x2A\x5E\x3B\x34\x45\x98\x9C\x76\xF6\x11\xE0\xFE\xC6\xD9\x1A\xCA\xCC" + ) + ] + ) + ] + where + key :: B.ByteString + key = + "\xEA\xEB\xEC\xED\xEE\xEF\xF0\xF1\xF2\xF3\xF4\xF5\xF6\xF7\xF8\xF9\xFA\xFB\xFC\xFD\xFE\xFF\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09" + + iv = B.replicate 8 0 + +newtype RandomVector = RandomVector Vector + deriving (Show, Eq) + +instance Arbitrary RandomVector where + arbitrary = RandomVector <$> elements vectors + +spec :: Spec +spec = do + describe "KAT" $ + sequence_ $ + zipWith + (\i (r, k, n, e) -> it (show (i :: Int)) $ salsaRunSimple e r k n) + [1 ..] + vectors + prop "generate-combine" salsaGenerateCombine + prop "chunking-generate" salsaGenerateChunks + prop "chunking-combine" salsaCombineChunks + where + salsaRunSimple expected rounds key nonce = + let salsa = Salsa.initialize rounds key nonce + in salsaLoop 0 salsa expected `shouldBe` map snd expected + + salsaLoop _ _ [] = [] + salsaLoop current salsa (r@(ofs, expectBs) : rs) + | current < ofs = + let (_, salsaNext) = Salsa.generate salsa (ofs - current) :: (ByteString, Salsa.State) + in salsaLoop ofs salsaNext (r : rs) + | current == ofs = + let (e, salsaNext) = Salsa.generate salsa (B.length expectBs) + in e : salsaLoop (current + B.length expectBs) salsaNext rs + | otherwise = error "internal error in salsaLoop" + + salsaGenerateCombine :: ChunkingLen0_127 -> RandomVector -> Int0_2901 -> Bool + salsaGenerateCombine (ChunkingLen0_127 ckLen) (RandomVector (rounds, key, iv, _)) (Int0_2901 nbBytes) = + let initSalsa = Salsa.initialize rounds key iv + in loop nbBytes ckLen initSalsa + where + loop n [] salsa = loop n ckLen salsa + loop 0 _ _ = True + loop n (x : xs) salsa = + let len = min x n + (c1, next) = Salsa.generate salsa len + (c2, _) = Salsa.combine salsa (B.replicate len 0) + in if c1 == c2 then loop (n - len) xs next else False + + salsaGenerateChunks :: ChunkingLen -> RandomVector -> Bool + salsaGenerateChunks (ChunkingLen ckLen) (RandomVector (rounds, key, iv, _)) = + let initSalsa = Salsa.initialize rounds key iv + nbBytes = 1048 + (expected, _) = Salsa.generate initSalsa nbBytes + chunks = loop nbBytes ckLen (Salsa.initialize rounds key iv) + in expected == B.concat chunks + where + loop n [] salsa = loop n ckLen salsa + loop 0 _ _ = [] + loop n (x : xs) salsa = + let len = min x n + (c, next) = Salsa.generate salsa len + in c : loop (n - len) xs next + + salsaCombineChunks :: ChunkingLen -> RandomVector -> ArbitraryBS0_2901 -> Bool + salsaCombineChunks (ChunkingLen ckLen) (RandomVector (rounds, key, iv, _)) (ArbitraryBS0_2901 wholebs) = + let initSalsa = Salsa.initialize rounds key iv + (expected, _) = Salsa.combine initSalsa wholebs + chunks = loop wholebs ckLen initSalsa + in expected `propertyEq` B.concat chunks + where + loop bs [] salsa = loop bs ckLen salsa + loop bs (x : xs) salsa + | B.null bs = [] + | otherwise = + let (bs1, bs2) = B.splitAt (min x (B.length bs)) bs + (c, next) = Salsa.combine salsa bs1 + in c : loop bs2 xs next diff --git a/tests/StreamCipher/XSalsaSpec.hs b/tests/StreamCipher/XSalsaSpec.hs new file mode 100644 --- /dev/null +++ b/tests/StreamCipher/XSalsaSpec.hs @@ -0,0 +1,173 @@ +{-# LANGUAGE OverloadedStrings #-} + +module StreamCipher.XSalsaSpec (spec) where + +import qualified Crypto.Cipher.XSalsa as XSalsa +import qualified Data.ByteString as B + +import Imports + +type Vector = (Int, B.ByteString, B.ByteString, B.ByteString, B.ByteString) + +-- Test vectors generated by naclcrypto library (https://nacl.cr.yp.to) +vectors :: [Vector] +vectors = + [ + ( 20 + , "\xA6\xA7\x25\x1C\x1E\x72\x91\x6D\x11\xC2\xCB\x21\x4D\x3C\x25\x25\x39\x12\x1D\x8E\x23\x4E\x65\x2D\x65\x1F\xA4\xC8\xCF\xF8\x80\x30" + , "\x9E\x64\x5A\x74\xE9\xE0\xA6\x0D\x82\x43\xAC\xD9\x17\x7A\xB5\x1A\x1B\xEB\x8D\x5A\x2F\x5D\x70\x0C" + , "\x09\x3C\x5E\x55\x85\x57\x96\x25\x33\x7B\xD3\xAB\x61\x9D\x61\x57\x60\xD8\xC5\xB2\x24\xA8\x5B\x1D\x0E\xFE\x0E\xB8\xA7\xEE\x16\x3A\xBB\x03\x76\x52\x9F\xCC\x09\xBA\xB5\x06\xC6\x18\xE1\x3C\xE7\x77\xD8\x2C\x3A\xE9\xD1\xA6\xF9\x72\xD4\x16\x02\x87\xCB\xFE\x60\xBF\x21\x30\xFC\x0A\x6F\xF6\x04\x9D\x0A\x5C\x8A\x82\xF4\x29\x23\x1F\x00\x80\x82\xE8\x45\xD7\xE1\x89\xD3\x7F\x9E\xD2\xB4\x64\xE6\xB9\x19\xE6\x52\x3A\x8C\x12\x10\xBD\x52\xA0\x2A\x4C\x3F\xE4\x06\xD3\x08\x5F\x50\x68\xD1\x90\x9E\xEE\xCA\x63\x69\xAB\xC9\x81\xA4\x2E\x87\xFE\x66\x55\x83\xF0\xAB\x85\xAE\x71\xF6\xF8\x4F\x52\x8E\x6B\x39\x7A\xF8\x6F\x69\x17\xD9\x75\x4B\x73\x20\xDB\xDC\x2F\xEA\x81\x49\x6F\x27\x32\xF5\x32\xAC\x78\xC4\xE9\xC6\xCF\xB1\x8F\x8E\x9B\xDF\x74\x62\x2E\xB1\x26\x14\x14\x16\x77\x69\x71\xA8\x4F\x94\xD1\x56\xBE\xAF\x67\xAE\xCB\xF2\xAD\x41\x2E\x76\xE6\x6E\x8F\xAD\x76\x33\xF5\xB6\xD7\xF3\xD6\x4B\x5C\x6C\x69\xCE\x29\x00\x3C\x60\x24\x46\x5A\xE3\xB8\x9B\xE7\x8E\x91\x5D\x88\xB4\xB5\x62\x1D" + , "\xB2\xAF\x68\x8E\x7D\x8F\xC4\xB5\x08\xC0\x5C\xC3\x9D\xD5\x83\xD6\x71\x43\x22\xC6\x4D\x7F\x3E\x63\x14\x7A\xED\xE2\xD9\x53\x49\x34\xB0\x4F\xF6\xF3\x37\xB0\x31\x81\x5C\xD0\x94\xBD\xBC\x6D\x7A\x92\x07\x7D\xCE\x70\x94\x12\x28\x68\x22\xEF\x07\x37\xEE\x47\xF6\xB7\xFF\xA2\x2F\x9D\x53\xF1\x1D\xD2\xB0\xA3\xBB\x9F\xC0\x1D\x9A\x88\xF9\xD5\x3C\x26\xE9\x36\x5C\x2C\x3C\x06\x3B\xC4\x84\x0B\xFC\x81\x2E\x4B\x80\x46\x3E\x69\xD1\x79\x53\x0B\x25\xC1\x58\xF5\x43\x19\x1C\xFF\x99\x31\x06\x51\x1A\xA0\x36\x04\x3B\xBC\x75\x86\x6A\xB7\xE3\x4A\xFC\x57\xE2\xCC\xE4\x93\x4A\x5F\xAA\xE6\xEA\xBE\x4F\x22\x17\x70\x18\x3D\xD0\x60\x46\x78\x27\xC2\x7A\x35\x41\x59\xA0\x81\x27\x5A\x29\x1F\x69\xD9\x46\xD6\xFE\x28\xED\x0B\x9C\xE0\x82\x06\xCF\x48\x49\x25\xA5\x1B\x94\x98\xDB\xDE\x17\x8D\xDD\x3A\xE9\x1A\x85\x81\xB9\x16\x82\xD8\x60\xF8\x40\x78\x2F\x6E\xEA\x49\xDB\xB9\xBD\x72\x15\x01\xD2\xC6\x71\x22\xDE\xA3\xB7\x28\x38\x48\xC5\xF1\x3E\x0C\x0D\xE8\x76\xBD\x22\x7A\x85\x6E\x4D\xE5\x93\xA3" + ) + , + ( 20 + , "\x9E\x1D\xA2\x39\xD1\x55\xF5\x2A\xD3\x7F\x75\xC7\x36\x8A\x53\x66\x68\xB0\x51\x95\x29\x23\xAD\x44\xF5\x7E\x75\xAB\x58\x8E\x47\x5A" + , "\xAF\x06\xF1\x78\x59\xDF\xFA\x79\x98\x91\xC4\x28\x8F\x66\x35\xB5\xC5\xA4\x5E\xEE\x90\x17\xFD\x72" + , "\xFE\xAC\x9D\x54\xFC\x8C\x11\x5A\xE2\x47\xD9\xA7\xE9\x19\xDD\x76\xCF\xCB\xC7\x2D\x32\xCA\xE4\x94\x48\x60\x81\x7C\xBD\xFB\x8C\x04\xE6\xB1\xDF\x76\xA1\x65\x17\xCD\x33\xCC\xF1\xAC\xDA\x92\x06\x38\x9E\x9E\x31\x8F\x59\x66\xC0\x93\xCF\xB3\xEC\x2D\x9E\xE2\xDE\x85\x64\x37\xED\x58\x1F\x55\x2F\x26\xAC\x29\x07\x60\x9D\xF8\xC6\x13\xB9\xE3\x3D\x44\xBF\xC2\x1F\xF7\x91\x53\xE9\xEF\x81\xA9\xD6\x6C\xC3\x17\x85\x7F\x75\x2C\xC1\x75\xFD\x88\x91\xFE\xFE\xBB\x7D\x04\x1E\x65\x17\xC3\x16\x2D\x19\x7E\x21\x12\x83\x7D\x3B\xC4\x10\x43\x12\xAD\x35\xB7\x5E\xA6\x86\xE7\xC7\x0D\x4E\xC0\x47\x46\xB5\x2F\xF0\x9C\x42\x14\x51\x45\x9F\xB5\x9F" + , "\x2C\x26\x1A\x2F\x4E\x61\xA6\x2E\x1B\x27\x68\x99\x16\xBF\x03\x45\x3F\xCB\xC9\x7B\xB2\xAF\x6F\x32\x93\x91\xEF\x06\x3B\x5A\x21\x9B\xF9\x84\xD0\x7D\x70\xF6\x02\xD8\x5F\x6D\xB6\x14\x74\xE9\xD9\xF5\xA2\xDE\xEC\xB4\xFC\xD9\x01\x84\xD1\x6F\x3B\x5B\x5E\x16\x8E\xE0\x3E\xA8\xC9\x3F\x39\x33\xA2\x2B\xC3\xD1\xA5\xAE\x8C\x2D\x8B\x02\x75\x7C\x87\xC0\x73\x40\x90\x52\xA2\xA8\xA4\x1E\x7F\x48\x7E\x04\x1F\x9A\x49\xA0\x99\x7B\x54\x0E\x18\x62\x1C\xAD\x3A\x24\xF0\xA5\x6D\x9B\x19\x22\x79\x29\x05\x7A\xB3\xBA\x95\x0F\x62\x74\xB1\x21\xF1\x93\xE3\x2E\x06\xE5\x38\x87\x81\xA1\xCB\x57\x31\x7C\x0B\xA6\x30\x5E\x91\x09\x61\xD0\x10\x02\xF0" + ) + , + ( 20 + , "\xD5\xC7\xF6\x79\x7B\x7E\x7E\x9C\x1D\x7F\xD2\x61\x0B\x2A\xBF\x2B\xC5\xA7\x88\x5F\xB3\xFF\x78\x09\x2F\xB3\xAB\xE8\x98\x6D\x35\xE2" + , "\x74\x4E\x17\x31\x2B\x27\x96\x9D\x82\x64\x44\x64\x0E\x9C\x4A\x37\x8A\xE3\x34\xF1\x85\x36\x9C\x95" + , "\x77\x58\x29\x8C\x62\x8E\xB3\xA4\xB6\x96\x3C\x54\x45\xEF\x66\x97\x12\x22\xBE\x5D\x1A\x4A\xD8\x39\x71\x5D\x11\x88\x07\x17\x39\xB7\x7C\xC6\xE0\x5D\x54\x10\xF9\x63\xA6\x41\x67\x62\x97\x57" + , "\x27\xB8\xCF\xE8\x14\x16\xA7\x63\x01\xFD\x1E\xEC\x6A\x4D\x99\x67\x50\x69\xB2\xDA\x27\x76\xC3\x60\xDB\x1B\xDF\xEA\x7C\x0A\xA6\x13\x91\x3E\x10\xF7\xA6\x0F\xEC\x04\xD1\x1E\x65\xF2\xD6\x4E" + ) + , + ( 20 + , "\x73\x7D\x78\x11\xCE\x96\x47\x2E\xFE\xD1\x22\x58\xB7\x81\x22\xF1\x1D\xEA\xEC\x87\x59\xCC\xBD\x71\xEA\xC6\xBB\xEF\xA6\x27\x78\x5C" + , "\x6F\xB2\xEE\x3D\xDA\x6D\xBD\x12\xF1\x27\x4F\x12\x67\x01\xEC\x75\xC3\x5C\x86\x60\x7A\xDB\x3E\xDD" + , "\x50\x13\x25\xFB\x26\x45\x26\x48\x64\xDF\x11\xFA\xA1\x7B\xBD\x58\x31\x2B\x77\xCA\xD3\xD9\x4A\xC8\xFB\x85\x42\xF0\xEB\x65\x3A\xD7\x3D\x7F\xCE\x93\x2B\xB8\x74\xCB\x89\xAC\x39\xFC\x47\xF8\x26\x7C\xF0\xF0\xC2\x09\xF2\x04\xB2\xD8\x57\x8A\x3B\xDF\x46\x1C\xB6\xA2\x71\xA4\x68\xBE\xBA\xCC\xD9\x68\x50\x14\xCC\xBC\x9A\x73\x61\x8C\x6A\x5E\x77\x8A\x21\xCC\x84\x16\xC6\x0A\xD2\x4D\xDC\x41\x7A\x13\x0D\x53\xED\xA6\xDF\xBF\xE4\x7D\x09\x17\x0A\x7B\xE1\xA7\x08\xB7\xB5\xF3\xAD\x46\x43\x10\xBE\x36\xD9\xA2\xA9\x5D\xC3\x9E\x83\xD3\x86\x67\xE8\x42\xEB\x64\x11\xE8\xA2\x37\x12\x29\x7B\x16\x5F\x69\x0C\x2D\x7C\xA1\xB1\x34\x6E\x3C\x1F\xCC\xF5\xCA\xFD\x4F\x8B\xE0" + , "\x67\x24\xC3\x72\xD2\xE9\x07\x4D\xA5\xE2\x7A\x6C\x54\xB2\xD7\x03\xDC\x1D\x4C\x9B\x1F\x8D\x90\xF0\x0C\x12\x2E\x69\x2A\xCE\x77\x00\xEA\xDC\xA9\x42\x54\x45\x07\xF1\x37\x5B\x65\x81\xD5\xA8\xFB\x39\x98\x1C\x1C\x0E\x6E\x1F\xF2\x14\x0B\x08\x2E\x9E\xC0\x16\xFC\xE1\x41\xD5\x19\x96\x47\xD4\x3B\x0B\x68\xBF\xD0\xFE\xA5\xE0\x0F\x46\x89\x62\xC7\x38\x4D\xD6\x12\x9A\xEA\x6A\x3F\xDF\xE7\x5A\xBB\x21\x0E\xD5\x60\x7C\xEF\x8F\xA0\xE1\x52\x83\x3D\x5A\xC3\x7D\x52\xE5\x57\xB9\x10\x98\xA3\x22\xE7\x6A\x45\xBB\xBC\xF4\x89\x9E\x79\x06\x18\xAA\x3F\x4C\x2E\x5E\x0F\xC3\xDE\x93\x26\x9A\x57\x7D\x77\xA5\x50\x2E\x8E\xA0\x2F\x71\x7B\x1D\xD2\xDF\x1E\xC6\x9D\x8B\x61\xCA" + ) + , + ( 20 + , "\x76\x01\x58\xDA\x09\xF8\x9B\xBA\xB2\xC9\x9E\x69\x97\xF9\x52\x3A\x95\xFC\xEF\x10\x23\x9B\xCC\xA2\x57\x3B\x71\x05\xF6\x89\x8D\x34" + , "\x43\x63\x6B\x2C\xC3\x46\xFC\x8B\x7C\x85\xA1\x9B\xF5\x07\xBD\xC3\xDA\xFE\x95\x3B\x88\xC6\x9D\xBA" + , "\xD3\x0A\x6D\x42\xDF\xF4\x9F\x0E\xD0\x39\xA3\x06\xBA\xE9\xDE\xC8\xD9\xE8\x83\x66\xCC\x19\xE8\xC3\x64\x2F\xD5\x8F\xA0\x79\x4E\xBF\x80\x29\xD9\x49\x73\x03\x39\xB0\x82\x3A\x51\xF0\xF4\x9F\x0D\x2C\x71\xF1\x05\x1C\x1E\x0E\x2C\x86\x94\x1F\x17\x27\x89\xCD\xB1\xB0\x10\x74\x13\xE7\x0F\x98\x2F\xF9\x76\x18\x77\xBB\x52\x6E\xF1\xC3\xEB\x11\x06\xA9\x48\xD6\x0E\xF2\x1B\xD3\x5D\x32\xCF\xD6\x4F\x89\xB7\x9E\xD6\x3E\xCC\x5C\xCA\x56\x24\x6A\xF7\x36\x76\x6F\x28\x5D\x8E\x6B\x0D\xA9\xCB\x1C\xD2\x10\x20\x22\x3F\xFA\xCC\x5A\x32" + , "\xC8\x15\xB6\xB7\x9B\x64\xF9\x36\x9A\xEC\x8D\xCE\x8C\x75\x3D\xF8\xA5\x0F\x2B\xC9\x7C\x70\xCE\x2F\x01\x4D\xB3\x3A\x65\xAC\x58\x16\xBA\xC9\xE3\x0A\xC0\x8B\xDD\xED\x30\x8C\x65\xCB\x87\xE2\x8E\x2E\x71\xB6\x77\xDC\x25\xC5\xA6\x49\x9C\x15\x53\x55\x5D\xAF\x1F\x55\x27\x0A\x56\x95\x9D\xFF\xA0\xC6\x6F\x24\xE0\xAF\x00\x95\x1E\xC4\xBB\x59\xCC\xC3\xA6\xC5\xF5\x2E\x09\x81\x64\x7E\x53\xE4\x39\x31\x3A\x52\xC4\x0F\xA7\x00\x4C\x85\x5B\x6E\x6E\xB2\x5B\x21\x2A\x13\x8E\x84\x3A\x9B\xA4\x6E\xDB\x2A\x03\x9E\xE8\x2A\x26\x3A\xBE" + ) + , + ( 20 + , "\x27\xBA\x7E\x81\xE7\xED\xD4\xE7\x1B\xE5\x3C\x07\xCE\x8E\x63\x31\x38\xF2\x87\xE1\x55\xC7\xFA\x9E\x84\xC4\xAD\x80\x4B\x7F\xA1\xB9" + , "\xEA\x05\xF4\xEB\xCD\x2F\xB6\xB0\x00\xDA\x06\x12\x86\x1B\xA5\x4F\xF5\xC1\x76\xFB\x60\x13\x91\xAA" + , "\xE0\x9F\xF5\xD2\xCB\x05\x0D\x69\xB2\xD4\x24\x94\xBD\xE5\x82\x52\x38\xC7\x56\xD6\x99\x1D\x99\xD7\xA2\x0D\x1E\xF0\xB8\x3C\x37\x1C\x89\x87\x26\x90\xB2\xFC\x11\xD5\x36\x9F\x4F\xC4\x97\x1B\x6D\x3D\x6C\x07\x8A\xEF\x9B\x0F\x05\xC0\xE6\x1A\xB8\x9C\x02\x51\x68\x05\x4D\xEF\xEB\x03\xFE\xF6\x33\x85\x87\x00\xC5\x8B\x12\x62\xCE\x01\x13\x00\x01\x26\x73\xE8\x93\xE4\x49\x01\xDC\x18\xEE\xE3\x10\x56\x99\xC4\x4C\x80\x58\x97\xBD\xAF\x77\x6A\xF1\x83\x31\x62\xA2\x1A" + , "\xA2\x3E\x7E\xF9\x3C\x5D\x06\x67\xC9\x6D\x9E\x40\x4D\xCB\xE6\xBE\x62\x02\x6F\xA9\x8F\x7A\x3F\xF9\xBA\x5D\x45\x86\x43\xA1\x6A\x1C\xEF\x72\x72\xDC\x60\x97\xA9\xB5\x2F\x35\x98\x35\x57\xC7\x7A\x11\xB3\x14\xB4\xF7\xD5\xDC\x2C\xCA\x15\xEE\x47\x61\x6F\x86\x18\x73\xCB\xFE\xD1\xD3\x23\x72\x17\x1A\x61\xE3\x8E\x44\x7F\x3C\xF3\x62\xB3\xAB\xBB\x2E\xD4\x17\x0D\x89\xDC\xB2\x81\x87\xB7\xBF\xD2\x06\xA3\xE0\x26\xF0\x84\xA7\xE0\xED\x63\xD3\x19\xDE\x6B\xC9\xAF\xC0" + ) + , + ( 20 + , "\x67\x99\xD7\x6E\x5F\xFB\x5B\x49\x20\xBC\x27\x68\xBA\xFD\x3F\x8C\x16\x55\x4E\x65\xEF\xCF\x9A\x16\xF4\x68\x3A\x7A\x06\x92\x7C\x11" + , "\x61\xAB\x95\x19\x21\xE5\x4F\xF0\x6D\x9B\x77\xF3\x13\xA4\xE4\x9D\xF7\xA0\x57\xD5\xFD\x62\x79\x89" + , "\x47\x27\x66" + , "\x8F\xD7\xDF" + ) + , + ( 20 + , "\xF6\x82\x38\xC0\x83\x65\xBB\x29\x3D\x26\x98\x0A\x60\x64\x88\xD0\x9C\x2F\x10\x9E\xDA\xFA\x0B\xBA\xE9\x93\x7B\x5C\xC2\x19\xA4\x9C" + , "\x51\x90\xB5\x1E\x9B\x70\x86\x24\x82\x0B\x5A\xBD\xF4\xE4\x0F\xAD\x1F\xB9\x50\xAD\x1A\xDC\x2D\x26" + , "\x47\xEC\x6B\x1F\x73\xC4\xB7\xFF\x52\x74\xA0\xBF\xD7\xF4\x5F\x86\x48\x12\xC8\x5A\x12\xFB\xCB\x3C\x2C\xF8\xA3\xE9\x0C\xF6\x6C\xCF\x2E\xAC\xB5\x21\xE7\x48\x36\x3C\x77\xF5\x2E\xB4\x26\xAE\x57\xA0\xC6\xC7\x8F\x75\xAF\x71\x28\x45\x69\xE7\x9D\x1A\x92\xF9\x49\xA9\xD6\x9C\x4E\xFC\x0B\x69\x90\x2F\x1E\x36\xD7\x56\x27\x65\x54\x3E\x2D\x39\x42\xD9\xF6\xFF\x59\x48\xD8\xA3\x12\xCF\xF7\x2C\x1A\xFD\x9E\xA3\x08\x8A\xFF\x76\x40\xBF\xD2\x65\xF7\xA9\x94\x6E\x60\x6A\xBC\x77\xBC\xED\xAE\x6B\xDD\xC7\x5A\x0D\xBA\x0B\xD9\x17\xD7\x3E\x3B\xD1\x26\x8F\x72\x7E\x00\x96\x34\x5D\xA1\xED\x25\xCF\x55\x3E\xA7\xA9\x8F\xEA\x6B\x6F\x28\x57\x32\xDE\x37\x43\x15\x61\xEE\x1B\x30\x64\x88\x7F\xBC\xBD\x71\x93\x5E\x02" + , "\x36\x16\x0E\x88\xD3\x50\x05\x29\xBA\x4E\xDB\xA1\x7B\xC2\x4D\x8C\xFA\xCA\x9A\x06\x80\xB3\xB1\xFC\x97\xCF\x03\xF3\x67\x5B\x7A\xC3\x01\xC8\x83\xA6\x8C\x07\x1B\xC5\x4A\xCD\xD3\xB6\x3A\xF4\xA2\xD7\x2F\x98\x5E\x51\xF9\xD6\x0A\x4C\x7F\xD4\x81\xAF\x10\xB2\xFC\x75\xE2\x52\xFD\xEE\x7E\xA6\xB6\x45\x31\x90\x61\x7D\xCC\x6E\x2F\xE1\xCD\x56\x58\x5F\xC2\xF0\xB0\xE9\x7C\x5C\x3F\x8A\xD7\xEB\x4F\x31\xBC\x48\x90\xC0\x38\x82\xAA\xC2\x4C\xC5\x3A\xCC\x19\x82\x29\x65\x26\x69\x0A\x22\x02\x71\xC2\xF6\xE3\x26\x75\x0D\x3F\xBD\xA5\xD5\xB6\x35\x12\xC8\x31\xF6\x78\x30\xF5\x9A\xC4\x9A\xAE\x33\x0B\x3E\x0E\x02\xC9\xEA\x00\x91\xD1\x98\x41\xF1\xB0\xE1\x3D\x69\xC9\xFB\xFE\x8A\x12\xD6\xF3\x0B\xB7\x34\xD9\xD2" + ) + , + ( 20 + , "\x45\xB2\xBD\x0D\xE4\xED\x92\x93\xEC\x3E\x26\xC4\x84\x0F\xAA\xF6\x4B\x7D\x61\x9D\x51\xE9\xD7\xA2\xC7\xE3\x6C\x83\xD5\x84\xC3\xDF" + , "\x54\x6C\x8C\x5D\x6B\xE8\xF9\x09\x52\xCA\xB3\xF3\x6D\x7C\x19\x57\xBA\xAA\x7A\x59\xAB\xE3\xD7\xE5" + , "\x50\x07\xC8\xCD\x5B\x3C\x40\xE1\x7D\x7F\xE4\x23\xA8\x7A\xE0\xCE\xD8\x6B\xEC\x1C\x39\xDC\x07\xA2\x57\x72\xF3\xE9\x6D\xAB\xD5\x6C\xD3\xFD\x73\x19\xF6\xC9\x65\x49\x25\xF2\xD8\x70\x87\xA7\x00\xE1\xB1\x30\xDA\x79\x68\x95\xD1\xC9\xB9\xAC\xD6\x2B\x26\x61\x44\x06\x7D\x37\x3E\xD5\x1E\x78\x74\x98\xB0\x3C\x52\xFA\xAD\x16\xBB\x38\x26\xFA\x51\x1B\x0E\xD2\xA1\x9A\x86\x63\xF5\xBA\x2D\x6E\xA7\xC3\x8E\x72\x12\xE9\x69\x7D\x91\x48\x6C\x49\xD8\xA0\x00\xB9\xA1\x93\x5D\x6A\x7F\xF7\xEF\x23\xE7\x20\xA4\x58\x55\x48\x14\x40\x46\x3B\x4A\xC8\xC4\xF6\xE7\x06\x2A\xDC\x1F\x1E\x1E\x25\xD3\xD6\x5A\x31\x81\x2F\x58\xA7\x11\x60" + , "\x8E\xAC\xFB\xA5\x68\x89\x8B\x10\xC0\x95\x7A\x7D\x44\x10\x06\x85\xE8\x76\x3A\x71\xA6\x9A\x8D\x16\xBC\x7B\x3F\x88\x08\x5B\xB9\xA2\xF0\x96\x42\xE4\xD0\x9A\x9F\x0A\xD0\x9D\x0A\xAD\x66\xB2\x26\x10\xC8\xBD\x02\xFF\x66\x79\xBB\x92\xC2\xC0\x26\xA2\x16\xBF\x42\x5C\x6B\xE3\x5F\xB8\xDA\xE7\xFF\x0C\x72\xB0\xEF\xD6\xA1\x80\x37\xC7\x0E\xED\x0C\xA9\x00\x62\xA4\x9A\x3C\x97\xFD\xC9\x0A\x8F\x9C\x2E\xA5\x36\xBF\xDC\x41\x91\x8A\x75\x82\xC9\x92\x7F\xAE\x47\xEF\xAA\x3D\xC8\x79\x67\xB7\x88\x7D\xEE\x1B\xF0\x71\x73\x4C\x76\x65\x90\x1D\x91\x05\xDA\xE2\xFD\xF6\x6B\x49\x18\xE5\x1D\x8F\x4A\x48\xC6\x0D\x19\xFB\xFB\xBC\xBA" + ) + , + ( 20 + , "\xFE\x55\x9C\x9A\x28\x2B\xEB\x40\x81\x4D\x01\x6D\x6B\xFC\xB2\xC0\xC0\xD8\xBF\x07\x7B\x11\x10\xB8\x70\x3A\x3C\xE3\x9D\x70\xE0\xE1" + , "\xB0\x76\x20\x0C\xC7\x01\x12\x59\x80\x5E\x18\xB3\x04\x09\x27\x54\x00\x27\x23\xEB\xEC\x5D\x62\x00" + , "\x6D\xB6\x5B\x9E\xC8\xB1\x14\xA9\x44\x13\x7C\x82\x1F\xD6\x06\xBE\x75\x47\x8D\x92\x83\x66\xD5\x28\x40\x96\xCD\xEF\x78\x2F\xCF\xF7\xE8\xF5\x9C\xB8\xFF\xCD\xA9\x79\x75\x79\x02\xC5\xFF\xA6\xBC\x47\x7C\xEA\xA4\xCB\x5D\x5E\xA7\x6F\x94\xD9\x1E\x83\x3F\x82\x3A\x6B\xC7\x8F\x10\x55\xDF\xA6\xA9\x7B\xEA\x89\x65\xC1\xCD\xE6\x7A\x66\x8E\x00\x12\x57\x33\x4A\x58\x57\x27\xD9\xE0\xF7\xC1\xA0\x6E\x88\xD3\xD2\x5A\x4E\x6D\x90\x96\xC9\x68\xBF\x13\x8E\x11\x6A\x3E\xBE\xFF\xD4\xBB\x48\x08\xAD\xB1\xFD\x69\x81\x64\xBA\x0A\x35\xC7\x09\xA4\x7F\x16\xF1\xF4\x43\x5A\x23\x45\xA9\x19\x4A\x00\xB9\x5A\xBD\x51\x85\x1D\x50\x58\x09\xA6\x07\x7D\xA9\xBA\xCA\x58\x31\xAF\xFF\x31\x57\x8C\x48\x7E\xE6\x8F\x27\x67\x97\x4A\x98\xA7\xE8\x03\xAA\xC7\x88\xDA\x98\x31\x9C\x4E\xA8\xEA\xA3\xD3\x94\x85\x56\x51\xF4\x84\xCE\xF5\x43\xF5\x37\xE3\x51\x58\xEE\x29" + , "\x4D\xCE\x9C\x8F\x97\xA0\x28\x05\x1B\x07\x27\xF3\x4E\x1B\x9E\xF2\x1F\x06\xF0\x76\x0F\x36\xE7\x17\x13\x20\x40\x27\x90\x20\x90\xBA\x2B\xB6\xB1\x34\x36\xEE\x77\x8D\x9F\x50\x53\x0E\xFB\xD7\xA3\x2B\x0D\x41\x44\x3F\x58\xCC\xAE\xE7\x81\xC7\xB7\x16\xD3\xA9\x6F\xDE\xC0\xE3\x76\x4E\xD7\x95\x9F\x34\xC3\x94\x12\x78\x59\x1E\xA0\x33\xB5\xCB\xAD\xC0\xF1\x91\x60\x32\xE9\xBE\xBB\xD1\xA8\x39\x5B\x83\xFB\x63\xB1\x45\x4B\xD7\x75\xBD\x20\xB3\xA2\xA9\x6F\x95\x12\x46\xAC\x14\xDA\xF6\x81\x66\xBA\x62\xF6\xCB\xFF\x8B\xD1\x21\xAC\x94\x98\xFF\x88\x52\xFD\x2B\xE9\x75\xDF\x52\xB5\xDA\xEF\x38\x29\xD1\x8E\xDA\x42\xE7\x15\x02\x2D\xCB\xF9\x30\xD0\xA7\x89\xEE\x6A\x14\x6C\x2C\x70\x88\xC3\x57\x73\xC6\x3C\x06\xB4\xAF\x45\x59\x85\x6A\xC1\x99\xCE\xD8\x68\x63\xE4\x29\x47\x07\x82\x53\x37\xC5\x85\x79\x70\xEB\x7F\xDD\xEB\x26\x37\x81\x30\x90\x11" + ) + , + ( 20 + , "\x0A\xE1\x00\x12\xD7\xE5\x66\x14\xB0\x3D\xCC\x89\xB1\x4B\xAE\x92\x42\xFF\xE6\x30\xF3\xD7\xE3\x5C\xE8\xBB\xB9\x7B\xBC\x2C\x92\xC3" + , "\xF9\x6B\x02\x5D\x6C\xF4\x6A\x8A\x12\xAC\x2A\xF1\xE2\xAE\xF1\xFB\x83\x59\x0A\xDA\xDA\xA5\xC5\xEA" + , "\xEA\x0F\x35\x4E\x96\xF1\x2B\xC7\x2B\xBA\xA3\xD1\x2B\x4A\x8E\xD8\x79\xB0\x42\xF0\x68\x98\x78\xF4\x6B\x65\x1C\xC4\x11\x6D\x6F\x78\x40\x9B\x11\x43\x0B\x3A\xAA\x30\xB2\x07\x68\x91\xE8\xE1\xFA\x52\x8F\x2F\xD1\x69\xED\x93\xDC\x9F\x84\xE2\x44\x09\xEE\xC2\x10\x1D\xAF\x4D\x05\x7B\xE2\x49\x2D\x11\xDE\x64\x0C\xBD\x7B\x35\x5A\xD2\x9F\xB7\x04\x00\xFF\xFD\x7C\xD6\xD4\x25\xAB\xEE\xB7\x32\xA0\xEA\xA4\x33\x0A\xF4\xC6\x56\x25\x2C\x41\x73\xDE\xAB\x65\x3E\xB8\x5C\x58\x46\x2D\x7A\xB0\xF3\x5F\xD1\x2B\x61\x3D\x29\xD4\x73\xD3\x30\x31\x0D\xC3\x23\xD3\xC6\x63\x48\xBB\xDB\xB6\x8A\x32\x63\x24\x65\x7C\xAE\x7B\x77\xA9\xE3\x43\x58\xF2\xCE\xC5\x0C\x85\x60\x9E\x73\x05\x68\x56\x79\x6E\x3B\xE8\xD6\x2B\x6E\x2F\xE9\xF9\x53" + , "\xE8\xAB\xD4\x89\x24\xB5\x4E\x5B\x80\x86\x6B\xE7\xD4\xEB\xE5\xCF\x42\x74\xCA\xFF\xF0\x8B\x39\xCB\x2D\x40\xA8\xF0\xB4\x72\x39\x8A\xED\xC7\x76\xE0\x79\x38\x12\xFB\xF1\xF6\x00\x78\x63\x5D\x2E\xD8\x6B\x15\xEF\xCD\xBA\x60\x41\x1E\xE2\x3B\x07\x23\x35\x92\xA4\x4E\xC3\x1B\x10\x13\xCE\x89\x64\x23\x66\x75\xF8\xF1\x83\xAE\xF8\x85\xE8\x64\xF2\xA7\x2E\xDF\x42\x15\xB5\x33\x8F\xA2\xB5\x46\x53\xDF\xA1\xA8\xC5\x5C\xE5\xD9\x5C\xC6\x05\xB9\xB3\x11\x52\x7F\x2E\x34\x63\xFF\xBE\xC7\x8A\x9D\x1D\x65\xDA\xBA\xD2\xF3\x38\x76\x9C\x9F\x43\xF1\x33\xA7\x91\xA1\x1C\x7E\xCA\x9A\xF0\xB7\x71\xA4\xAC\x32\x96\x3D\xC8\xF6\x31\xA2\xC1\x12\x17\xAC\x6E\x1B\x94\x30\xC1\xAA\xE1\xCE\xEB\xE2\x27\x03\xF4\x29\x99\x8A\x8F\xB8\xC6\x41" + ) + , + ( 20 + , "\x08\x2C\x53\x9B\xC5\xB2\x0F\x97\xD7\x67\xCD\x3F\x22\x9E\xDA\x80\xB2\xAD\xC4\xFE\x49\xC8\x63\x29\xB5\xCD\x62\x50\xA9\x87\x74\x50" + , "\x84\x55\x43\x50\x2E\x8B\x64\x91\x2D\x8F\x2C\x8D\x9F\xFF\xB3\xC6\x93\x65\x68\x65\x87\xC0\x8D\x0C" + , "\xA9\x6B\xB7\xE9\x10\x28\x1A\x6D\xFA\xD7\xC8\xA9\xC3\x70\x67\x4F\x0C\xEE\xC1\xAD\x8D\x4F\x0D\xE3\x2F\x9A\xE4\xA2\x3E\xD3\x29\xE3\xD6\xBC\x70\x8F\x87\x66\x40\xA2\x29\x15\x3A\xC0\xE7\x28\x1A\x81\x88\xDD\x77\x69\x51\x38\xF0\x1C\xDA\x5F\x41\xD5\x21\x5F\xD5\xC6\xBD\xD4\x6D\x98\x2C\xB7\x3B\x1E\xFE\x29\x97\x97\x0A\x9F\xDB\xDB\x1E\x76\x8D\x7E\x5D\xB7\x12\x06\x8D\x8B\xA1\xAF\x60\x67\xB5\x75\x34\x95\xE2\x3E\x6E\x19\x63\xAF\x01\x2F\x9C\x7C\xE4\x50\xBF\x2D\xE6\x19\xD3\xD5\x95\x42\xFB\x55\xF3" + , "\x83\x5D\xA7\x4F\xC6\xDE\x08\xCB\xDA\x27\x7A\x79\x66\xA0\x7C\x8D\xCD\x62\x7E\x7B\x17\xAD\xDE\x6D\x93\x0B\x65\x81\xE3\x12\x4B\x8B\xAA\xD0\x96\xF6\x93\x99\x1F\xED\xB1\x57\x29\x30\x60\x1F\xC7\x70\x95\x41\x83\x9B\x8E\x3F\xFD\x5F\x03\x3D\x20\x60\xD9\x99\xC6\xC6\xE3\x04\x82\x76\x61\x3E\x64\x80\x00\xAC\xB5\x21\x2C\xC6\x32\xA9\x16\xAF\xCE\x29\x0E\x20\xEB\xDF\x61\x2D\x08\xA6\xAA\x4C\x79\xA7\x4B\x07\x0D\x3F\x87\x2A\x86\x1F\x8D\xC6\xBB\x07\x61\x4D\xB5\x15\xD3\x63\x34\x9D\x3A\x8E\x33\x36\xA3" + ) + , + ( 20 + , "\x3D\x02\xBF\xF3\x37\x5D\x40\x30\x27\x35\x6B\x94\xF5\x14\x20\x37\x37\xEE\x9A\x85\xD2\x05\x2D\xB3\xE4\xE5\xA2\x17\xC2\x59\xD1\x8A" + , "\x74\x21\x6C\x95\x03\x18\x95\xF4\x8C\x1D\xBA\x65\x15\x55\xEB\xFA\x3C\xA3\x26\xA7\x55\x23\x70\x25" + , "\x0D\x4B\x0F\x54\xFD\x09\xAE\x39\xBA\xA5\xFA\x4B\xAC\xCF\x2E\x66\x82\xE6\x1B\x25\x7E\x01\xF4\x2B\x8F" + , "\x16\xC4\x00\x6C\x28\x36\x51\x90\x41\x1E\xB1\x59\x38\x14\xCF\x15\xE7\x4C\x22\x23\x8F\x21\x0A\xFC\x3D" + ) + , + ( 20 + , "\xAD\x1A\x5C\x47\x68\x88\x74\xE6\x66\x3A\x0F\x3F\xA1\x6F\xA7\xEF\xB7\xEC\xAD\xC1\x75\xC4\x68\xE5\x43\x29\x14\xBD\xB4\x80\xFF\xC6" + , "\xE4\x89\xEE\xD4\x40\xF1\xAA\xE1\xFA\xC8\xFB\x7A\x98\x25\x63\x54\x54\xF8\xF8\xF1\xF5\x2E\x2F\xCC" + , "\xAA\x6C\x1E\x53\x58\x0F\x03\xA9\xAB\xB7\x3B\xFD\xAD\xED\xFE\xCA\xDA\x4C\x6B\x0E\xBE\x02\x0E\xF1\x0D\xB7\x45\xE5\x4B\xA8\x61\xCA\xF6\x5F\x0E\x40\xDF\xC5\x20\x20\x3B\xB5\x4D\x29\xE0\xA8\xF7\x8F\x16\xB3\xF1\xAA\x52\x5D\x6B\xFA\x33\xC5\x47\x26\xE5\x99\x88\xCF\xBE\xC7\x80\x56" + , "\x02\xFE\x84\xCE\x81\xE1\x78\xE7\xAA\xBD\xD3\xBA\x92\x5A\x76\x6C\x3C\x24\x75\x6E\xEF\xAE\x33\x94\x2A\xF7\x5E\x8B\x46\x45\x56\xB5\x99\x7E\x61\x6F\x3F\x2D\xFC\x7F\xCE\x91\x84\x8A\xFD\x79\x91\x2D\x9F\xB5\x52\x01\xB5\x81\x3A\x5A\x07\x4D\x2C\x0D\x42\x92\xC1\xFD\x44\x18\x07\xC5" + ) + , + ( 20 + , "\x05\x3A\x02\xBE\xDD\x63\x68\xC1\xFB\x8A\xFC\x7A\x1B\x19\x9F\x7F\x7E\xA2\x22\x0C\x9A\x4B\x64\x2A\x68\x50\x09\x1C\x9D\x20\xAB\x9C" + , "\xC7\x13\xEE\xA5\xC2\x6D\xAD\x75\xAD\x3F\x52\x45\x1E\x00\x3A\x9C\xB0\xD6\x49\xF9\x17\xC8\x9D\xDE" + , "\x8F\x0A\x8A\x16\x47\x60\x42\x65\x67\xE3\x88\x84\x02\x76\xDE\x3F\x95\xCB\x5E\x3F\xAD\xC6\xED\x3F\x3E\x4F\xE8\xBC\x16\x9D\x93\x88\x80\x4D\xCB\x94\xB6\x58\x7D\xBB\x66\xCB\x0B\xD5\xF8\x7B\x8E\x98\xB5\x2A\xF3\x7B\xA2\x90\x62\x9B\x85\x8E\x0E\x2A\xA7\x37\x80\x47\xA2\x66\x02" + , "\x51\x67\x10\xE5\x98\x43\xE6\xFB\xD4\xF2\x5D\x0D\x8C\xA0\xEC\x0D\x47\xD3\x9D\x12\x5E\x9D\xAD\x98\x7E\x05\x18\xD4\x91\x07\x01\x4C\xB0\xAE\x40\x5E\x30\xC2\xEB\x37\x94\x75\x0B\xCA\x14\x2C\xE9\x5E\x29\x0C\xF9\x5A\xBE\x15\xE8\x22\x82\x3E\x2E\x7D\x3A\xB2\x1B\xC8\xFB\xD4\x45" + ) + , + ( 20 + , "\x5B\x14\xAB\x0F\xBE\xD4\xC5\x89\x52\x54\x8A\x6C\xB1\xE0\x00\x0C\xF4\x48\x14\x21\xF4\x12\x88\xEA\x0A\xA8\x4A\xDD\x9F\x7D\xEB\x96" + , "\x54\xBF\x52\xB9\x11\x23\x1B\x95\x2B\xA1\xA6\xAF\x8E\x45\xB1\xC5\xA2\x9D\x97\xE2\xAB\xAD\x7C\x83" + , "\x37\xFB\x44\xA6\x75\x97\x8B\x56\x0F\xF9\xA4\xA8\x70\x11\xD6\xF3\xAD\x2D\x37\xA2\xC3\x81\x5B\x45\xA3\xC0\xE6\xD1\xB1\xD8\xB1\x78\x4C\xD4\x68\x92\x7C\x2E\xE3\x9E\x1D\xCC\xD4\x76\x5E\x1C\x3D\x67\x6A\x33\x5B\xE1\xCC\xD6\x90\x0A\x45\xF5\xD4\x1A\x31\x76\x48\x31\x5D\x8A\x8C\x24\xAD\xC6\x4E\xB2\x85\xF6\xAE\xBA\x05\xB9\x02\x95\x86\x35\x3D\x30\x3F\x17\xA8\x07\x65\x8B\x9F\xF7\x90\x47\x4E\x17\x37\xBD\x5F\xDC\x60\x4A\xEF\xF8\xDF\xCA\xF1\x42\x7D\xCC\x3A\xAC\xBB\x02\x56\xBA\xDC\xD1\x83\xED\x75\xA2\xDC\x52\x45\x2F\x87\xD3\xC1\xED\x2A\xA5\x83\x47\x2B\x0A\xB9\x1C\xDA\x20\x61\x4E\x9B\x6F\xDB\xDA\x3B\x49\xB0\x98\xC9\x58\x23\xCC\x72\xD8\xE5\xB7\x17\xF2\x31\x4B\x03\x24\xE9\xCE" + , "\xAE\x6D\xEB\x5D\x6C\xE4\x3D\x4B\x09\xD0\xE6\xB1\xC0\xE9\xF4\x61\x57\xBC\xD8\xAB\x50\xEA\xA3\x19\x7F\xF9\xFA\x2B\xF7\xAF\x64\x9E\xB5\x2C\x68\x54\x4F\xD3\xAD\xFE\x6B\x1E\xB3\x16\xF1\xF2\x35\x38\xD4\x70\xC3\x0D\xBF\xEC\x7E\x57\xB6\x0C\xBC\xD0\x96\xC7\x82\xE7\x73\x6B\x66\x91\x99\xC8\x25\x3E\x70\x21\x4C\xF2\xA0\x98\xFD\xA8\xEA\xC5\xDA\x79\xA9\x49\x6A\x3A\xAE\x75\x4D\x03\xB1\x7C\x6D\x70\xD1\x02\x7F\x42\xBF\x7F\x95\xCE\x3D\x1D\x9C\x33\x88\x54\xE1\x58\xFC\xC8\x03\xE4\xD6\x26\x2F\xB6\x39\x52\x1E\x47\x11\x6E\xF7\x8A\x7A\x43\x7C\xA9\x42\x7B\xA6\x45\xCD\x64\x68\x32\xFE\xAB\x82\x2A\x20\x82\x78\xE4\x5E\x93\xE1\x18\xD7\x80\xB9\x88\xD6\x53\x97\xED\xDF\xD7\xA8\x19\x52\x6E" + ) + , + ( 20 + , "\xD7\x46\x36\xE3\x41\x3A\x88\xD8\x5F\x32\x2C\xA8\x0F\xB0\xBD\x65\x0B\xD0\xBF\x01\x34\xE2\x32\x91\x60\xB6\x96\x09\xCD\x58\xA4\xB0" + , "\xEF\xB6\x06\xAA\x1D\x9D\x9F\x0F\x46\x5E\xAA\x7F\x81\x65\xF1\xAC\x09\xF5\xCB\x46\xFE\xCF\x2A\x57" + , "\xF8\x54\x71\xB7\x5F\x6E\xC8\x1A\xBA\xC2\x79\x9E\xC0\x9E\x98\xE2\x80\xB2\xFF\xD6\x4C\xA2\x85\xE5\xA0\x10\x9C\xFB\x31\xFF\xAB\x2D\x61\x7B\x2C\x29\x52\xA2\xA8\xA7\x88\xFC\x0D\xA2\xAF\x7F\x53\x07\x58\xF7\x4F\x1A\xB5\x63\x91\xAB\x5F\xF2\xAD\xBC\xC5\xBE\x2D\x6C\x7F\x49\xFB\xE8\x11\x81\x04\xC6\xFF\x9A\x23\xC6\xDF\xE5\x2F\x57\x95\x4E\x6A\x69\xDC\xEE\x5D\xB0\x6F\x51\x4F\x4A\x0A\x57\x2A\x9A\x85\x25\xD9\x61\xDA\xE7\x22\x69\xB9\x87\x18\x9D\x46\x5D\xF6\x10\x71\x19\xC7\xFA\x79\x08\x53\xE0\x63\xCB\xA0\xFA\xB7\x80\x0C\xA9\x32\xE2\x58\x88\x0F\xD7\x4C\x33\xC7\x84\x67\x5B\xED\xAD\x0E\x7C\x09\xE9\xCC\x4D\x63\xDD\x5E\x97\x13\xD5\xD4\xA0\x19\x6E\x6B\x56\x22\x26\xAC\x31\xB4\xF5\x7C\x04\xF9\x0A\x18\x19\x73\x73\x7D\xDC\x7E\x80\xF3\x64\x11\x2A\x9F\xBB\x43\x5E\xBD\xBC\xAB\xF7\xD4\x90\xCE\x52" + , "\xB2\xB7\x95\xFE\x6C\x1D\x4C\x83\xC1\x32\x7E\x01\x5A\x67\xD4\x46\x5F\xD8\xE3\x28\x13\x57\x5C\xBA\xB2\x63\xE2\x0E\xF0\x58\x64\xD2\xDC\x17\xE0\xE4\xEB\x81\x43\x6A\xDF\xE9\xF6\x38\xDC\xC1\xC8\xD7\x8F\x6B\x03\x06\xBA\xF9\x38\xE5\xD2\xAB\x0B\x3E\x05\xE7\x35\xCC\x6F\xFF\x2D\x6E\x02\xE3\xD6\x04\x84\xBE\xA7\xC7\xA8\xE1\x3E\x23\x19\x7F\xEA\x7B\x04\xD4\x7D\x48\xF4\xA4\xE5\x94\x41\x74\x53\x94\x92\x80\x0D\x3E\xF5\x1E\x2E\xE5\xE4\xC8\xA0\xBD\xF0\x50\xC2\xDD\x3D\xD7\x4F\xCE\x5E\x7E\x5C\x37\x36\x4F\x75\x47\xA1\x14\x80\xA3\x06\x3B\x9A\x0A\x15\x7B\x15\xB1\x0A\x5A\x95\x4D\xE2\x73\x1C\xED\x05\x5A\xA2\xE2\x76\x7F\x08\x91\xD4\x32\x9C\x42\x6F\x38\x08\xEE\x86\x7B\xED\x0D\xC7\x5B\x59\x22\xB7\xCF\xB8\x95\x70\x0F\xDA\x01\x61\x05\xA4\xC7\xB7\xF0\xBB\x90\xF0\x29\xF6\xBB\xCB\x04\xAC\x36\xAC\x16" + ) + ] + +-- Test vector from paper "Cryptography in NaCl" +vectorsCB :: [Vector] +vectorsCB = + [ + ( 20 + , "\x4A\x5D\x9D\x5B\xA4\xCE\x2D\xE1\x72\x8E\x3B\xF4\x80\x35\x0F\x25\xE0\x7E\x21\xC9\x47\xD1\x9E\x33\x76\xF0\x9B\x3C\x1E\x16\x17\x42" + , "\x69\x69\x6E\xE9\x55\xB6\x2B\x73\xCD\x62\xBD\xA8\x75\xFC\x73\xD6\x82\x19\xE0\x03\x6B\x7A\x0B\x37" + , "\xBE\x07\x5F\xC5\x3C\x81\xF2\xD5\xCF\x14\x13\x16\xEB\xEB\x0C\x7B\x52\x28\xC5\x2A\x4C\x62\xCB\xD4\x4B\x66\x84\x9B\x64\x24\x4F\xFC\xE5\xEC\xBA\xAF\x33\xBD\x75\x1A\x1A\xC7\x28\xD4\x5E\x6C\x61\x29\x6C\xDC\x3C\x01\x23\x35\x61\xF4\x1D\xB6\x6C\xCE\x31\x4A\xDB\x31\x0E\x3B\xE8\x25\x0C\x46\xF0\x6D\xCE\xEA\x3A\x7F\xA1\x34\x80\x57\xE2\xF6\x55\x6A\xD6\xB1\x31\x8A\x02\x4A\x83\x8F\x21\xAF\x1F\xDE\x04\x89\x77\xEB\x48\xF5\x9F\xFD\x49\x24\xCA\x1C\x60\x90\x2E\x52\xF0\xA0\x89\xBC\x76\x89\x70\x40\xE0\x82\xF9\x37\x76\x38\x48\x64\x5E\x07\x05" + , "\x8E\x99\x3B\x9F\x48\x68\x12\x73\xC2\x96\x50\xBA\x32\xFC\x76\xCE\x48\x33\x2E\xA7\x16\x4D\x96\xA4\x47\x6F\xB8\xC5\x31\xA1\x18\x6A\xC0\xDF\xC1\x7C\x98\xDC\xE8\x7B\x4D\xA7\xF0\x11\xEC\x48\xC9\x72\x71\xD2\xC2\x0F\x9B\x92\x8F\xE2\x27\x0D\x6F\xB8\x63\xD5\x17\x38\xB4\x8E\xEE\xE3\x14\xA7\xCC\x8A\xB9\x32\x16\x45\x48\xE5\x26\xAE\x90\x22\x43\x68\x51\x7A\xCF\xEA\xBD\x6B\xB3\x73\x2B\xC0\xE9\xDA\x99\x83\x2B\x61\xCA\x01\xB6\xDE\x56\x24\x4A\x9E\x88\xD5\xF9\xB3\x79\x73\xF6\x22\xA4\x3D\x14\xA6\x59\x9B\x1F\x65\x4C\xB4\x5A\x74\xE3\x55\xA5" + ) + ] + +spec :: Spec +spec = do + describe "KAT" $ + sequence_ $ + zipWith + (\i (r, k, n, p, e) -> it (show (i :: Int)) $ salsaRunSimple r k n p e) + [1 ..] + vectors + describe "crypto_box encryption" $ + sequence_ $ + zipWith + (\i (r, k, n, p, e) -> it (show (i :: Int)) $ cryptoBoxEnc r k n p e) + [1 ..] + vectorsCB + where + salsaRunSimple rounds key nonce plain expected = + let salsa = XSalsa.initialize rounds key nonce + in fst (XSalsa.combine salsa plain) `shouldBe` expected + + cryptoBoxEnc rounds shared nonce plain expected = + let zero = B.replicate 16 0 + (iv0, iv1) = B.splitAt 8 nonce + salsa0 = XSalsa.initialize rounds shared (zero `B.append` iv0) + salsa1 = XSalsa.derive salsa0 iv1 + (_, salsa2) = XSalsa.generate salsa1 32 :: (B.ByteString, XSalsa.State) + in fst (XSalsa.combine salsa2 plain) `shouldBe` expected diff --git a/tests/Tests.hs b/tests/Tests.hs deleted file mode 100644 --- a/tests/Tests.hs +++ /dev/null @@ -1,118 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module Main where - -import Imports - -import Crypto.System.CPU - -import qualified BCrypt -import qualified BCryptPBKDF -import qualified ChaCha -import qualified ChaChaPoly1305 -import qualified ECC -import qualified ECC.Edwards25519 -import qualified ECDSA -import qualified Hash -import qualified KAT_Argon2 -import qualified KAT_Blake2 -import qualified KAT_CMAC -import qualified KAT_Curve25519 -import qualified KAT_Curve448 -import qualified KAT_Ed25519 -import qualified KAT_Ed448 -import qualified KAT_EdDSA -import qualified KAT_HKDF -import qualified KAT_HMAC -import qualified KAT_KMAC -import qualified KAT_MiyaguchiPreneel -import qualified KAT_OTP -import qualified KAT_PBKDF2 -import qualified KAT_PubKey -import qualified KAT_Scrypt -import qualified Number -import qualified Number.F2m -import qualified Poly1305 -import qualified Salsa -import qualified XSalsa - --- symmetric cipher -------------------- -import qualified KAT_AES -import qualified KAT_AESGCMSIV -import qualified KAT_Blowfish -import qualified KAT_CAST5 -import qualified KAT_Camellia -import qualified KAT_DES -import qualified KAT_RC4 -import qualified KAT_TripleDES -import qualified KAT_Twofish - --- misc -------------------------------- -import qualified KAT_AFIS -import qualified Padding - -tests = - testGroup - "crypton" - [ testGroup - "runtime" - [ testCaseInfo "CPU" (return $ show processorOptions) - ] - , Number.tests - , Number.F2m.tests - , Hash.tests - , Padding.tests - , testGroup - "ConstructHash" - [ KAT_MiyaguchiPreneel.tests - ] - , testGroup - "MAC" - [ Poly1305.tests - , KAT_Blake2.tests - , KAT_CMAC.tests - , KAT_HMAC.tests - , KAT_KMAC.tests - ] - , KAT_Curve25519.tests - , KAT_Curve448.tests - , KAT_Ed25519.tests - , KAT_Ed448.tests - , KAT_EdDSA.tests - , KAT_PubKey.tests - , KAT_OTP.tests - , testGroup - "KDF" - [ KAT_PBKDF2.tests - , KAT_Scrypt.tests - , BCrypt.tests - , BCryptPBKDF.tests - , KAT_HKDF.tests - , KAT_Argon2.tests - ] - , testGroup - "block-cipher" - [ KAT_AES.tests - , KAT_AESGCMSIV.tests - , KAT_Blowfish.tests - , KAT_CAST5.tests - , KAT_Camellia.tests - , KAT_DES.tests - , KAT_TripleDES.tests - , KAT_Twofish.tests - ] - , testGroup - "stream-cipher" - [ KAT_RC4.tests - , ChaCha.tests - , ChaChaPoly1305.tests - , Salsa.tests - , XSalsa.tests - ] - , KAT_AFIS.tests - , ECC.tests - , ECC.Edwards25519.tests - , ECDSA.tests - ] - -main = defaultMain tests diff --git a/tests/Utils.hs b/tests/Utils.hs --- a/tests/Utils.hs +++ b/tests/Utils.hs @@ -13,8 +13,9 @@ import Data.Word import Prelude -import Test.Tasty.HUnit ((@=?)) -import Test.Tasty.QuickCheck +import Control.Monad (unless) +import Test.Hspec (Expectation, expectationFailure, shouldBe) +import Test.QuickCheck hiding (maxSize) newtype TestDRG = TestDRG (Word64, Word64, Word64, Word64, Word64) deriving (Show, Eq) @@ -158,5 +159,15 @@ | otherwise = (name ++ ": expected " ++ show a ++ " but got: " ++ show b) : acc -propertyHoldCase :: [PropertyTest] -> IO () -propertyHoldCase l = True @=? propertyHold l +propertyHoldCase :: [PropertyTest] -> Expectation +propertyHoldCase l = propertyHold l `shouldBe` True + +-- | The HUnit assertions the suite used under tasty, on top of hspec. +assertBool :: String -> Bool -> Expectation +assertBool msg b = unless b (expectationFailure msg) + +assertFailure :: String -> Expectation +assertFailure = expectationFailure + +assertEqual :: (Eq a, Show a) => String -> a -> a -> Expectation +assertEqual _ expected actual = actual `shouldBe` expected diff --git a/tests/XSalsa.hs b/tests/XSalsa.hs deleted file mode 100644 --- a/tests/XSalsa.hs +++ /dev/null @@ -1,173 +0,0 @@ -{-# LANGUAGE OverloadedStrings #-} - -module XSalsa (tests) where - -import qualified Crypto.Cipher.XSalsa as XSalsa -import qualified Data.ByteString as B - -import Imports - -type Vector = (Int, B.ByteString, B.ByteString, B.ByteString, B.ByteString) - --- Test vectors generated by naclcrypto library (https://nacl.cr.yp.to) -vectors :: [Vector] -vectors = - [ - ( 20 - , "\xA6\xA7\x25\x1C\x1E\x72\x91\x6D\x11\xC2\xCB\x21\x4D\x3C\x25\x25\x39\x12\x1D\x8E\x23\x4E\x65\x2D\x65\x1F\xA4\xC8\xCF\xF8\x80\x30" - , "\x9E\x64\x5A\x74\xE9\xE0\xA6\x0D\x82\x43\xAC\xD9\x17\x7A\xB5\x1A\x1B\xEB\x8D\x5A\x2F\x5D\x70\x0C" - , "\x09\x3C\x5E\x55\x85\x57\x96\x25\x33\x7B\xD3\xAB\x61\x9D\x61\x57\x60\xD8\xC5\xB2\x24\xA8\x5B\x1D\x0E\xFE\x0E\xB8\xA7\xEE\x16\x3A\xBB\x03\x76\x52\x9F\xCC\x09\xBA\xB5\x06\xC6\x18\xE1\x3C\xE7\x77\xD8\x2C\x3A\xE9\xD1\xA6\xF9\x72\xD4\x16\x02\x87\xCB\xFE\x60\xBF\x21\x30\xFC\x0A\x6F\xF6\x04\x9D\x0A\x5C\x8A\x82\xF4\x29\x23\x1F\x00\x80\x82\xE8\x45\xD7\xE1\x89\xD3\x7F\x9E\xD2\xB4\x64\xE6\xB9\x19\xE6\x52\x3A\x8C\x12\x10\xBD\x52\xA0\x2A\x4C\x3F\xE4\x06\xD3\x08\x5F\x50\x68\xD1\x90\x9E\xEE\xCA\x63\x69\xAB\xC9\x81\xA4\x2E\x87\xFE\x66\x55\x83\xF0\xAB\x85\xAE\x71\xF6\xF8\x4F\x52\x8E\x6B\x39\x7A\xF8\x6F\x69\x17\xD9\x75\x4B\x73\x20\xDB\xDC\x2F\xEA\x81\x49\x6F\x27\x32\xF5\x32\xAC\x78\xC4\xE9\xC6\xCF\xB1\x8F\x8E\x9B\xDF\x74\x62\x2E\xB1\x26\x14\x14\x16\x77\x69\x71\xA8\x4F\x94\xD1\x56\xBE\xAF\x67\xAE\xCB\xF2\xAD\x41\x2E\x76\xE6\x6E\x8F\xAD\x76\x33\xF5\xB6\xD7\xF3\xD6\x4B\x5C\x6C\x69\xCE\x29\x00\x3C\x60\x24\x46\x5A\xE3\xB8\x9B\xE7\x8E\x91\x5D\x88\xB4\xB5\x62\x1D" - , "\xB2\xAF\x68\x8E\x7D\x8F\xC4\xB5\x08\xC0\x5C\xC3\x9D\xD5\x83\xD6\x71\x43\x22\xC6\x4D\x7F\x3E\x63\x14\x7A\xED\xE2\xD9\x53\x49\x34\xB0\x4F\xF6\xF3\x37\xB0\x31\x81\x5C\xD0\x94\xBD\xBC\x6D\x7A\x92\x07\x7D\xCE\x70\x94\x12\x28\x68\x22\xEF\x07\x37\xEE\x47\xF6\xB7\xFF\xA2\x2F\x9D\x53\xF1\x1D\xD2\xB0\xA3\xBB\x9F\xC0\x1D\x9A\x88\xF9\xD5\x3C\x26\xE9\x36\x5C\x2C\x3C\x06\x3B\xC4\x84\x0B\xFC\x81\x2E\x4B\x80\x46\x3E\x69\xD1\x79\x53\x0B\x25\xC1\x58\xF5\x43\x19\x1C\xFF\x99\x31\x06\x51\x1A\xA0\x36\x04\x3B\xBC\x75\x86\x6A\xB7\xE3\x4A\xFC\x57\xE2\xCC\xE4\x93\x4A\x5F\xAA\xE6\xEA\xBE\x4F\x22\x17\x70\x18\x3D\xD0\x60\x46\x78\x27\xC2\x7A\x35\x41\x59\xA0\x81\x27\x5A\x29\x1F\x69\xD9\x46\xD6\xFE\x28\xED\x0B\x9C\xE0\x82\x06\xCF\x48\x49\x25\xA5\x1B\x94\x98\xDB\xDE\x17\x8D\xDD\x3A\xE9\x1A\x85\x81\xB9\x16\x82\xD8\x60\xF8\x40\x78\x2F\x6E\xEA\x49\xDB\xB9\xBD\x72\x15\x01\xD2\xC6\x71\x22\xDE\xA3\xB7\x28\x38\x48\xC5\xF1\x3E\x0C\x0D\xE8\x76\xBD\x22\x7A\x85\x6E\x4D\xE5\x93\xA3" - ) - , - ( 20 - , "\x9E\x1D\xA2\x39\xD1\x55\xF5\x2A\xD3\x7F\x75\xC7\x36\x8A\x53\x66\x68\xB0\x51\x95\x29\x23\xAD\x44\xF5\x7E\x75\xAB\x58\x8E\x47\x5A" - , "\xAF\x06\xF1\x78\x59\xDF\xFA\x79\x98\x91\xC4\x28\x8F\x66\x35\xB5\xC5\xA4\x5E\xEE\x90\x17\xFD\x72" - , "\xFE\xAC\x9D\x54\xFC\x8C\x11\x5A\xE2\x47\xD9\xA7\xE9\x19\xDD\x76\xCF\xCB\xC7\x2D\x32\xCA\xE4\x94\x48\x60\x81\x7C\xBD\xFB\x8C\x04\xE6\xB1\xDF\x76\xA1\x65\x17\xCD\x33\xCC\xF1\xAC\xDA\x92\x06\x38\x9E\x9E\x31\x8F\x59\x66\xC0\x93\xCF\xB3\xEC\x2D\x9E\xE2\xDE\x85\x64\x37\xED\x58\x1F\x55\x2F\x26\xAC\x29\x07\x60\x9D\xF8\xC6\x13\xB9\xE3\x3D\x44\xBF\xC2\x1F\xF7\x91\x53\xE9\xEF\x81\xA9\xD6\x6C\xC3\x17\x85\x7F\x75\x2C\xC1\x75\xFD\x88\x91\xFE\xFE\xBB\x7D\x04\x1E\x65\x17\xC3\x16\x2D\x19\x7E\x21\x12\x83\x7D\x3B\xC4\x10\x43\x12\xAD\x35\xB7\x5E\xA6\x86\xE7\xC7\x0D\x4E\xC0\x47\x46\xB5\x2F\xF0\x9C\x42\x14\x51\x45\x9F\xB5\x9F" - , "\x2C\x26\x1A\x2F\x4E\x61\xA6\x2E\x1B\x27\x68\x99\x16\xBF\x03\x45\x3F\xCB\xC9\x7B\xB2\xAF\x6F\x32\x93\x91\xEF\x06\x3B\x5A\x21\x9B\xF9\x84\xD0\x7D\x70\xF6\x02\xD8\x5F\x6D\xB6\x14\x74\xE9\xD9\xF5\xA2\xDE\xEC\xB4\xFC\xD9\x01\x84\xD1\x6F\x3B\x5B\x5E\x16\x8E\xE0\x3E\xA8\xC9\x3F\x39\x33\xA2\x2B\xC3\xD1\xA5\xAE\x8C\x2D\x8B\x02\x75\x7C\x87\xC0\x73\x40\x90\x52\xA2\xA8\xA4\x1E\x7F\x48\x7E\x04\x1F\x9A\x49\xA0\x99\x7B\x54\x0E\x18\x62\x1C\xAD\x3A\x24\xF0\xA5\x6D\x9B\x19\x22\x79\x29\x05\x7A\xB3\xBA\x95\x0F\x62\x74\xB1\x21\xF1\x93\xE3\x2E\x06\xE5\x38\x87\x81\xA1\xCB\x57\x31\x7C\x0B\xA6\x30\x5E\x91\x09\x61\xD0\x10\x02\xF0" - ) - , - ( 20 - , "\xD5\xC7\xF6\x79\x7B\x7E\x7E\x9C\x1D\x7F\xD2\x61\x0B\x2A\xBF\x2B\xC5\xA7\x88\x5F\xB3\xFF\x78\x09\x2F\xB3\xAB\xE8\x98\x6D\x35\xE2" - , "\x74\x4E\x17\x31\x2B\x27\x96\x9D\x82\x64\x44\x64\x0E\x9C\x4A\x37\x8A\xE3\x34\xF1\x85\x36\x9C\x95" - , "\x77\x58\x29\x8C\x62\x8E\xB3\xA4\xB6\x96\x3C\x54\x45\xEF\x66\x97\x12\x22\xBE\x5D\x1A\x4A\xD8\x39\x71\x5D\x11\x88\x07\x17\x39\xB7\x7C\xC6\xE0\x5D\x54\x10\xF9\x63\xA6\x41\x67\x62\x97\x57" - , "\x27\xB8\xCF\xE8\x14\x16\xA7\x63\x01\xFD\x1E\xEC\x6A\x4D\x99\x67\x50\x69\xB2\xDA\x27\x76\xC3\x60\xDB\x1B\xDF\xEA\x7C\x0A\xA6\x13\x91\x3E\x10\xF7\xA6\x0F\xEC\x04\xD1\x1E\x65\xF2\xD6\x4E" - ) - , - ( 20 - , "\x73\x7D\x78\x11\xCE\x96\x47\x2E\xFE\xD1\x22\x58\xB7\x81\x22\xF1\x1D\xEA\xEC\x87\x59\xCC\xBD\x71\xEA\xC6\xBB\xEF\xA6\x27\x78\x5C" - , "\x6F\xB2\xEE\x3D\xDA\x6D\xBD\x12\xF1\x27\x4F\x12\x67\x01\xEC\x75\xC3\x5C\x86\x60\x7A\xDB\x3E\xDD" - , "\x50\x13\x25\xFB\x26\x45\x26\x48\x64\xDF\x11\xFA\xA1\x7B\xBD\x58\x31\x2B\x77\xCA\xD3\xD9\x4A\xC8\xFB\x85\x42\xF0\xEB\x65\x3A\xD7\x3D\x7F\xCE\x93\x2B\xB8\x74\xCB\x89\xAC\x39\xFC\x47\xF8\x26\x7C\xF0\xF0\xC2\x09\xF2\x04\xB2\xD8\x57\x8A\x3B\xDF\x46\x1C\xB6\xA2\x71\xA4\x68\xBE\xBA\xCC\xD9\x68\x50\x14\xCC\xBC\x9A\x73\x61\x8C\x6A\x5E\x77\x8A\x21\xCC\x84\x16\xC6\x0A\xD2\x4D\xDC\x41\x7A\x13\x0D\x53\xED\xA6\xDF\xBF\xE4\x7D\x09\x17\x0A\x7B\xE1\xA7\x08\xB7\xB5\xF3\xAD\x46\x43\x10\xBE\x36\xD9\xA2\xA9\x5D\xC3\x9E\x83\xD3\x86\x67\xE8\x42\xEB\x64\x11\xE8\xA2\x37\x12\x29\x7B\x16\x5F\x69\x0C\x2D\x7C\xA1\xB1\x34\x6E\x3C\x1F\xCC\xF5\xCA\xFD\x4F\x8B\xE0" - , "\x67\x24\xC3\x72\xD2\xE9\x07\x4D\xA5\xE2\x7A\x6C\x54\xB2\xD7\x03\xDC\x1D\x4C\x9B\x1F\x8D\x90\xF0\x0C\x12\x2E\x69\x2A\xCE\x77\x00\xEA\xDC\xA9\x42\x54\x45\x07\xF1\x37\x5B\x65\x81\xD5\xA8\xFB\x39\x98\x1C\x1C\x0E\x6E\x1F\xF2\x14\x0B\x08\x2E\x9E\xC0\x16\xFC\xE1\x41\xD5\x19\x96\x47\xD4\x3B\x0B\x68\xBF\xD0\xFE\xA5\xE0\x0F\x46\x89\x62\xC7\x38\x4D\xD6\x12\x9A\xEA\x6A\x3F\xDF\xE7\x5A\xBB\x21\x0E\xD5\x60\x7C\xEF\x8F\xA0\xE1\x52\x83\x3D\x5A\xC3\x7D\x52\xE5\x57\xB9\x10\x98\xA3\x22\xE7\x6A\x45\xBB\xBC\xF4\x89\x9E\x79\x06\x18\xAA\x3F\x4C\x2E\x5E\x0F\xC3\xDE\x93\x26\x9A\x57\x7D\x77\xA5\x50\x2E\x8E\xA0\x2F\x71\x7B\x1D\xD2\xDF\x1E\xC6\x9D\x8B\x61\xCA" - ) - , - ( 20 - , "\x76\x01\x58\xDA\x09\xF8\x9B\xBA\xB2\xC9\x9E\x69\x97\xF9\x52\x3A\x95\xFC\xEF\x10\x23\x9B\xCC\xA2\x57\x3B\x71\x05\xF6\x89\x8D\x34" - , "\x43\x63\x6B\x2C\xC3\x46\xFC\x8B\x7C\x85\xA1\x9B\xF5\x07\xBD\xC3\xDA\xFE\x95\x3B\x88\xC6\x9D\xBA" - , "\xD3\x0A\x6D\x42\xDF\xF4\x9F\x0E\xD0\x39\xA3\x06\xBA\xE9\xDE\xC8\xD9\xE8\x83\x66\xCC\x19\xE8\xC3\x64\x2F\xD5\x8F\xA0\x79\x4E\xBF\x80\x29\xD9\x49\x73\x03\x39\xB0\x82\x3A\x51\xF0\xF4\x9F\x0D\x2C\x71\xF1\x05\x1C\x1E\x0E\x2C\x86\x94\x1F\x17\x27\x89\xCD\xB1\xB0\x10\x74\x13\xE7\x0F\x98\x2F\xF9\x76\x18\x77\xBB\x52\x6E\xF1\xC3\xEB\x11\x06\xA9\x48\xD6\x0E\xF2\x1B\xD3\x5D\x32\xCF\xD6\x4F\x89\xB7\x9E\xD6\x3E\xCC\x5C\xCA\x56\x24\x6A\xF7\x36\x76\x6F\x28\x5D\x8E\x6B\x0D\xA9\xCB\x1C\xD2\x10\x20\x22\x3F\xFA\xCC\x5A\x32" - , "\xC8\x15\xB6\xB7\x9B\x64\xF9\x36\x9A\xEC\x8D\xCE\x8C\x75\x3D\xF8\xA5\x0F\x2B\xC9\x7C\x70\xCE\x2F\x01\x4D\xB3\x3A\x65\xAC\x58\x16\xBA\xC9\xE3\x0A\xC0\x8B\xDD\xED\x30\x8C\x65\xCB\x87\xE2\x8E\x2E\x71\xB6\x77\xDC\x25\xC5\xA6\x49\x9C\x15\x53\x55\x5D\xAF\x1F\x55\x27\x0A\x56\x95\x9D\xFF\xA0\xC6\x6F\x24\xE0\xAF\x00\x95\x1E\xC4\xBB\x59\xCC\xC3\xA6\xC5\xF5\x2E\x09\x81\x64\x7E\x53\xE4\x39\x31\x3A\x52\xC4\x0F\xA7\x00\x4C\x85\x5B\x6E\x6E\xB2\x5B\x21\x2A\x13\x8E\x84\x3A\x9B\xA4\x6E\xDB\x2A\x03\x9E\xE8\x2A\x26\x3A\xBE" - ) - , - ( 20 - , "\x27\xBA\x7E\x81\xE7\xED\xD4\xE7\x1B\xE5\x3C\x07\xCE\x8E\x63\x31\x38\xF2\x87\xE1\x55\xC7\xFA\x9E\x84\xC4\xAD\x80\x4B\x7F\xA1\xB9" - , "\xEA\x05\xF4\xEB\xCD\x2F\xB6\xB0\x00\xDA\x06\x12\x86\x1B\xA5\x4F\xF5\xC1\x76\xFB\x60\x13\x91\xAA" - , "\xE0\x9F\xF5\xD2\xCB\x05\x0D\x69\xB2\xD4\x24\x94\xBD\xE5\x82\x52\x38\xC7\x56\xD6\x99\x1D\x99\xD7\xA2\x0D\x1E\xF0\xB8\x3C\x37\x1C\x89\x87\x26\x90\xB2\xFC\x11\xD5\x36\x9F\x4F\xC4\x97\x1B\x6D\x3D\x6C\x07\x8A\xEF\x9B\x0F\x05\xC0\xE6\x1A\xB8\x9C\x02\x51\x68\x05\x4D\xEF\xEB\x03\xFE\xF6\x33\x85\x87\x00\xC5\x8B\x12\x62\xCE\x01\x13\x00\x01\x26\x73\xE8\x93\xE4\x49\x01\xDC\x18\xEE\xE3\x10\x56\x99\xC4\x4C\x80\x58\x97\xBD\xAF\x77\x6A\xF1\x83\x31\x62\xA2\x1A" - , "\xA2\x3E\x7E\xF9\x3C\x5D\x06\x67\xC9\x6D\x9E\x40\x4D\xCB\xE6\xBE\x62\x02\x6F\xA9\x8F\x7A\x3F\xF9\xBA\x5D\x45\x86\x43\xA1\x6A\x1C\xEF\x72\x72\xDC\x60\x97\xA9\xB5\x2F\x35\x98\x35\x57\xC7\x7A\x11\xB3\x14\xB4\xF7\xD5\xDC\x2C\xCA\x15\xEE\x47\x61\x6F\x86\x18\x73\xCB\xFE\xD1\xD3\x23\x72\x17\x1A\x61\xE3\x8E\x44\x7F\x3C\xF3\x62\xB3\xAB\xBB\x2E\xD4\x17\x0D\x89\xDC\xB2\x81\x87\xB7\xBF\xD2\x06\xA3\xE0\x26\xF0\x84\xA7\xE0\xED\x63\xD3\x19\xDE\x6B\xC9\xAF\xC0" - ) - , - ( 20 - , "\x67\x99\xD7\x6E\x5F\xFB\x5B\x49\x20\xBC\x27\x68\xBA\xFD\x3F\x8C\x16\x55\x4E\x65\xEF\xCF\x9A\x16\xF4\x68\x3A\x7A\x06\x92\x7C\x11" - , "\x61\xAB\x95\x19\x21\xE5\x4F\xF0\x6D\x9B\x77\xF3\x13\xA4\xE4\x9D\xF7\xA0\x57\xD5\xFD\x62\x79\x89" - , "\x47\x27\x66" - , "\x8F\xD7\xDF" - ) - , - ( 20 - , "\xF6\x82\x38\xC0\x83\x65\xBB\x29\x3D\x26\x98\x0A\x60\x64\x88\xD0\x9C\x2F\x10\x9E\xDA\xFA\x0B\xBA\xE9\x93\x7B\x5C\xC2\x19\xA4\x9C" - , "\x51\x90\xB5\x1E\x9B\x70\x86\x24\x82\x0B\x5A\xBD\xF4\xE4\x0F\xAD\x1F\xB9\x50\xAD\x1A\xDC\x2D\x26" - , "\x47\xEC\x6B\x1F\x73\xC4\xB7\xFF\x52\x74\xA0\xBF\xD7\xF4\x5F\x86\x48\x12\xC8\x5A\x12\xFB\xCB\x3C\x2C\xF8\xA3\xE9\x0C\xF6\x6C\xCF\x2E\xAC\xB5\x21\xE7\x48\x36\x3C\x77\xF5\x2E\xB4\x26\xAE\x57\xA0\xC6\xC7\x8F\x75\xAF\x71\x28\x45\x69\xE7\x9D\x1A\x92\xF9\x49\xA9\xD6\x9C\x4E\xFC\x0B\x69\x90\x2F\x1E\x36\xD7\x56\x27\x65\x54\x3E\x2D\x39\x42\xD9\xF6\xFF\x59\x48\xD8\xA3\x12\xCF\xF7\x2C\x1A\xFD\x9E\xA3\x08\x8A\xFF\x76\x40\xBF\xD2\x65\xF7\xA9\x94\x6E\x60\x6A\xBC\x77\xBC\xED\xAE\x6B\xDD\xC7\x5A\x0D\xBA\x0B\xD9\x17\xD7\x3E\x3B\xD1\x26\x8F\x72\x7E\x00\x96\x34\x5D\xA1\xED\x25\xCF\x55\x3E\xA7\xA9\x8F\xEA\x6B\x6F\x28\x57\x32\xDE\x37\x43\x15\x61\xEE\x1B\x30\x64\x88\x7F\xBC\xBD\x71\x93\x5E\x02" - , "\x36\x16\x0E\x88\xD3\x50\x05\x29\xBA\x4E\xDB\xA1\x7B\xC2\x4D\x8C\xFA\xCA\x9A\x06\x80\xB3\xB1\xFC\x97\xCF\x03\xF3\x67\x5B\x7A\xC3\x01\xC8\x83\xA6\x8C\x07\x1B\xC5\x4A\xCD\xD3\xB6\x3A\xF4\xA2\xD7\x2F\x98\x5E\x51\xF9\xD6\x0A\x4C\x7F\xD4\x81\xAF\x10\xB2\xFC\x75\xE2\x52\xFD\xEE\x7E\xA6\xB6\x45\x31\x90\x61\x7D\xCC\x6E\x2F\xE1\xCD\x56\x58\x5F\xC2\xF0\xB0\xE9\x7C\x5C\x3F\x8A\xD7\xEB\x4F\x31\xBC\x48\x90\xC0\x38\x82\xAA\xC2\x4C\xC5\x3A\xCC\x19\x82\x29\x65\x26\x69\x0A\x22\x02\x71\xC2\xF6\xE3\x26\x75\x0D\x3F\xBD\xA5\xD5\xB6\x35\x12\xC8\x31\xF6\x78\x30\xF5\x9A\xC4\x9A\xAE\x33\x0B\x3E\x0E\x02\xC9\xEA\x00\x91\xD1\x98\x41\xF1\xB0\xE1\x3D\x69\xC9\xFB\xFE\x8A\x12\xD6\xF3\x0B\xB7\x34\xD9\xD2" - ) - , - ( 20 - , "\x45\xB2\xBD\x0D\xE4\xED\x92\x93\xEC\x3E\x26\xC4\x84\x0F\xAA\xF6\x4B\x7D\x61\x9D\x51\xE9\xD7\xA2\xC7\xE3\x6C\x83\xD5\x84\xC3\xDF" - , "\x54\x6C\x8C\x5D\x6B\xE8\xF9\x09\x52\xCA\xB3\xF3\x6D\x7C\x19\x57\xBA\xAA\x7A\x59\xAB\xE3\xD7\xE5" - , "\x50\x07\xC8\xCD\x5B\x3C\x40\xE1\x7D\x7F\xE4\x23\xA8\x7A\xE0\xCE\xD8\x6B\xEC\x1C\x39\xDC\x07\xA2\x57\x72\xF3\xE9\x6D\xAB\xD5\x6C\xD3\xFD\x73\x19\xF6\xC9\x65\x49\x25\xF2\xD8\x70\x87\xA7\x00\xE1\xB1\x30\xDA\x79\x68\x95\xD1\xC9\xB9\xAC\xD6\x2B\x26\x61\x44\x06\x7D\x37\x3E\xD5\x1E\x78\x74\x98\xB0\x3C\x52\xFA\xAD\x16\xBB\x38\x26\xFA\x51\x1B\x0E\xD2\xA1\x9A\x86\x63\xF5\xBA\x2D\x6E\xA7\xC3\x8E\x72\x12\xE9\x69\x7D\x91\x48\x6C\x49\xD8\xA0\x00\xB9\xA1\x93\x5D\x6A\x7F\xF7\xEF\x23\xE7\x20\xA4\x58\x55\x48\x14\x40\x46\x3B\x4A\xC8\xC4\xF6\xE7\x06\x2A\xDC\x1F\x1E\x1E\x25\xD3\xD6\x5A\x31\x81\x2F\x58\xA7\x11\x60" - , "\x8E\xAC\xFB\xA5\x68\x89\x8B\x10\xC0\x95\x7A\x7D\x44\x10\x06\x85\xE8\x76\x3A\x71\xA6\x9A\x8D\x16\xBC\x7B\x3F\x88\x08\x5B\xB9\xA2\xF0\x96\x42\xE4\xD0\x9A\x9F\x0A\xD0\x9D\x0A\xAD\x66\xB2\x26\x10\xC8\xBD\x02\xFF\x66\x79\xBB\x92\xC2\xC0\x26\xA2\x16\xBF\x42\x5C\x6B\xE3\x5F\xB8\xDA\xE7\xFF\x0C\x72\xB0\xEF\xD6\xA1\x80\x37\xC7\x0E\xED\x0C\xA9\x00\x62\xA4\x9A\x3C\x97\xFD\xC9\x0A\x8F\x9C\x2E\xA5\x36\xBF\xDC\x41\x91\x8A\x75\x82\xC9\x92\x7F\xAE\x47\xEF\xAA\x3D\xC8\x79\x67\xB7\x88\x7D\xEE\x1B\xF0\x71\x73\x4C\x76\x65\x90\x1D\x91\x05\xDA\xE2\xFD\xF6\x6B\x49\x18\xE5\x1D\x8F\x4A\x48\xC6\x0D\x19\xFB\xFB\xBC\xBA" - ) - , - ( 20 - , "\xFE\x55\x9C\x9A\x28\x2B\xEB\x40\x81\x4D\x01\x6D\x6B\xFC\xB2\xC0\xC0\xD8\xBF\x07\x7B\x11\x10\xB8\x70\x3A\x3C\xE3\x9D\x70\xE0\xE1" - , "\xB0\x76\x20\x0C\xC7\x01\x12\x59\x80\x5E\x18\xB3\x04\x09\x27\x54\x00\x27\x23\xEB\xEC\x5D\x62\x00" - , "\x6D\xB6\x5B\x9E\xC8\xB1\x14\xA9\x44\x13\x7C\x82\x1F\xD6\x06\xBE\x75\x47\x8D\x92\x83\x66\xD5\x28\x40\x96\xCD\xEF\x78\x2F\xCF\xF7\xE8\xF5\x9C\xB8\xFF\xCD\xA9\x79\x75\x79\x02\xC5\xFF\xA6\xBC\x47\x7C\xEA\xA4\xCB\x5D\x5E\xA7\x6F\x94\xD9\x1E\x83\x3F\x82\x3A\x6B\xC7\x8F\x10\x55\xDF\xA6\xA9\x7B\xEA\x89\x65\xC1\xCD\xE6\x7A\x66\x8E\x00\x12\x57\x33\x4A\x58\x57\x27\xD9\xE0\xF7\xC1\xA0\x6E\x88\xD3\xD2\x5A\x4E\x6D\x90\x96\xC9\x68\xBF\x13\x8E\x11\x6A\x3E\xBE\xFF\xD4\xBB\x48\x08\xAD\xB1\xFD\x69\x81\x64\xBA\x0A\x35\xC7\x09\xA4\x7F\x16\xF1\xF4\x43\x5A\x23\x45\xA9\x19\x4A\x00\xB9\x5A\xBD\x51\x85\x1D\x50\x58\x09\xA6\x07\x7D\xA9\xBA\xCA\x58\x31\xAF\xFF\x31\x57\x8C\x48\x7E\xE6\x8F\x27\x67\x97\x4A\x98\xA7\xE8\x03\xAA\xC7\x88\xDA\x98\x31\x9C\x4E\xA8\xEA\xA3\xD3\x94\x85\x56\x51\xF4\x84\xCE\xF5\x43\xF5\x37\xE3\x51\x58\xEE\x29" - , "\x4D\xCE\x9C\x8F\x97\xA0\x28\x05\x1B\x07\x27\xF3\x4E\x1B\x9E\xF2\x1F\x06\xF0\x76\x0F\x36\xE7\x17\x13\x20\x40\x27\x90\x20\x90\xBA\x2B\xB6\xB1\x34\x36\xEE\x77\x8D\x9F\x50\x53\x0E\xFB\xD7\xA3\x2B\x0D\x41\x44\x3F\x58\xCC\xAE\xE7\x81\xC7\xB7\x16\xD3\xA9\x6F\xDE\xC0\xE3\x76\x4E\xD7\x95\x9F\x34\xC3\x94\x12\x78\x59\x1E\xA0\x33\xB5\xCB\xAD\xC0\xF1\x91\x60\x32\xE9\xBE\xBB\xD1\xA8\x39\x5B\x83\xFB\x63\xB1\x45\x4B\xD7\x75\xBD\x20\xB3\xA2\xA9\x6F\x95\x12\x46\xAC\x14\xDA\xF6\x81\x66\xBA\x62\xF6\xCB\xFF\x8B\xD1\x21\xAC\x94\x98\xFF\x88\x52\xFD\x2B\xE9\x75\xDF\x52\xB5\xDA\xEF\x38\x29\xD1\x8E\xDA\x42\xE7\x15\x02\x2D\xCB\xF9\x30\xD0\xA7\x89\xEE\x6A\x14\x6C\x2C\x70\x88\xC3\x57\x73\xC6\x3C\x06\xB4\xAF\x45\x59\x85\x6A\xC1\x99\xCE\xD8\x68\x63\xE4\x29\x47\x07\x82\x53\x37\xC5\x85\x79\x70\xEB\x7F\xDD\xEB\x26\x37\x81\x30\x90\x11" - ) - , - ( 20 - , "\x0A\xE1\x00\x12\xD7\xE5\x66\x14\xB0\x3D\xCC\x89\xB1\x4B\xAE\x92\x42\xFF\xE6\x30\xF3\xD7\xE3\x5C\xE8\xBB\xB9\x7B\xBC\x2C\x92\xC3" - , "\xF9\x6B\x02\x5D\x6C\xF4\x6A\x8A\x12\xAC\x2A\xF1\xE2\xAE\xF1\xFB\x83\x59\x0A\xDA\xDA\xA5\xC5\xEA" - , "\xEA\x0F\x35\x4E\x96\xF1\x2B\xC7\x2B\xBA\xA3\xD1\x2B\x4A\x8E\xD8\x79\xB0\x42\xF0\x68\x98\x78\xF4\x6B\x65\x1C\xC4\x11\x6D\x6F\x78\x40\x9B\x11\x43\x0B\x3A\xAA\x30\xB2\x07\x68\x91\xE8\xE1\xFA\x52\x8F\x2F\xD1\x69\xED\x93\xDC\x9F\x84\xE2\x44\x09\xEE\xC2\x10\x1D\xAF\x4D\x05\x7B\xE2\x49\x2D\x11\xDE\x64\x0C\xBD\x7B\x35\x5A\xD2\x9F\xB7\x04\x00\xFF\xFD\x7C\xD6\xD4\x25\xAB\xEE\xB7\x32\xA0\xEA\xA4\x33\x0A\xF4\xC6\x56\x25\x2C\x41\x73\xDE\xAB\x65\x3E\xB8\x5C\x58\x46\x2D\x7A\xB0\xF3\x5F\xD1\x2B\x61\x3D\x29\xD4\x73\xD3\x30\x31\x0D\xC3\x23\xD3\xC6\x63\x48\xBB\xDB\xB6\x8A\x32\x63\x24\x65\x7C\xAE\x7B\x77\xA9\xE3\x43\x58\xF2\xCE\xC5\x0C\x85\x60\x9E\x73\x05\x68\x56\x79\x6E\x3B\xE8\xD6\x2B\x6E\x2F\xE9\xF9\x53" - , "\xE8\xAB\xD4\x89\x24\xB5\x4E\x5B\x80\x86\x6B\xE7\xD4\xEB\xE5\xCF\x42\x74\xCA\xFF\xF0\x8B\x39\xCB\x2D\x40\xA8\xF0\xB4\x72\x39\x8A\xED\xC7\x76\xE0\x79\x38\x12\xFB\xF1\xF6\x00\x78\x63\x5D\x2E\xD8\x6B\x15\xEF\xCD\xBA\x60\x41\x1E\xE2\x3B\x07\x23\x35\x92\xA4\x4E\xC3\x1B\x10\x13\xCE\x89\x64\x23\x66\x75\xF8\xF1\x83\xAE\xF8\x85\xE8\x64\xF2\xA7\x2E\xDF\x42\x15\xB5\x33\x8F\xA2\xB5\x46\x53\xDF\xA1\xA8\xC5\x5C\xE5\xD9\x5C\xC6\x05\xB9\xB3\x11\x52\x7F\x2E\x34\x63\xFF\xBE\xC7\x8A\x9D\x1D\x65\xDA\xBA\xD2\xF3\x38\x76\x9C\x9F\x43\xF1\x33\xA7\x91\xA1\x1C\x7E\xCA\x9A\xF0\xB7\x71\xA4\xAC\x32\x96\x3D\xC8\xF6\x31\xA2\xC1\x12\x17\xAC\x6E\x1B\x94\x30\xC1\xAA\xE1\xCE\xEB\xE2\x27\x03\xF4\x29\x99\x8A\x8F\xB8\xC6\x41" - ) - , - ( 20 - , "\x08\x2C\x53\x9B\xC5\xB2\x0F\x97\xD7\x67\xCD\x3F\x22\x9E\xDA\x80\xB2\xAD\xC4\xFE\x49\xC8\x63\x29\xB5\xCD\x62\x50\xA9\x87\x74\x50" - , "\x84\x55\x43\x50\x2E\x8B\x64\x91\x2D\x8F\x2C\x8D\x9F\xFF\xB3\xC6\x93\x65\x68\x65\x87\xC0\x8D\x0C" - , "\xA9\x6B\xB7\xE9\x10\x28\x1A\x6D\xFA\xD7\xC8\xA9\xC3\x70\x67\x4F\x0C\xEE\xC1\xAD\x8D\x4F\x0D\xE3\x2F\x9A\xE4\xA2\x3E\xD3\x29\xE3\xD6\xBC\x70\x8F\x87\x66\x40\xA2\x29\x15\x3A\xC0\xE7\x28\x1A\x81\x88\xDD\x77\x69\x51\x38\xF0\x1C\xDA\x5F\x41\xD5\x21\x5F\xD5\xC6\xBD\xD4\x6D\x98\x2C\xB7\x3B\x1E\xFE\x29\x97\x97\x0A\x9F\xDB\xDB\x1E\x76\x8D\x7E\x5D\xB7\x12\x06\x8D\x8B\xA1\xAF\x60\x67\xB5\x75\x34\x95\xE2\x3E\x6E\x19\x63\xAF\x01\x2F\x9C\x7C\xE4\x50\xBF\x2D\xE6\x19\xD3\xD5\x95\x42\xFB\x55\xF3" - , "\x83\x5D\xA7\x4F\xC6\xDE\x08\xCB\xDA\x27\x7A\x79\x66\xA0\x7C\x8D\xCD\x62\x7E\x7B\x17\xAD\xDE\x6D\x93\x0B\x65\x81\xE3\x12\x4B\x8B\xAA\xD0\x96\xF6\x93\x99\x1F\xED\xB1\x57\x29\x30\x60\x1F\xC7\x70\x95\x41\x83\x9B\x8E\x3F\xFD\x5F\x03\x3D\x20\x60\xD9\x99\xC6\xC6\xE3\x04\x82\x76\x61\x3E\x64\x80\x00\xAC\xB5\x21\x2C\xC6\x32\xA9\x16\xAF\xCE\x29\x0E\x20\xEB\xDF\x61\x2D\x08\xA6\xAA\x4C\x79\xA7\x4B\x07\x0D\x3F\x87\x2A\x86\x1F\x8D\xC6\xBB\x07\x61\x4D\xB5\x15\xD3\x63\x34\x9D\x3A\x8E\x33\x36\xA3" - ) - , - ( 20 - , "\x3D\x02\xBF\xF3\x37\x5D\x40\x30\x27\x35\x6B\x94\xF5\x14\x20\x37\x37\xEE\x9A\x85\xD2\x05\x2D\xB3\xE4\xE5\xA2\x17\xC2\x59\xD1\x8A" - , "\x74\x21\x6C\x95\x03\x18\x95\xF4\x8C\x1D\xBA\x65\x15\x55\xEB\xFA\x3C\xA3\x26\xA7\x55\x23\x70\x25" - , "\x0D\x4B\x0F\x54\xFD\x09\xAE\x39\xBA\xA5\xFA\x4B\xAC\xCF\x2E\x66\x82\xE6\x1B\x25\x7E\x01\xF4\x2B\x8F" - , "\x16\xC4\x00\x6C\x28\x36\x51\x90\x41\x1E\xB1\x59\x38\x14\xCF\x15\xE7\x4C\x22\x23\x8F\x21\x0A\xFC\x3D" - ) - , - ( 20 - , "\xAD\x1A\x5C\x47\x68\x88\x74\xE6\x66\x3A\x0F\x3F\xA1\x6F\xA7\xEF\xB7\xEC\xAD\xC1\x75\xC4\x68\xE5\x43\x29\x14\xBD\xB4\x80\xFF\xC6" - , "\xE4\x89\xEE\xD4\x40\xF1\xAA\xE1\xFA\xC8\xFB\x7A\x98\x25\x63\x54\x54\xF8\xF8\xF1\xF5\x2E\x2F\xCC" - , "\xAA\x6C\x1E\x53\x58\x0F\x03\xA9\xAB\xB7\x3B\xFD\xAD\xED\xFE\xCA\xDA\x4C\x6B\x0E\xBE\x02\x0E\xF1\x0D\xB7\x45\xE5\x4B\xA8\x61\xCA\xF6\x5F\x0E\x40\xDF\xC5\x20\x20\x3B\xB5\x4D\x29\xE0\xA8\xF7\x8F\x16\xB3\xF1\xAA\x52\x5D\x6B\xFA\x33\xC5\x47\x26\xE5\x99\x88\xCF\xBE\xC7\x80\x56" - , "\x02\xFE\x84\xCE\x81\xE1\x78\xE7\xAA\xBD\xD3\xBA\x92\x5A\x76\x6C\x3C\x24\x75\x6E\xEF\xAE\x33\x94\x2A\xF7\x5E\x8B\x46\x45\x56\xB5\x99\x7E\x61\x6F\x3F\x2D\xFC\x7F\xCE\x91\x84\x8A\xFD\x79\x91\x2D\x9F\xB5\x52\x01\xB5\x81\x3A\x5A\x07\x4D\x2C\x0D\x42\x92\xC1\xFD\x44\x18\x07\xC5" - ) - , - ( 20 - , "\x05\x3A\x02\xBE\xDD\x63\x68\xC1\xFB\x8A\xFC\x7A\x1B\x19\x9F\x7F\x7E\xA2\x22\x0C\x9A\x4B\x64\x2A\x68\x50\x09\x1C\x9D\x20\xAB\x9C" - , "\xC7\x13\xEE\xA5\xC2\x6D\xAD\x75\xAD\x3F\x52\x45\x1E\x00\x3A\x9C\xB0\xD6\x49\xF9\x17\xC8\x9D\xDE" - , "\x8F\x0A\x8A\x16\x47\x60\x42\x65\x67\xE3\x88\x84\x02\x76\xDE\x3F\x95\xCB\x5E\x3F\xAD\xC6\xED\x3F\x3E\x4F\xE8\xBC\x16\x9D\x93\x88\x80\x4D\xCB\x94\xB6\x58\x7D\xBB\x66\xCB\x0B\xD5\xF8\x7B\x8E\x98\xB5\x2A\xF3\x7B\xA2\x90\x62\x9B\x85\x8E\x0E\x2A\xA7\x37\x80\x47\xA2\x66\x02" - , "\x51\x67\x10\xE5\x98\x43\xE6\xFB\xD4\xF2\x5D\x0D\x8C\xA0\xEC\x0D\x47\xD3\x9D\x12\x5E\x9D\xAD\x98\x7E\x05\x18\xD4\x91\x07\x01\x4C\xB0\xAE\x40\x5E\x30\xC2\xEB\x37\x94\x75\x0B\xCA\x14\x2C\xE9\x5E\x29\x0C\xF9\x5A\xBE\x15\xE8\x22\x82\x3E\x2E\x7D\x3A\xB2\x1B\xC8\xFB\xD4\x45" - ) - , - ( 20 - , "\x5B\x14\xAB\x0F\xBE\xD4\xC5\x89\x52\x54\x8A\x6C\xB1\xE0\x00\x0C\xF4\x48\x14\x21\xF4\x12\x88\xEA\x0A\xA8\x4A\xDD\x9F\x7D\xEB\x96" - , "\x54\xBF\x52\xB9\x11\x23\x1B\x95\x2B\xA1\xA6\xAF\x8E\x45\xB1\xC5\xA2\x9D\x97\xE2\xAB\xAD\x7C\x83" - , "\x37\xFB\x44\xA6\x75\x97\x8B\x56\x0F\xF9\xA4\xA8\x70\x11\xD6\xF3\xAD\x2D\x37\xA2\xC3\x81\x5B\x45\xA3\xC0\xE6\xD1\xB1\xD8\xB1\x78\x4C\xD4\x68\x92\x7C\x2E\xE3\x9E\x1D\xCC\xD4\x76\x5E\x1C\x3D\x67\x6A\x33\x5B\xE1\xCC\xD6\x90\x0A\x45\xF5\xD4\x1A\x31\x76\x48\x31\x5D\x8A\x8C\x24\xAD\xC6\x4E\xB2\x85\xF6\xAE\xBA\x05\xB9\x02\x95\x86\x35\x3D\x30\x3F\x17\xA8\x07\x65\x8B\x9F\xF7\x90\x47\x4E\x17\x37\xBD\x5F\xDC\x60\x4A\xEF\xF8\xDF\xCA\xF1\x42\x7D\xCC\x3A\xAC\xBB\x02\x56\xBA\xDC\xD1\x83\xED\x75\xA2\xDC\x52\x45\x2F\x87\xD3\xC1\xED\x2A\xA5\x83\x47\x2B\x0A\xB9\x1C\xDA\x20\x61\x4E\x9B\x6F\xDB\xDA\x3B\x49\xB0\x98\xC9\x58\x23\xCC\x72\xD8\xE5\xB7\x17\xF2\x31\x4B\x03\x24\xE9\xCE" - , "\xAE\x6D\xEB\x5D\x6C\xE4\x3D\x4B\x09\xD0\xE6\xB1\xC0\xE9\xF4\x61\x57\xBC\xD8\xAB\x50\xEA\xA3\x19\x7F\xF9\xFA\x2B\xF7\xAF\x64\x9E\xB5\x2C\x68\x54\x4F\xD3\xAD\xFE\x6B\x1E\xB3\x16\xF1\xF2\x35\x38\xD4\x70\xC3\x0D\xBF\xEC\x7E\x57\xB6\x0C\xBC\xD0\x96\xC7\x82\xE7\x73\x6B\x66\x91\x99\xC8\x25\x3E\x70\x21\x4C\xF2\xA0\x98\xFD\xA8\xEA\xC5\xDA\x79\xA9\x49\x6A\x3A\xAE\x75\x4D\x03\xB1\x7C\x6D\x70\xD1\x02\x7F\x42\xBF\x7F\x95\xCE\x3D\x1D\x9C\x33\x88\x54\xE1\x58\xFC\xC8\x03\xE4\xD6\x26\x2F\xB6\x39\x52\x1E\x47\x11\x6E\xF7\x8A\x7A\x43\x7C\xA9\x42\x7B\xA6\x45\xCD\x64\x68\x32\xFE\xAB\x82\x2A\x20\x82\x78\xE4\x5E\x93\xE1\x18\xD7\x80\xB9\x88\xD6\x53\x97\xED\xDF\xD7\xA8\x19\x52\x6E" - ) - , - ( 20 - , "\xD7\x46\x36\xE3\x41\x3A\x88\xD8\x5F\x32\x2C\xA8\x0F\xB0\xBD\x65\x0B\xD0\xBF\x01\x34\xE2\x32\x91\x60\xB6\x96\x09\xCD\x58\xA4\xB0" - , "\xEF\xB6\x06\xAA\x1D\x9D\x9F\x0F\x46\x5E\xAA\x7F\x81\x65\xF1\xAC\x09\xF5\xCB\x46\xFE\xCF\x2A\x57" - , "\xF8\x54\x71\xB7\x5F\x6E\xC8\x1A\xBA\xC2\x79\x9E\xC0\x9E\x98\xE2\x80\xB2\xFF\xD6\x4C\xA2\x85\xE5\xA0\x10\x9C\xFB\x31\xFF\xAB\x2D\x61\x7B\x2C\x29\x52\xA2\xA8\xA7\x88\xFC\x0D\xA2\xAF\x7F\x53\x07\x58\xF7\x4F\x1A\xB5\x63\x91\xAB\x5F\xF2\xAD\xBC\xC5\xBE\x2D\x6C\x7F\x49\xFB\xE8\x11\x81\x04\xC6\xFF\x9A\x23\xC6\xDF\xE5\x2F\x57\x95\x4E\x6A\x69\xDC\xEE\x5D\xB0\x6F\x51\x4F\x4A\x0A\x57\x2A\x9A\x85\x25\xD9\x61\xDA\xE7\x22\x69\xB9\x87\x18\x9D\x46\x5D\xF6\x10\x71\x19\xC7\xFA\x79\x08\x53\xE0\x63\xCB\xA0\xFA\xB7\x80\x0C\xA9\x32\xE2\x58\x88\x0F\xD7\x4C\x33\xC7\x84\x67\x5B\xED\xAD\x0E\x7C\x09\xE9\xCC\x4D\x63\xDD\x5E\x97\x13\xD5\xD4\xA0\x19\x6E\x6B\x56\x22\x26\xAC\x31\xB4\xF5\x7C\x04\xF9\x0A\x18\x19\x73\x73\x7D\xDC\x7E\x80\xF3\x64\x11\x2A\x9F\xBB\x43\x5E\xBD\xBC\xAB\xF7\xD4\x90\xCE\x52" - , "\xB2\xB7\x95\xFE\x6C\x1D\x4C\x83\xC1\x32\x7E\x01\x5A\x67\xD4\x46\x5F\xD8\xE3\x28\x13\x57\x5C\xBA\xB2\x63\xE2\x0E\xF0\x58\x64\xD2\xDC\x17\xE0\xE4\xEB\x81\x43\x6A\xDF\xE9\xF6\x38\xDC\xC1\xC8\xD7\x8F\x6B\x03\x06\xBA\xF9\x38\xE5\xD2\xAB\x0B\x3E\x05\xE7\x35\xCC\x6F\xFF\x2D\x6E\x02\xE3\xD6\x04\x84\xBE\xA7\xC7\xA8\xE1\x3E\x23\x19\x7F\xEA\x7B\x04\xD4\x7D\x48\xF4\xA4\xE5\x94\x41\x74\x53\x94\x92\x80\x0D\x3E\xF5\x1E\x2E\xE5\xE4\xC8\xA0\xBD\xF0\x50\xC2\xDD\x3D\xD7\x4F\xCE\x5E\x7E\x5C\x37\x36\x4F\x75\x47\xA1\x14\x80\xA3\x06\x3B\x9A\x0A\x15\x7B\x15\xB1\x0A\x5A\x95\x4D\xE2\x73\x1C\xED\x05\x5A\xA2\xE2\x76\x7F\x08\x91\xD4\x32\x9C\x42\x6F\x38\x08\xEE\x86\x7B\xED\x0D\xC7\x5B\x59\x22\xB7\xCF\xB8\x95\x70\x0F\xDA\x01\x61\x05\xA4\xC7\xB7\xF0\xBB\x90\xF0\x29\xF6\xBB\xCB\x04\xAC\x36\xAC\x16" - ) - ] - --- Test vector from paper "Cryptography in NaCl" -vectorsCB :: [Vector] -vectorsCB = - [ - ( 20 - , "\x4A\x5D\x9D\x5B\xA4\xCE\x2D\xE1\x72\x8E\x3B\xF4\x80\x35\x0F\x25\xE0\x7E\x21\xC9\x47\xD1\x9E\x33\x76\xF0\x9B\x3C\x1E\x16\x17\x42" - , "\x69\x69\x6E\xE9\x55\xB6\x2B\x73\xCD\x62\xBD\xA8\x75\xFC\x73\xD6\x82\x19\xE0\x03\x6B\x7A\x0B\x37" - , "\xBE\x07\x5F\xC5\x3C\x81\xF2\xD5\xCF\x14\x13\x16\xEB\xEB\x0C\x7B\x52\x28\xC5\x2A\x4C\x62\xCB\xD4\x4B\x66\x84\x9B\x64\x24\x4F\xFC\xE5\xEC\xBA\xAF\x33\xBD\x75\x1A\x1A\xC7\x28\xD4\x5E\x6C\x61\x29\x6C\xDC\x3C\x01\x23\x35\x61\xF4\x1D\xB6\x6C\xCE\x31\x4A\xDB\x31\x0E\x3B\xE8\x25\x0C\x46\xF0\x6D\xCE\xEA\x3A\x7F\xA1\x34\x80\x57\xE2\xF6\x55\x6A\xD6\xB1\x31\x8A\x02\x4A\x83\x8F\x21\xAF\x1F\xDE\x04\x89\x77\xEB\x48\xF5\x9F\xFD\x49\x24\xCA\x1C\x60\x90\x2E\x52\xF0\xA0\x89\xBC\x76\x89\x70\x40\xE0\x82\xF9\x37\x76\x38\x48\x64\x5E\x07\x05" - , "\x8E\x99\x3B\x9F\x48\x68\x12\x73\xC2\x96\x50\xBA\x32\xFC\x76\xCE\x48\x33\x2E\xA7\x16\x4D\x96\xA4\x47\x6F\xB8\xC5\x31\xA1\x18\x6A\xC0\xDF\xC1\x7C\x98\xDC\xE8\x7B\x4D\xA7\xF0\x11\xEC\x48\xC9\x72\x71\xD2\xC2\x0F\x9B\x92\x8F\xE2\x27\x0D\x6F\xB8\x63\xD5\x17\x38\xB4\x8E\xEE\xE3\x14\xA7\xCC\x8A\xB9\x32\x16\x45\x48\xE5\x26\xAE\x90\x22\x43\x68\x51\x7A\xCF\xEA\xBD\x6B\xB3\x73\x2B\xC0\xE9\xDA\x99\x83\x2B\x61\xCA\x01\xB6\xDE\x56\x24\x4A\x9E\x88\xD5\xF9\xB3\x79\x73\xF6\x22\xA4\x3D\x14\xA6\x59\x9B\x1F\x65\x4C\xB4\x5A\x74\xE3\x55\xA5" - ) - ] - -tests = - testGroup - "XSalsa" - [ testGroup "KAT" $ - zipWith - (\i (r, k, n, p, e) -> testCase (show (i :: Int)) $ salsaRunSimple r k n p e) - [1 ..] - vectors - , testGroup "crypto_box encryption" $ - zipWith - (\i (r, k, n, p, e) -> testCase (show (i :: Int)) $ cryptoBoxEnc r k n p e) - [1 ..] - vectorsCB - ] - where - salsaRunSimple rounds key nonce plain expected = - let salsa = XSalsa.initialize rounds key nonce - in fst (XSalsa.combine salsa plain) @?= expected - - cryptoBoxEnc rounds shared nonce plain expected = - let zero = B.replicate 16 0 - (iv0, iv1) = B.splitAt 8 nonce - salsa0 = XSalsa.initialize rounds shared (zero `B.append` iv0) - salsa1 = XSalsa.derive salsa0 iv1 - (_, salsa2) = XSalsa.generate salsa1 32 :: (B.ByteString, XSalsa.State) - in fst (XSalsa.combine salsa2 plain) @?= expected